{"id":161471,"date":"2026-08-25T13:50:40","date_gmt":"2026-08-25T21:50:40","guid":{"rendered":"https:\/\/xira.com\/p\/2026\/08\/25\/netdocuments-asks-how-much-are-lawyers-paying-to-get-the-right-answer-from-ai\/"},"modified":"2026-08-25T13:50:40","modified_gmt":"2026-08-25T21:50:40","slug":"netdocuments-asks-how-much-are-lawyers-paying-to-get-the-right-answer-from-ai","status":"publish","type":"post","link":"https:\/\/xira.com\/p\/2026\/08\/25\/netdocuments-asks-how-much-are-lawyers-paying-to-get-the-right-answer-from-ai\/","title":{"rendered":"NetDocuments Asks How Much Are Lawyers Paying To Get The \u2018Right\u2019 Answer From AI"},"content":{"rendered":"

As Lyle Lanley might say, a law firm with AI is a little like the mule with a spinning wheel \u2014 no one knows how he got it and danged if he knows how to use it. Firms are falling all over themselves to be able to tell the world they have cutting edge AI capabilities, but ask a lawyer why its AI assistant runs on the most expensive frontier model on the menu and the answer lands somewhere between \u201cit\u2019s the best one\u201d and a shrug. Ask what it should actually cost<\/em> to get the right answer and you get the shrug without the preamble.<\/p>\n

That\u2019s a survivable state of affairs when everything gets priced out as a flat-rate seat license and the compute bill becomes somebody else\u2019s problem. That won\u2019t cut it forever and when consumption pricing finally takes over, every question a lawyer fires at a chatbot generates a line item, and nobody in the building can tell you whether that line item is a penny or 70 cents. As NetDocuments<\/a> CEO Josh Baxter put it in announcing<\/a> the company\u2019s new benchmark product, the industry is currently oscillating between \u201cspending without limits and cutting without strategy.\u201d<\/p>\n

NetDocuments published the Legal Context Engineering Benchmark<\/a> ahead of ILTACON, and the proposition is pretty straigthforward. Three things determine whether a legal AI agent works: the model, the harness wrapped around it, and the context it can reach while it works. Existing benchmarks offer an incomplete picture. The goal of the NetDocuments product is to freeze the model and harness, run the same 300 questions across 10 real matters twice \u2014 once with only search and retrieval, once with the Legal Context Graph<\/a> switched on \u2014 and measure what moves.<\/p>\n

What this reveals is the key detail other benchmarks miss:<\/p>\n

\n

Accuracy alone would be the wrong scorecard, because accuracy can nearly always be bought with more spending: a bigger model, more reasoning, more retrieval.<\/p>\n<\/blockquote>\n

NetDocuments, as a document management system, is focused on the role context plays. Their point with this exercise isn\u2019t to prove that users can spend less without forfeiting accuracy, but that excellent, well-managed context makes AI run more efficiently \u2014 which can either be used to save money or reinvested into chasing more accuracy.<\/p>\n

But along the way, the results lent support to the idea that while the market obsesses over models achieving accuracy-above-all, as a practical matter, if \u201chuman-in-the-loop\u201d means anything at all, it means someone should be there to intervene to address a couple percent accuracy dip when it saves the user hundreds of thousands of dollars. <\/p>\n

Trading off accuracy gets some people around these legal tech conversations skittish. But think of it this way \u2014 assigning some tasks to a first-year rather than a mid-level trades accuracy too, but we do it because we know it\u2019s cheaper and the sacrifice is something we can easily address on the back end. Why isn\u2019t that how we think about AI?<\/p>\n

The headline result is that the context layer cuts it roughly in half. They call the ratio the Context Value Ratio and it\u2019s what shows how much the firm can accomplish with more robust context. According to the company, using their process to enrich context and prevent models from casually wandering around millions of tokens of unfocused context, token consumption per answer fell 52 percent.<\/p>\n

NetDocuments ran the whole benchmark on three tiers of the same model family. On the economy tier, answering all 300 questions cost $4.35 and produced 191.9 fully accurate answers. On the frontier tier, the same 300 questions cost $151.10 and produced 221.8 correct answers. Thirty-five times the spend to get 30 more right answers out of three hundred.<\/p>\n

The economy model availed of the NetDocuments context graph gets 195.3 questions right for $2.83. Comparing it to the 221.8 answers of the frontier model with the benefit of the context graph, that means 26 additional correct answers cost about $148 \u2014 roughly $5.60 apiece, against an average of a penny and a half.<\/p>\n

On bet-the-company questions $5.60 is worth it. But most legal queries do not carry final bet-the-company stakes and wasting $5.60 on them is a decision<\/em> that nobody is actually aware enough to be making. Firms are throwing money at the frontier tier the way they buy Herman Miller chairs, and the report all but says so, noting that the gains from context are front-loaded across tiers and that the mid tier \u201cis where most production work will actually run.\u201d<\/p>\n

At a press briefing, NetDocuments projecting a firm could save around $1 million a year in token savings. That estimate rested on a 2,000-lawyer firm asking about four million questions a year, which works out to eight AI queries per professional per working day. <\/p>\n

That seems reasonable \u2014 and it might even undercount usage once the agents takeover and start churning. <\/p>\n

\n

\"Headshot\"Joe Patrice<\/a>\u00a0is a senior editor at Above the Law and co-host of Thinking Like A Lawyer<\/a>. Feel free to\u00a0email<\/a> any tips, questions, or comments. Follow him on\u00a0Twitter<\/a>\u00a0or Bluesky<\/a> if you\u2019re interested in law, politics, and a healthy dose of college sports news.<\/em><\/strong><\/p>\n

The post NetDocuments Asks How Much Are Lawyers Paying To Get The \u2018Right\u2019 Answer From AI<\/a> appeared first on Above the Law<\/a>.<\/p>\n

\"\"<\/figure>\n

As Lyle Lanley might say, a law firm with AI is a little like the mule with a spinning wheel \u2014 no one knows how he got it and danged if he knows how to use it. Firms are falling all over themselves to be able to tell the world they have cutting edge AI capabilities, but ask a lawyer why its AI assistant runs on the most expensive frontier model on the menu and the answer lands somewhere between \u201cit\u2019s the best one\u201d and a shrug. Ask what it should actually cost<\/em> to get the right answer and you get the shrug without the preamble.<\/p>\n

That\u2019s a survivable state of affairs when everything gets priced out as a flat-rate seat license and the compute bill becomes somebody else\u2019s problem. That won\u2019t cut it forever and when consumption pricing finally takes over, every question a lawyer fires at a chatbot generates a line item, and nobody in the building can tell you whether that line item is a penny or 70 cents. As NetDocuments<\/a> CEO Josh Baxter put it in announcing<\/a> the company\u2019s new benchmark product, the industry is currently oscillating between \u201cspending without limits and cutting without strategy.\u201d<\/p>\n

NetDocuments published the Legal Context Engineering Benchmark<\/a> ahead of ILTACON, and the proposition is pretty straigthforward. Three things determine whether a legal AI agent works: the model, the harness wrapped around it, and the context it can reach while it works. Existing benchmarks offer an incomplete picture. The goal of the NetDocuments product is to freeze the model and harness, run the same 300 questions across 10 real matters twice \u2014 once with only search and retrieval, once with the Legal Context Graph<\/a> switched on \u2014 and measure what moves.<\/p>\n

What this reveals is the key detail other benchmarks miss:<\/p>\n

\n

Accuracy alone would be the wrong scorecard, because accuracy can nearly always be bought with more spending: a bigger model, more reasoning, more retrieval.<\/p>\n<\/blockquote>\n

NetDocuments, as a document management system, is focused on the role context plays. Their point with this exercise isn\u2019t to prove that users can spend less without forfeiting accuracy, but that excellent, well-managed context makes AI run more efficiently \u2014 which can either be used to save money or reinvested into chasing more accuracy.<\/p>\n

But along the way, the results lent support to the idea that while the market obsesses over models achieving accuracy-above-all, as a practical matter, if \u201chuman-in-the-loop\u201d means anything at all, it means someone should be there to intervene to address a couple percent accuracy dip when it saves the user hundreds of thousands of dollars. <\/p>\n

Trading off accuracy gets some people around these legal tech conversations skittish. But think of it this way \u2014 assigning some tasks to a first-year rather than a mid-level trades accuracy too, but we do it because we know it\u2019s cheaper and the sacrifice is something we can easily address on the back end. Why isn\u2019t that how we think about AI?<\/p>\n

The headline result is that the context layer cuts it roughly in half. They call the ratio the Context Value Ratio and it\u2019s what shows how much the firm can accomplish with more robust context. According to the company, using their process to enrich context and prevent models from casually wandering around millions of tokens of unfocused context, token consumption per answer fell 52 percent.<\/p>\n

NetDocuments ran the whole benchmark on three tiers of the same model family. On the economy tier, answering all 300 questions cost $4.35 and produced 191.9 fully accurate answers. On the frontier tier, the same 300 questions cost $151.10 and produced 221.8 correct answers. Thirty-five times the spend to get 30 more right answers out of three hundred.<\/p>\n

The economy model availed of the NetDocuments context graph gets 195.3 questions right for $2.83. Comparing it to the 221.8 answers of the frontier model with the benefit of the context graph, that means 26 additional correct answers cost about $148 \u2014 roughly $5.60 apiece, against an average of a penny and a half.<\/p>\n

On bet-the-company questions $5.60 is worth it. But most legal queries do not carry final bet-the-company stakes and wasting $5.60 on them is a decision<\/em> that nobody is actually aware enough to be making. Firms are throwing money at the frontier tier the way they buy Herman Miller chairs, and the report all but says so, noting that the gains from context are front-loaded across tiers and that the mid tier \u201cis where most production work will actually run.\u201d<\/p>\n

At a press briefing, NetDocuments projecting a firm could save around $1 million a year in token savings. That estimate rested on a 2,000-lawyer firm asking about four million questions a year, which works out to eight AI queries per professional per working day. <\/p>\n

That seems reasonable \u2014 and it might even undercount usage once the agents takeover and start churning. <\/p>\n

\n

\"Headshot\"Joe Patrice<\/a>\u00a0is a senior editor at Above the Law and co-host of Thinking Like A Lawyer<\/a>. Feel free to\u00a0email<\/a> any tips, questions, or comments. Follow him on\u00a0Twitter<\/a>\u00a0or Bluesky<\/a> if you\u2019re interested in law, politics, and a healthy dose of college sports news.<\/em><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

As Lyle Lanley might say, a law firm with AI is a little like the mule with a spinning wheel \u2014 no one knows how he got it and danged if he knows how to use it. Firms are falling all over themselves to be able to tell the world they have cutting edge AI […]<\/p>\n","protected":false},"author":3,"featured_media":161458,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[16],"tags":[],"class_list":["post-161471","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-above_the_law"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/xira.com\/p\/wp-content\/uploads\/2026\/08\/Headshot-300x200-Odl0mQ.jpg?fit=300%2C200&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/161471","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/comments?post=161471"}],"version-history":[{"count":0,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/161471\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media\/161458"}],"wp:attachment":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media?parent=161471"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/categories?post=161471"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/tags?post=161471"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}