{"id":167968,"date":"2026-11-18T11:02:10","date_gmt":"2026-11-18T19:02:10","guid":{"rendered":"https:\/\/xira.com\/p\/2026\/11\/18\/in-house-and-ai-you-cant-demand-what-you-dont-understand\/"},"modified":"2026-11-18T11:02:10","modified_gmt":"2026-11-18T19:02:10","slug":"in-house-and-ai-you-cant-demand-what-you-dont-understand","status":"publish","type":"post","link":"https:\/\/xira.com\/p\/2026\/11\/18\/in-house-and-ai-you-cant-demand-what-you-dont-understand\/","title":{"rendered":"In-House And AI: You Can\u2019t Demand What You Don\u2019t Understand"},"content":{"rendered":"<figure class=\"wp-block-image alignright\"><img decoding=\"async\" loading=\"lazy\" width=\"1280\" height=\"853\" src=\"https:\/\/abovethelaw.com\/wp-content\/uploads\/sites\/4\/2024\/02\/artificial-intelligence-4389372_1280.jpg\" alt=\"\" class=\"wp-image-85986\" title=\"\"><figcaption><\/figcaption><\/figure>\n<p class=\"wp-block-paragraph\">It\u2019s a mystery. Most in-house counsel want their outside firms to aggressively use AI, and most expect their bills to be reduced. Yet, they rarely demand these things from their outside firms. It\u2019s a counterintuitive gap. What gives?<\/p>\n<p class=\"wp-block-paragraph\">A recent <a href=\"https:\/\/www.axiomlaw.com\/resources\/articles\/legal-ai-survey-report\" rel=\"nofollow noopener\" target=\"_blank\">Axiom study<\/a> sheds some light. Put simply, a lot of in-house counsel don\u2019t have any more of a clue how to use AI effectively than do their outside firms. If the people paying the bills don\u2019t understand what AI can realistically accomplish, how can they know what to demand? How can they hold the people sending the bills accountable?<\/p>\n<p class=\"wp-block-paragraph\"><strong>The Study and Its Source<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Now we do need to start with a little healthy skepticism. <a href=\"https:\/\/www.axiomlaw.com\/\" rel=\"nofollow noopener\" target=\"_blank\">Axiom<\/a> is an alternative legal service provider (ALSP). Its primary mission is to supply in-house legal departments with legal talent for specific and often temporary needs. So it\u2019s in Axiom\u2019s interest if in-house legal needs help with AI. That does not necessarily mean the survey results are wrong. I have no evidence of that. But it\u2019s always good to keep in mind who is paying for work like this.<\/p>\n<p class=\"wp-block-paragraph\">Axiom sponsored the study and commissioned <a href=\"https:\/\/www.insightdynamo.com\/?utm_source=chatgpt.com\" rel=\"nofollow noopener\" target=\"_blank\">InsightDynamo<\/a>, which describes itself on its website as a healthcare research firm (Hmm \u2026). InsightDynamo surveyed 528 in-house legal professionals online at companies with annual revenues of at least $100 million. The survey was done between March 13, 2026, and March 25, 2026.<\/p>\n<p class=\"wp-block-paragraph\"><strong>A Lack Of Metrics, Skills, And Use<\/strong><\/p>\n<p class=\"wp-block-paragraph\">The results paint a bleak picture of in-house legal\u2019s understanding and effective use of AI. Of those surveyed, 83% don\u2019t know how to measure AI\u2019s ROI or assess how their department\u2019s AI spending is working out. That means fewer than 20% have established metrics to assess the effectiveness of what they\u2019ve spent.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">Only 7% are optimizing and measuring AI across their organization.\u00a0Think about that: they\u2019re spending lots of money on AI but have no clue if it\u2019s producing meaningful returns.<\/p>\n<p class=\"wp-block-paragraph\">Most respondents assessed their AI skills as mainly intermediate. Only 10% believe that they are advanced. There\u2019s a reason for that: only 25% send their personnel out for external training. Axiom notes that departments are increasing AI spending \u201cwith a workforce that has had at most one working day of training.\u201d Jeez.<\/p>\n<p class=\"wp-block-paragraph\">Only 17% of those surveyed report that a majority of their workers use AI tools daily. According to the report, only one in four use AI monthly. As Axiom puts it, \u201cthe skills gap is not access or adoption; it is training and regular experience using the tools.\u201d Only 40% have implemented AI planning and governance practices. Another 40% say they plan to, which means they haven\u2019t yet. (Or as they say where I grew up, they\u2019re fixin\u2019 to. Right.)<\/p>\n<p class=\"wp-block-paragraph\">The barriers that the respondents identify to determining AI value are also instructive. They include:<\/p>\n<p class=\"wp-block-paragraph\">\u2013 a belief that the workflow changes would be too disruptive<\/p>\n<p class=\"wp-block-paragraph\">\u2013 a feeling that team members will resist change<\/p>\n<p class=\"wp-block-paragraph\">\u2013 the view that skills are lacking<\/p>\n<p class=\"wp-block-paragraph\">\u2013 a sense that ROI is just too hard to measure<\/p>\n<p class=\"wp-block-paragraph\">Sound familiar? It\u2019s the same list you\u2019d probably get from a room full of law firm managing partners.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">Nonetheless, 95% magically expect outside counsel to use AI (and use it effectively), and 92% expect lower rates when firms do use AI, an expectation that is largely unsatisfied. But <a href=\"https:\/\/abovethelaw.com\/2026\/10\/who-will-control-the-ai-relationship-in-house-or-outside-counsel\/\" rel=\"nofollow noopener\" target=\"_blank\">another survey shows<\/a> that 80% of in-house don\u2019t require or even encourage their firms to use AI.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Legal AI? What Legal AI?<\/strong><\/p>\n<p class=\"wp-block-paragraph\">And what AI tools are being used? 67% use general-purpose or frontier models, not legal-specific models. Tools like ChatGPT, Copilot, and Gemini. The same tools most of us use for nonlegal work. As the report puts it, \u201cMost legal teams are running AI the same way a marketing department would.\u201d That suggests most legal departments have no idea what they\u2019re doing with AI.\u00a0<\/p>\n<p class=\"wp-block-paragraph\"><strong>Selection Process? What Selection Process?<\/strong><\/p>\n<p class=\"wp-block-paragraph\">And the selection process for AI tools? In a word: abysmal. Only 1 in 10 teams conduct rigorous evaluation before determining what product to buy. Fewer than 1 in 4 run structured pilots with any sort of defined goals, timelines, or measurement.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">If you don\u2019t know what you want a tool to accomplish before you buy it, it\u2019s pretty hard to determine whether it accomplished anything afterward. It\u2019s no wonder the ROI is elusive.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Guidance? What Guidance?<\/strong><\/p>\n<p class=\"wp-block-paragraph\">And in-house teams are desperate for guidance. Almost all, 98%, say they wish they had outside guidance on tool selections, and 96% say that they would use an ALSP for adoption or implementation advice, not a law firm. This is probably because ALSPs generally have an economic incentive to use AI and are therefore perceived as further along the learning curve. It also reflects a lack of trust in law firms. (Axiom is an ALSP.)<\/p>\n<p class=\"wp-block-paragraph\"><strong>What Does It All Mean?<\/strong><\/p>\n<p class=\"wp-block-paragraph\">It\u2019s not surprising that legal departments aren\u2019t seeing their bills reduced. In-house legal departments don\u2019t really know how to use AI effectively. The numbers clearly show a real lack of knowledge and understanding among in-house legal teams when it comes to AI. They can\u2019t figure out the return on investment, which means they can\u2019t identify where AI would be valuable or how to get there.<\/p>\n<p class=\"wp-block-paragraph\">If they can\u2019t figure out the value, how in the hell can they tell outside counsel how to use the tools in ways that reduce their bills? How can they have a meaningful conversation with outside counsel about where AI should be used to reduce fees without sacrificing quality?<\/p>\n<p class=\"wp-block-paragraph\">Moreover, outside counsel clearly won\u2019t use AI to reduce their bills until clients demand it. First of all, most don\u2019t have any more of a clue about its use than in-house. More importantly, those firms that bill by the hour (as do many of the outside firms that work for these companies) have no economic incentive to use tools and technologies that they fear will ultimately reduce their revenue. As <a href=\"https:\/\/en.wikipedia.org\/wiki\/Richard_Susskind\" rel=\"nofollow noopener\" target=\"_blank\">Richard Susskind<\/a> famously noted, it\u2019s hard to tell a room full of millionaires their business model is all wrong. It\u2019s even harder to convince them.<\/p>\n<p class=\"wp-block-paragraph\">In-house counsel, on the other hand, have the economic incentive to use AI. But they aren\u2019t using it as effectively as they should. They don\u2019t know which product to use, nor do they know how to conduct a robust evaluation to decide what to use. And just like their outside brethren, they view AI use as disruptive and resist change.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">So they\u2019re not much different from the outside firms they now expect to use AI despite those firms\u2019 disincentive to do so. Perhaps that expectation is not unreasonable, since outside firms are in the business of serving their clients. So it would be in their clients\u2019 best interests for them to use AI. That may be true for some firms, but that\u2019s simply not reality for most.<\/p>\n<p class=\"wp-block-paragraph\"><strong>The Blind Leading The Blind<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So here we sit in 2026. You have law firms reluctant to use AI and fearful of what it could mean for their economics. You have in-house legal departments that don\u2019t understand AI\u2019s value or how to use it effectively. That creates a stubborn gap between what in-house counsel want and what outside firms are doing.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">But unless and until in-house legal departments better understand how to use AI and what can be realistically demanded of outside firms, that gap will remain. You can\u2019t demand what you don\u2019t understand.<\/p>\n<p class=\"wp-block-paragraph\">Want your outside firms to use AI effectively? Learn how to use it in-house and then demand that they do the same.\u00a0<\/p>\n<hr class=\"wp-block-separator has-alpha-channel-opacity\">\n<p class=\"wp-block-paragraph\"><strong><em>Stephen Embry is a lawyer, speaker, blogger, and writer. He publishes\u00a0<\/em><\/strong><a href=\"https:\/\/www.techlawcrossroads.com\/\" target=\"_blank\" rel=\"noopener nofollow\"><strong><em>TechLaw Crossroads<\/em><\/strong><\/a><strong><em>, a blog devoted to the examination of the tension between technology, the law, and the practice of law<\/em><\/strong>.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\n<p>The post <a href=\"https:\/\/abovethelaw.com\/2026\/11\/in-house-and-ai-you-cant-demand-what-you-dont-understand\/\" rel=\"nofollow noopener\" target=\"_blank\">In-House And AI: You Can\u2019t Demand What You Don\u2019t Understand<\/a> appeared first on <a href=\"https:\/\/abovethelaw.com\/\" rel=\"nofollow noopener\" target=\"_blank\">Above the Law<\/a>.<\/p>\n<figure><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"853\" src=\"https:\/\/abovethelaw.com\/wp-content\/uploads\/sites\/4\/2024\/02\/artificial-intelligence-4389372_1280.jpg\" alt=\"\" title=\"\"><\/figure>\n<p>It\u2019s a mystery. Most in-house counsel want their outside firms to aggressively use AI, and most expect their bills to be reduced. Yet, they rarely demand these things from their outside firms. It\u2019s a counterintuitive gap. What gives?<\/p>\n<p>A recent <a href=\"https:\/\/www.axiomlaw.com\/resources\/articles\/legal-ai-survey-report\" rel=\"nofollow noopener\" target=\"_blank\">Axiom study<\/a> sheds some light. Put simply, a lot of in-house counsel don\u2019t have any more of a clue how to use AI effectively than do their outside firms. If the people paying the bills don\u2019t understand what AI can realistically accomplish, how can they know what to demand? How can they hold the people sending the bills accountable?<\/p>\n<p><strong>The Study and Its Source<\/strong><\/p>\n<p>Now we do need to start with a little healthy skepticism. <a href=\"https:\/\/www.axiomlaw.com\/\" rel=\"nofollow noopener\" target=\"_blank\">Axiom<\/a> is an alternative legal service provider (ALSP). Its primary mission is to supply in-house legal departments with legal talent for specific and often temporary needs. So it\u2019s in Axiom\u2019s interest if in-house legal needs help with AI. That does not necessarily mean the survey results are wrong. I have no evidence of that. But it\u2019s always good to keep in mind who is paying for work like this.<\/p>\n<p>Axiom sponsored the study and commissioned <a href=\"https:\/\/www.insightdynamo.com\/?utm_source=chatgpt.com\" rel=\"nofollow noopener\" target=\"_blank\">InsightDynamo<\/a>, which describes itself on its website as a healthcare research firm (Hmm \u2026). InsightDynamo surveyed 528 in-house legal professionals online at companies with annual revenues of at least $100 million. The survey was done between March 13, 2026, and March 25, 2026.<\/p>\n<p><strong>A Lack Of Metrics, Skills, And Use<\/strong><\/p>\n<p>The results paint a bleak picture of in-house legal\u2019s understanding and effective use of AI. Of those surveyed, 83% don\u2019t know how to measure AI\u2019s ROI or assess how their department\u2019s AI spending is working out. That means fewer than 20% have established metrics to assess the effectiveness of what they\u2019ve spent.\u00a0<\/p>\n<p>Only 7% are optimizing and measuring AI across their organization.\u00a0Think about that: they\u2019re spending lots of money on AI but have no clue if it\u2019s producing meaningful returns.<\/p>\n<p>Most respondents assessed their AI skills as mainly intermediate. Only 10% believe that they are advanced. There\u2019s a reason for that: only 25% send their personnel out for external training. Axiom notes that departments are increasing AI spending \u201cwith a workforce that has had at most one working day of training.\u201d Jeez.<\/p>\n<p>Only 17% of those surveyed report that a majority of their workers use AI tools daily. According to the report, only one in four use AI monthly. As Axiom puts it, \u201cthe skills gap is not access or adoption; it is training and regular experience using the tools.\u201d Only 40% have implemented AI planning and governance practices. Another 40% say they plan to, which means they haven\u2019t yet. (Or as they say where I grew up, they\u2019re fixin\u2019 to. Right.)<\/p>\n<p>The barriers that the respondents identify to determining AI value are also instructive. They include:<\/p>\n<p>\u2013 a belief that the workflow changes would be too disruptive<\/p>\n<p>\u2013 a feeling that team members will resist change<\/p>\n<p>\u2013 the view that skills are lacking<\/p>\n<p>\u2013 a sense that ROI is just too hard to measure<\/p>\n<p>Sound familiar? It\u2019s the same list you\u2019d probably get from a room full of law firm managing partners.\u00a0<\/p>\n<p>Nonetheless, 95% magically expect outside counsel to use AI (and use it effectively), and 92% expect lower rates when firms do use AI, an expectation that is largely unsatisfied. But <a href=\"https:\/\/abovethelaw.com\/2026\/10\/who-will-control-the-ai-relationship-in-house-or-outside-counsel\/\" rel=\"nofollow noopener\" target=\"_blank\">another survey shows<\/a> that 80% of in-house don\u2019t require or even encourage their firms to use AI.<\/p>\n<p><strong>Legal AI? What Legal AI?<\/strong><\/p>\n<p>And what AI tools are being used? 67% use general-purpose or frontier models, not legal-specific models. Tools like ChatGPT, Copilot, and Gemini. The same tools most of us use for nonlegal work. As the report puts it, \u201cMost legal teams are running AI the same way a marketing department would.\u201d That suggests most legal departments have no idea what they\u2019re doing with AI.\u00a0<\/p>\n<p><strong>Selection Process? What Selection Process?<\/strong><\/p>\n<p>And the selection process for AI tools? In a word: abysmal. Only 1 in 10 teams conduct rigorous evaluation before determining what product to buy. Fewer than 1 in 4 run structured pilots with any sort of defined goals, timelines, or measurement.\u00a0<\/p>\n<p>If you don\u2019t know what you want a tool to accomplish before you buy it, it\u2019s pretty hard to determine whether it accomplished anything afterward. It\u2019s no wonder the ROI is elusive.<\/p>\n<p><strong>Guidance? What Guidance?<\/strong><\/p>\n<p>And in-house teams are desperate for guidance. Almost all, 98%, say they wish they had outside guidance on tool selections, and 96% say that they would use an ALSP for adoption or implementation advice, not a law firm. This is probably because ALSPs generally have an economic incentive to use AI and are therefore perceived as further along the learning curve. It also reflects a lack of trust in law firms. (Axiom is an ALSP.)<\/p>\n<p><strong>What Does It All Mean?<\/strong><\/p>\n<p>It\u2019s not surprising that legal departments aren\u2019t seeing their bills reduced. In-house legal departments don\u2019t really know how to use AI effectively. The numbers clearly show a real lack of knowledge and understanding among in-house legal teams when it comes to AI. They can\u2019t figure out the return on investment, which means they can\u2019t identify where AI would be valuable or how to get there.<\/p>\n<p>If they can\u2019t figure out the value, how in the hell can they tell outside counsel how to use the tools in ways that reduce their bills? How can they have a meaningful conversation with outside counsel about where AI should be used to reduce fees without sacrificing quality?<\/p>\n<p>Moreover, outside counsel clearly won\u2019t use AI to reduce their bills until clients demand it. First of all, most don\u2019t have any more of a clue about its use than in-house. More importantly, those firms that bill by the hour (as do many of the outside firms that work for these companies) have no economic incentive to use tools and technologies that they fear will ultimately reduce their revenue. As <a href=\"https:\/\/en.wikipedia.org\/wiki\/Richard_Susskind\" rel=\"nofollow noopener\" target=\"_blank\">Richard Susskind<\/a> famously noted, it\u2019s hard to tell a room full of millionaires their business model is all wrong. It\u2019s even harder to convince them.<\/p>\n<p>In-house counsel, on the other hand, have the economic incentive to use AI. But they aren\u2019t using it as effectively as they should. They don\u2019t know which product to use, nor do they know how to conduct a robust evaluation to decide what to use. And just like their outside brethren, they view AI use as disruptive and resist change.\u00a0<\/p>\n<p>So they\u2019re not much different from the outside firms they now expect to use AI despite those firms\u2019 disincentive to do so. Perhaps that expectation is not unreasonable, since outside firms are in the business of serving their clients. So it would be in their clients\u2019 best interests for them to use AI. That may be true for some firms, but that\u2019s simply not reality for most.<\/p>\n<p><strong>The Blind Leading The Blind<\/strong><\/p>\n<p>So here we sit in 2026. You have law firms reluctant to use AI and fearful of what it could mean for their economics. You have in-house legal departments that don\u2019t understand AI\u2019s value or how to use it effectively. That creates a stubborn gap between what in-house counsel want and what outside firms are doing.\u00a0<\/p>\n<p>But unless and until in-house legal departments better understand how to use AI and what can be realistically demanded of outside firms, that gap will remain. You can\u2019t demand what you don\u2019t understand.<\/p>\n<p>Want your outside firms to use AI effectively? Learn how to use it in-house and then demand that they do the same.\u00a0<\/p>\n<hr \/>\n<p><strong><em>Stephen Embry is a lawyer, speaker, blogger, and writer. He publishes\u00a0<\/em><\/strong><a href=\"https:\/\/www.techlawcrossroads.com\/\" target=\"_blank\" rel=\"noopener nofollow\"><strong><em>TechLaw Crossroads<\/em><\/strong><\/a><strong><em>, a blog devoted to the examination of the tension between technology, the law, and the practice of law<\/em><\/strong>.\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p>It\u2019s a mystery. Most in-house counsel want their outside firms to aggressively use AI, and most expect their bills to be reduced. Yet, they rarely demand these things from their outside firms. It\u2019s a counterintuitive gap. What gives? A recent Axiom study sheds some light. Put simply, a lot of in-house counsel don\u2019t have any [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":167969,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-167968","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-above_the_law"],"_links":{"self":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/167968","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/comments?post=167968"}],"version-history":[{"count":0,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/167968\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media\/167969"}],"wp:attachment":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media?parent=167968"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/categories?post=167968"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/tags?post=167968"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}