Ed. note: This article first appeared in ILTA’s Peer-to-Peer Magazine.
Breaking news: business and society, including the legal industry, have generally decided AI may have some benefit that is worth leveraging. Many of us have grown markedly more comfortable with using AI in our day-to-day lives and activities. Need help polishing that email you want to send? Check. Want a quick outline to facilitate a strategic discussion? Check. Want to quickly summarize or compare a set of documents? Check. Want to see what you’d look like as an action figure?! Check.
Just make sure you remove any use of the em dash from AI’s output to avoid associated stigma. (Frustrating, on a personal note, as a writer who used to make frequent use of the em dash before AI was doing so!)
The evidence has continued to mount over the last few years. According to 2025 Pew Research Center findings, Americans are interested in AI and want it involved in their daily activities. Stanford’s 2025 AI Index also found that a majority of businesses are using, investing in, and experimenting with AI. Based on Thomson Reuters 2025 Generative AI in Professional Services Report & ILTA’s 2025 Technology Survey, the legal industry is here for it; legal continues to embrace and expand its usage of AI.
However, there’s a catch: comfort and adoption with AI are rising faster than judgment maturity. I’m a big advocate and user of AI, but I also recognize time-tested pearls of wisdom apply here: “Just because you can do something, doesn’t mean you should do that thing.” Or perhaps you prefer “with great power comes great responsibility.”
The Stakes: What If the Machine Is Wrong?
The legal profession and industry are built on core principles of logic and rules. The intentional use of language and reasoning are central to the practice of law, and practitioners carry an important responsibility in serving clients. All attorneys know these tenets well, as they are woven into the fundamentals of legal training. IRAC, which stands for Issue, Rule, Application, Conclusion, is the foundational legal analysis and writing outline taught in law schools, and it demands attention to detail and clear congruence between facts and applicable rules. As service professionals, legal practitioners are also duty-bound to follow the rules of professional conduct and shape their practice in accordance with ethical standards.
Adherence to these duties and principles is imperative for legal. They reflect the exercise of professional responsibility and the very essence of client service. They demand the autonomous exercise of professional judgment. They cannot and should not be delegated to AI.
There is ample evidence that the legal profession and industry are aware of the dangers of not exercising proper professional judgment over the use of AI:
• The oft-cited Mata v. Avianca, Inc., where attorneys were sanctioned for submitting hallucinated cases, emphasized that the duty of verification cannot be delegated. The issue was that professional judgment was not exercised; the core issue wasn’t that AI was used, but rather that AI was not properly supervised.
• Professional liability insurance is evolving, and related guidance cautions against lawyers’ reliance on unverified AI outputs, as outlined in “From innovation to exposure: artificial intelligence risks for legal professionals.” Failure to supervise AI could expose firms to malpractice claims.
• Beyond Mata, courts are increasingly scrutinizing AI-assisted filings and requiring variable certifications. This is highlighted in “Which Federal Courts Have AI Judicial Standing Orders?” The message is not anti-technology; it is pro-judgment and verification.
• The American Bar Association’s (ABA) Standing Committee on Ethics and Professional Responsibility issued Formal Opinion 512 in July 2024, emphasizing, among other things, that lawyers are required to develop a reasonable understanding of the capabilities and limitations of AI use as part of the duty of competence.
Judgment maturity is becoming a market expectation. In-house departments are not passive observers. They see the value of integrating AI capabilities and are increasingly seeking ways to bring legal work in-house, as borne out in the Association of Corporate Counsel’s survey results, “Generative AI’s Growing Strategic Value for Corporate Law Departments.” As a result, corporate legal teams are critically evaluating how law firms deploy generative AI and report a perceived lack of awareness regarding outside counsel’s use of the technology on their legal matters. Some even expect to push for changes to billable hour arrangements with outside counsel because of generative AI’s impact.
The challenge is converting system awareness and industry pressure into personal professional judgment.
Shifting From Tool Training to Judgment Development
Clearly, a majority of law firms recognize that AI is the biggest technology pushing significant change in the industry over the next 3-5 years, but there is a great deal of variation in the approaches firms are taking in response to this realization. Early AI enablement efforts at law firms focused on core elements of access, security policies centered around proper use, and basic prompting techniques. Prudence dictated building a solid training foundation.
But there is emerging recognition that the legal AI journey requires evolving beyond the basics toward something less technical, and yet just as fundamentally important and substantive: developing human judgment and instinct that guides our use (and at times, intentional non-use) of AI. Recent initiatives by law firms and legal AI providers have included:
• Evolving approaches to junior associate training to encourage and facilitate development of the lawyer’s “judgment” muscle through multi-modal, scenario-based training models, as highlighted in “Reinventing Associate Training for the Age of AI”
• Treating AI training as a continuing education responsibility of firms, core to associate and attorney development. According to “The Grace to Dabble: Two Biglaw Firms Look to an AI-First Future,” some firms are even putting skin in the game by providing billable hour credit to associates.
• The growth of AI-specific leadership roles within firms to inject context and oversight at the practice level within practice groups, rather than centralizing AI governance solely as an IT function; and
• Practice-specific training simulations teaching and guiding attorneys to know and practically apply the limitations of AI-generated work product, including identifying hallucinated citations, comparing AI-generated drafts to human drafts, and conducting structured verification reviews.
Early Building Blocks for an AI Judgment Framework
While there is variation in how organizations are supporting the development and maturation of judgment skills, below are some common threads that have been emerging.
1. Teaching AI Literacy and Limitations through Scenarios. Beyond helping legal professionals to understand the basics of AI and the pitfalls to be on the lookout for, such as hallucinations, bias, and outdated training data, programming presents users with realistic scenarios where AI outputs are plausible but may be wrong. This requires learners to critically assess and verify, rather than passively accept outputs. Harvard Law School even offers an “AI and the Law” executive education program where scenario-based roleplays are used to navigate AI-fueled dilemmas.
2. Ethical Reasoning Baked In. As previously mentioned, attorneys have ethical obligations tied to their use of AI, particularly in relation to competence, confidentiality, communication, and supervision. Programs have seen success embedding these topics into scenario-based training materials. The key is to not treat ethics as a separate compliance module, but instead treat it as an organizing framework for judgment calls attorneys must make in their AI use.
3. Risk-Tiered AI Use Cases. Not all uses of AI carry the same risk. Differentiating these use cases can help to heighten risk awareness, while avoiding arbitrarily dampening the prospective value of AI for lower-risk tasks. Borrowing from ediscovery and technology-assisted review defensibility frameworks, organizations can generally categorize AI use cases and scale human oversight according to risk. For example:
| Low-Risk | Ideation |
| Medium-Risk | Drafting |
| High-Risk | Filing or Regulatory Submissions |
4. AI Reliance Checklists/ Practice Resources. Workflows that incorporate the use or reliance on AI should also integrate protocols to facilitate baseline Human-in-the-Loop verification on proper usage. This step is analogous to citation checking, due diligence checklists, or document review quality control measures. AI output review is a skill to be developed, not a formality. Modeling from guidance emerging from judicial orders and bar opinions, these checklists might include queries such as:
• Have all citations been independently verified?
• Are assumptions factually supported?
• Does this output affect client rights?
• Would disclosure be required or prudent?
5. Practice- and Role-Tailored Training. Resist the temptation to deploy one-size-fits-all training programs. I am a big believer that, like politics, all change and innovation is local. Learning to exercise judgment on the use of AI cannot be abstract.
A litigator assessing AI-assisted research output applies different filters and faces different judgment demands than a transactional attorney reviewing an AI-drafted contract clause. Likewise, a first-year associate drafting a contract with the assistance of AI has different judgment demands than a general counsel evaluating an enterprise AI platform. Effective training programs segment by practice group, seniority level, and/or function and build practice-specific modules that situate reasoning in recognizable professional contexts that resonate with learners.
6. Measuring Judgment Maturity. As the management principle states, we should measure what matters. KPIs for AI training should move beyond indicators on whether users are accessing AI tools to tracking metrics that demonstrate proficiency and maturation in the exercise of professional judgment, such as:
• Citation defect rates
• Review protocol compliance
• Audit documentation of AI-assisted workflows
• Practice-specific AI standards adoption
7. Cultural Shift to AI-Enabled Talent. Finally, the ultimate, and perhaps longer-term, success of building AI judgment frameworks will be evidenced by a mindset shift. As AI-assisted workflows become the norm, traditional views on professional identity and legal team roles are evolving. “Human-in-the-Loop” will take on a new and very tangible meaning. This shift may look something like:
| Historical Roles | Emerging Roles |
| Senior lawyer = drafterAssociate/ paralegal = researcher | Lawyer = risk calibratorAI = draft acceleratorHuman = validator |
In the emerging model of AI-enabled talent, the speed of content generation is likely not the top indicator of premium skill. The focus will be more on context and risk calibration, error detection, ethical reasoning, and supervision of AI outputs, informed by legal subject-area expertise.
The Skill Machines Can’t Replace
Early on, the AI race was about getting access and standing up use cases. The next phase, which is critical to sustainability in legal, is about judgment and defensibility. Courts have sanctioned AI misuse, and bars have clarified overriding duties. Clients are asking questions about AI governance. The organizations that will lead are those that move beyond teaching legal professionals to prompt better, to teaching them how to decide better.
Brendan W. Miller, J.D. is a legal innovator: a curious, seasoned litigator and corporate attorney, technologist, strategist, and change agent. To Brendan, legal innovation is about continually being relevant for clients, by making the business and practice of law easier, better, and more valuable.
The post Teaching Judgment In The Age Of AI: Building Legal Professionals Who Rely On AI Responsibly appeared first on Above the Law.
Ed. note: This article first appeared in ILTA’s Peer-to-Peer Magazine.
Breaking news: business and society, including the legal industry, have generally decided AI may have some benefit that is worth leveraging. Many of us have grown markedly more comfortable with using AI in our day-to-day lives and activities. Need help polishing that email you want to send? Check. Want a quick outline to facilitate a strategic discussion? Check. Want to quickly summarize or compare a set of documents? Check. Want to see what you’d look like as an action figure?! Check.
Just make sure you remove any use of the em dash from AI’s output to avoid associated stigma. (Frustrating, on a personal note, as a writer who used to make frequent use of the em dash before AI was doing so!)
The evidence has continued to mount over the last few years. According to 2025 Pew Research Center findings, Americans are interested in AI and want it involved in their daily activities. Stanford’s 2025 AI Index also found that a majority of businesses are using, investing in, and experimenting with AI. Based on Thomson Reuters 2025 Generative AI in Professional Services Report & ILTA’s 2025 Technology Survey, the legal industry is here for it; legal continues to embrace and expand its usage of AI.
However, there’s a catch: comfort and adoption with AI are rising faster than judgment maturity. I’m a big advocate and user of AI, but I also recognize time-tested pearls of wisdom apply here: “Just because you can do something, doesn’t mean you should do that thing.” Or perhaps you prefer “with great power comes great responsibility.”
The Stakes: What If the Machine Is Wrong?
The legal profession and industry are built on core principles of logic and rules. The intentional use of language and reasoning are central to the practice of law, and practitioners carry an important responsibility in serving clients. All attorneys know these tenets well, as they are woven into the fundamentals of legal training. IRAC, which stands for Issue, Rule, Application, Conclusion, is the foundational legal analysis and writing outline taught in law schools, and it demands attention to detail and clear congruence between facts and applicable rules. As service professionals, legal practitioners are also duty-bound to follow the rules of professional conduct and shape their practice in accordance with ethical standards.
Adherence to these duties and principles is imperative for legal. They reflect the exercise of professional responsibility and the very essence of client service. They demand the autonomous exercise of professional judgment. They cannot and should not be delegated to AI.
There is ample evidence that the legal profession and industry are aware of the dangers of not exercising proper professional judgment over the use of AI:
• The oft-cited Mata v. Avianca, Inc., where attorneys were sanctioned for submitting hallucinated cases, emphasized that the duty of verification cannot be delegated. The issue was that professional judgment was not exercised; the core issue wasn’t that AI was used, but rather that AI was not properly supervised.
• Professional liability insurance is evolving, and related guidance cautions against lawyers’ reliance on unverified AI outputs, as outlined in “From innovation to exposure: artificial intelligence risks for legal professionals.” Failure to supervise AI could expose firms to malpractice claims.
• Beyond Mata, courts are increasingly scrutinizing AI-assisted filings and requiring variable certifications. This is highlighted in“Which Federal Courts Have AI Judicial Standing Orders?” The message is not anti-technology; it is pro-judgment and verification.
• The American Bar Association’s (ABA) Standing Committee on Ethics and Professional Responsibility issued Formal Opinion 512 in July 2024, emphasizing, among other things, that lawyers are required to develop a reasonable understanding of the capabilities and limitations of AI use as part of the duty of competence.
Judgment maturity is becoming a market expectation. In-house departments are not passive observers. They see the value of integrating AI capabilities and are increasingly seeking ways to bring legal work in-house, as borne out in the Association of Corporate Counsel’s survey results, “Generative AI’s Growing Strategic Value for Corporate Law Departments.” As a result, corporate legal teams are critically evaluating how law firms deploy generative AI and report a perceived lack of awareness regarding outside counsel’s use of the technology on their legal matters. Some even expect to push for changes to billable hour arrangements with outside counsel because of generative AI’s impact.
The challenge is converting system awareness and industry pressure into personal professional judgment.
Shifting From Tool Training to Judgment Development
Clearly, a majority of law firms recognize that AI is the biggest technology pushing significant change in the industry over the next 3-5 years, but there is a great deal of variation in the approaches firms are taking in response to this realization. Early AI enablement efforts at law firms focused on core elements of access, security policies centered around proper use, and basic prompting techniques. Prudence dictated building a solid training foundation.
But there is emerging recognition that the legal AI journey requires evolving beyond the basics toward something less technical, and yet just as fundamentally important and substantive: developing human judgment and instinct that guides our use (and at times, intentional non-use) of AI. Recent initiatives by law firms and legal AI providers have included:
• Evolving approaches to junior associate training to encourage and facilitate development of the lawyer’s “judgment” muscle through multi-modal, scenario-based training models, as highlighted in “Reinventing Associate Training for the Age of AI”
• Treating AI training as a continuing education responsibility of firms, core to associate and attorney development. According to “The Grace to Dabble: Two Biglaw Firms Look to an AI-First Future,” some firms are even putting skin in the game by providing billable hour credit to associates.
• The growth of AI-specific leadership roles within firms to inject context and oversight at the practice level within practice groups, rather than centralizing AI governance solely as an IT function; and
• Practice-specific training simulations teaching and guiding attorneys to know and practically apply the limitations of AI-generated work product, including identifying hallucinated citations, comparing AI-generated drafts to human drafts, and conducting structured verification reviews.
Early Building Blocks for an AI Judgment Framework
While there is variation in how organizations are supporting the development and maturation of judgment skills, below are some common threads that have been emerging.
1. Teaching AI Literacy and Limitations through Scenarios. Beyond helping legal professionals to understand the basics of AI and the pitfalls to be on the lookout for, such as hallucinations, bias, and outdated training data, programming presents users with realistic scenarios where AI outputs are plausible but may be wrong. This requires learners to critically assess and verify, rather than passively accept outputs. Harvard Law School even offers an “AI and the Law” executive education program where scenario-based roleplays are used to navigate AI-fueled dilemmas.
2. Ethical Reasoning Baked In. As previously mentioned, attorneys have ethical obligations tied to their use of AI, particularly in relation to competence, confidentiality, communication, and supervision. Programs have seen success embedding these topics into scenario-based training materials. The key is to not treat ethics as a separate compliance module, but instead treat it as an organizing framework for judgment calls attorneys must make in their AI use.
3. Risk-Tiered AI Use Cases. Not all uses of AI carry the same risk. Differentiating these use cases can help to heighten risk awareness, while avoiding arbitrarily dampening the prospective value of AI for lower-risk tasks. Borrowing from ediscovery and technology-assisted review defensibility frameworks, organizations can generally categorize AI use cases and scale human oversight according to risk. For example:
| Low-Risk | Ideation |
| Medium-Risk | Drafting |
| High-Risk | Filing or Regulatory Submissions |
4. AI Reliance Checklists/ Practice Resources. Workflows that incorporate the use or reliance on AI should also integrate protocols to facilitate baseline Human-in-the-Loop verification on proper usage. This step is analogous to citation checking, due diligence checklists, or document review quality control measures. AI output review is a skill to be developed, not a formality. Modeling from guidance emerging from judicial orders and bar opinions, these checklists might include queries such as:
• Have all citations been independently verified?
• Are assumptions factually supported?
• Does this output affect client rights?
• Would disclosure be required or prudent?
5. Practice- and Role-Tailored Training. Resist the temptation to deploy one-size-fits-all training programs. I am a big believer that, like politics, all change and innovation is local. Learning to exercise judgment on the use of AI cannot be abstract.
A litigator assessing AI-assisted research output applies different filters and faces different judgment demands than a transactional attorney reviewing an AI-drafted contract clause. Likewise, a first-year associate drafting a contract with the assistance of AI has different judgment demands than a general counsel evaluating an enterprise AI platform. Effective training programs segment by practice group, seniority level, and/or function and build practice-specific modules that situate reasoning in recognizable professional contexts that resonate with learners.
6. Measuring Judgment Maturity. As the management principle states, we should measure what matters. KPIs for AI training should move beyond indicators on whether users are accessing AI tools to tracking metrics that demonstrate proficiency and maturation in the exercise of professional judgment, such as:
• Citation defect rates
• Review protocol compliance
• Audit documentation of AI-assisted workflows
• Practice-specific AI standards adoption
7. Cultural Shift to AI-Enabled Talent. Finally, the ultimate, and perhaps longer-term, success of building AI judgment frameworks will be evidenced by a mindset shift. As AI-assisted workflows become the norm, traditional views on professional identity and legal team roles are evolving. “Human-in-the-Loop” will take on a new and very tangible meaning. This shift may look something like:
| Historical Roles | Emerging Roles |
| Senior lawyer = drafterAssociate/ paralegal = researcher | Lawyer = risk calibratorAI = draft acceleratorHuman = validator |
In the emerging model of AI-enabled talent, the speed of content generation is likely not the top indicator of premium skill. The focus will be more on context and risk calibration, error detection, ethical reasoning, and supervision of AI outputs, informed by legal subject-area expertise.
The Skill Machines Can’t Replace
Early on, the AI race was about getting access and standing up use cases. The next phase, which is critical to sustainability in legal, is about judgment and defensibility. Courts have sanctioned AI misuse, and bars have clarified overriding duties. Clients are asking questions about AI governance. The organizations that will lead are those that move beyond teaching legal professionals to prompt better, to teaching them how to decide better.
Brendan W. Miller, J.D. is a legal innovator: a curious, seasoned litigator and corporate attorney, technologist, strategist, and change agent. To Brendan, legal innovation is about continually being relevant for clients, by making the business and practice of law easier, better, and more valuable.
