{"id":146310,"date":"2026-03-18T05:01:37","date_gmt":"2026-03-18T13:01:37","guid":{"rendered":"https:\/\/xira.com\/p\/2026\/03\/18\/new-tool-catches-ai-hallucinations-in-legal-briefs\/"},"modified":"2026-03-18T05:01:37","modified_gmt":"2026-03-18T13:01:37","slug":"new-tool-catches-ai-hallucinations-in-legal-briefs","status":"publish","type":"post","link":"https:\/\/xira.com\/p\/2026\/03\/18\/new-tool-catches-ai-hallucinations-in-legal-briefs\/","title":{"rendered":"New Tool Catches AI Hallucinations In Legal Briefs"},"content":{"rendered":"

A few weeks ago, Gordon Rees found itself on the wrong end of a court filing flagging a number of misleading if not completely wrong citations<\/a> that looked an awful lot like someone told ChatGPT \u201cmake me legal filing!\u201d and hit enter. This would be a problem for any firm, but for Gordon Rees, this was at least its third brush with AI trouble \u2014 having previously admitted to filing a brief riddled with AI hallucinations in October<\/a> and then received a reprimand for filings with citations that \u201cdo not support the specific explanatory phrase\u201d in December. <\/p>\n

As we wrote about the most recent accusation, \u201cWhether any specific citation was generated by AI \u2014 indeed, whether any specific citation is even\u00a0wrong<\/em>\u00a0as opposed to merely debatable \u2014 opposing counsel now has every incentive to scrutinize any citation out of the firm with a jeweler\u2019s loupe.\u201d<\/p>\n

After that article, BriefCatch<\/a> founder Ross Guberman offered me a sneak peek at RealityCheck<\/a>, the company\u2019s new authority verification tool. Essentially, the product picks up where my jeweler\u2019s loupe analogy left off, providing a superpowered hallucination check for lawyers. Running it against the original brief from the October Gordon Rees story that the firm already acknowledged to contain hallucinations, the RealityCheck tool delivered exactly what you\u2019d want as opposing counsel.<\/p>\n

Or, ideally, the senior partner reviewing your own brief before signing your name to a bunch of hallucinatory nonsense. <\/p>\n

\"\"
<\/figcaption><\/figure>\n

At the time, RealityCheck\u2019s splashy announcement launch remained a few weeks off, but with its Legalweek-timed roll out, we can now talk a little about the new essential tool for the LitigationSlop era. <\/p>\n

Lawyers, ideally, painstakingly review every case in every filing. But, at this point, you can\u2019t even trust the Department of Justice<\/a> to check its briefs, let alone your adversaries (or, perhaps, your first-year associates). RealityCheck isn\u2019t trying to replace the process of cite checking, but it is trying to get it done faster and with more certainty.<\/p>\n

The tool uses a two-layer verification process, combining deterministic citation validation<\/em> \u2014 checking reporter volumes, court identifiers, and case names against authoritative legal databases all without any AI involvement \u2014 with AI-assisted analysis<\/em> that then evaluates the quoted language to make sure it actually appears in the cited opinion and actually supports the proposition it\u2019s cited for. Every citation is then scored visually with a Green-Verified, Yellow-Caution, or Red-Incorrect label and explained for the reviewer. <\/p>\n

BriefCatch performed a detailed case study on the Fifth Circuit\u2019s recent decision in Fletcher v. Experian Information Solutions<\/em>, where the court flagged fabricated quotations, misstated holdings, and citations resolving to entirely different cases. This is what the offending filing would look like for a RealityCheck user: <\/p>\n

\"\"
<\/figcaption><\/figure>\n

Unfortunately, this isn\u2019t a problem that\u2019s likely to go away. Researcher Damien Charlotin has now catalogued over 1,000 legal cases involving AI hallucinations<\/a>. Lawyers have started blaming legal AI research tools themselves for introducing errors into their briefs \u2014 which is a bit like blaming a vending machine for not giving you steakhouse dinner<\/a> \u2014 but it speaks to the reality that lawyers increasingly rely on tools for accuracy and mistakes follow. This is how those get caught. <\/p>\n

BriefCatch is making it available to its federal and state court clients. As Guberman put it: \u201cOnce courts are running filed briefs through RealityCheck, the calculus changes for every litigator. The question isn\u2019t whether to verify your citations. It\u2019s whether you want the court to find the errors before you do.\u201d<\/p>\n

Smart play to put the screws to the entire legal market like that.<\/p>\n

\n

\"Headshot\"Joe Patrice<\/a>\u00a0is a senior editor at Above the Law and co-host of Thinking Like A Lawyer<\/a>. Feel free to\u00a0email<\/a> any tips, questions, or comments. Follow him on\u00a0Twitter<\/a>\u00a0or Bluesky<\/a> if you\u2019re interested in law, politics, and a healthy dose of college sports news. Joe also serves as a Managing Director at RPN Executive Search<\/a>.<\/em><\/strong><\/p>\n

The post New Tool Catches AI Hallucinations In Legal Briefs<\/a> appeared first on Above the Law<\/a>.<\/p>\n

\"\"<\/figure>\n

A few weeks ago, Gordon Rees found itself on the wrong end of a court filing flagging a number of misleading if not completely wrong citations<\/a> that looked an awful lot like someone told ChatGPT \u201cmake me legal filing!\u201d and hit enter. This would be a problem for any firm, but for Gordon Rees, this was at least its third brush with AI trouble \u2014 having previously admitted to filing a brief riddled with AI hallucinations in October<\/a> and then received a reprimand for filings with citations that \u201cdo not support the specific explanatory phrase\u201d in December. <\/p>\n

As we wrote about the most recent accusation, \u201cWhether any specific citation was generated by AI \u2014 indeed, whether any specific citation is even\u00a0wrong<\/em>\u00a0as opposed to merely debatable \u2014 opposing counsel now has every incentive to scrutinize any citation out of the firm with a jeweler\u2019s loupe.\u201d<\/p>\n

After that article, BriefCatch<\/a> founder Ross Guberman offered me a sneak peek at RealityCheck<\/a>, the company\u2019s new authority verification tool. Essentially, the product picks up where my jeweler\u2019s loupe analogy left off, providing a superpowered hallucination check for lawyers. Running it against the original brief from the October Gordon Rees story that the firm already acknowledged to contain hallucinations, the RealityCheck tool delivered exactly what you\u2019d want as opposing counsel.<\/p>\n

Or, ideally, the senior partner reviewing your own brief before signing your name to a bunch of hallucinatory nonsense. <\/p>\n

\"\"
<\/figcaption><\/figure>\n

At the time, RealityCheck\u2019s splashy announcement launch remained a few weeks off, but with its Legalweek-timed roll out, we can now talk a little about the new essential tool for the LitigationSlop era. <\/p>\n

Lawyers, ideally, painstakingly review every case in every filing. But, at this point, you can\u2019t even trust the Department of Justice<\/a> to check its briefs, let alone your adversaries (or, perhaps, your first-year associates). RealityCheck isn\u2019t trying to replace the process of cite checking, but it is trying to get it done faster and with more certainty.<\/p>\n

The tool uses a two-layer verification process, combining deterministic citation validation<\/em> \u2014 checking reporter volumes, court identifiers, and case names against authoritative legal databases all without any AI involvement \u2014 with AI-assisted analysis<\/em> that then evaluates the quoted language to make sure it actually appears in the cited opinion and actually supports the proposition it\u2019s cited for. Every citation is then scored visually with a Green-Verified, Yellow-Caution, or Red-Incorrect label and explained for the reviewer. <\/p>\n

BriefCatch performed a detailed case study on the Fifth Circuit\u2019s recent decision in Fletcher v. Experian Information Solutions<\/em>, where the court flagged fabricated quotations, misstated holdings, and citations resolving to entirely different cases. This is what the offending filing would look like for a RealityCheck user: <\/p>\n

\"\"
<\/figcaption><\/figure>\n

Unfortunately, this isn\u2019t a problem that\u2019s likely to go away. Researcher Damien Charlotin has now catalogued over 1,000 legal cases involving AI hallucinations<\/a>. Lawyers have started blaming legal AI research tools themselves for introducing errors into their briefs \u2014 which is a bit like blaming a vending machine for not giving you steakhouse dinner<\/a> \u2014 but it speaks to the reality that lawyers increasingly rely on tools for accuracy and mistakes follow. This is how those get caught. <\/p>\n

BriefCatch is making it available to its federal and state court clients. As Guberman put it: \u201cOnce courts are running filed briefs through RealityCheck, the calculus changes for every litigator. The question isn\u2019t whether to verify your citations. It\u2019s whether you want the court to find the errors before you do.\u201d<\/p>\n

Smart play to put the screws to the entire legal market like that.<\/p>\n

\n

\"Headshot\"Joe Patrice<\/a>\u00a0is a senior editor at Above the Law and co-host of Thinking Like A Lawyer<\/a>. Feel free to\u00a0email<\/a> any tips, questions, or comments. Follow him on\u00a0Twitter<\/a>\u00a0or Bluesky<\/a> if you\u2019re interested in law, politics, and a healthy dose of college sports news. Joe also serves as a Managing Director at RPN Executive Search<\/a>.<\/em><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

A few weeks ago, Gordon Rees found itself on the wrong end of a court filing flagging a number of misleading if not completely wrong citations that looked an awful lot like someone told ChatGPT \u201cmake me legal filing!\u201d and hit enter. This would be a problem for any firm, but for Gordon Rees, this […]<\/p>\n","protected":false},"author":3,"featured_media":146277,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[16],"tags":[],"class_list":["post-146310","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-above_the_law"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/xira.com\/p\/wp-content\/uploads\/2026\/03\/Headshot-300x200-QWjbql.jpg?fit=300%2C200&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/146310","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/comments?post=146310"}],"version-history":[{"count":0,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/146310\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media\/146277"}],"wp:attachment":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media?parent=146310"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/categories?post=146310"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/tags?post=146310"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}