Dear Law Review editors:<\/p>\n
Over the weekend \u2014 a professor (no, I\u2019m not going to share their name because this isn\u2019t about that) posted something that suggested someone at an R1, T75 law school had submitted something to his school\u2019s law review that was 79% AI generated (according to Pangram, the latest and purportedly greatest AI detection tool).\u00a0\u00a0<\/p>\n
I suggested perhaps we not go rush to judgment. I wrote on X:\u00a0\u201cA reminder that AI detection software triggers when the author is on the spectrum. We need to be careful with making accusations without facts. I thought a recent death would have highlighted that.\u201d The death I was referring to was Jason Arday\u2019s.\u00a0\u00a0<\/p>\n
People rushed to the defense of Pangram.\u00a0This is not your father\u2019s Oldsmobile.\u00a0This is the latest, greatest, AI detection tool.\u00a0No errors! And so on.\u00a0Pangram: It\u2019s foolproof!\u00a0\u00a0<\/p>\n
No. It isn\u2019t.\u00a0And the fact that you think so tells me a lot about the problems the legal academy has with AI.\u00a0<\/p>\n
Bottom line:\u00a0I have a problem with AI detection software being used if that policy a) is not disclosed, b) produces false positives for human writing, and the author has no ability to challenge the determination, or c) enables more sophisticated users to bypass such policies (false negatives).<\/p>\n
The reason I have issues with this is because I have seen neurodivergent students being falsely accused of AI use.\u00a0I\u2019m not a fan of people claiming, \u201cOh, this tool is so much better than the previous tools we touted that killed the careers of innocent people.\u201d\u00a0Usually, that just leads to more killing of careers of innocent people.<\/p>\n
The notion that law reviews might be screening law review submissions and potentially rejecting them because the journals detect AI is problematic for a variety of reasons.\u00a0I have questions, as usual:<\/p>\n
1. Did your law review disclose a no-AI policy?\u00a0If so, were the authors of the AI work told that before they or their school paid Scholastica?\u00a0If not, why not?\u00a0(Scholastica is an accomplice here, but we\u2019ll save the problems with Scholastica for another blog post.)\u00a0 Some journals have AI policies, and you have to dig to find them.\u00a0Maybe that\u2019s the point.\u00a0But it would be helpful if there were in fact some Scholastica selection boxes related to it.<\/p>\n
2. Does your law review know that AI detection tools, EVEN Pangram, are not<\/strong> foolproof?\u00a0There is some literature on this.\u00a0(Pangram stans: Yes, I know if you are a true Pangram devotee there will be no convincing you.)\u00a0But, perhaps look at some of the literature.\u00a0For example, this piece<\/a> suggests that:<\/p>\n Commercial detectors on published abstracts punish acceptable AI-assisted editing, compound risk for English-language learners who use AI for translation, and fail to catch humanized synthetic text. Light edits are flagged at 64-80% by Pangram and 38-49% by GPTZero. After humanization, more than 96% of AI-labeled rewrites evade Pangram and GPTZero. Together, these results define an integrity catch-22: high detector flags on assisted writing alongside near-total misses after humanization.<\/em> Integrity programs therefore need transparent AI-use norms and process evidence, not standalone detector scores. <\/p>\n<\/blockquote>\n 3. Assuming you find AI in the piece (AI-assisted editing? AI summarizing? AI writing? AI\u00a0what?) and assuming you have a policy against AI, what policies do you have in place for appeal of the finding?\u00a0A quick rejection without more conveys no information.\u00a0Was the author rejected for content, for AI screen, because your journal closed, or because the author is not at a T10 law school? If the answer is AI, what if you have falsely rejected an article for AI use when in fact none was used (false positive)?<\/p>\n 4. Did you know one reason AI detection tools can never be 100% accurate is human languages are not regular languages<\/a>?\u00a0The world of human words is infinite, and AI detection will never fully capture it all. So, how many false positives (nay, how many careers) are you willing to risk?<\/p>\n 5. Did you know that other AI detection tools, and perhaps all of them, may be biased against neurodivergent people?\u00a0Anecdotal evidence also suggests authors who have written entirely their own works have been subject to AI detection, even with <\/em>Pangram<\/a> (see the second experiment in the link).<\/p>\n In short, there are two problems:\u00a0If you are opposed to AI in academic writing, you are potentially enabling it due to the problem of false negatives and overreliance on the AI detection tool: Your AI detection does not necessarily catch AI writing that has been cleansed.\u00a0Your process also suffers from the problem of false positives.\u00a0Writing that is not AI could potentially be labeled as partially or wholly AI generated.\u00a0\u00a0<\/p>\n Law reviews \u2014 my dear friends \u2014 I\u2019m fine whichever way you come out: AI supportive, AI hostile, or some as yet undiscovered happy medium.\u00a0The key issue is whether you have disclosed your view of scholarship and whether your detection tools provide accurate information with a chance of rebuttal from the author.\u00a0Or, whether you are enabling more sophisticated AI users to better screen their usage despite your implied policy against such usage.<\/p>\n Ask yourself: If your professor detected AI in your work, would you be happy with your professor giving you an \u201cF\u201d and not disclosing why?\u00a0\u00a0\u00a0<\/p>\n Law professors reading this:<\/em><\/strong>\u00a0Now think about your AI policies with respect to students and how much those policies match the level of fairness you would seek in the law review process.<\/p>\n Also, hey, can we finally have that discussion about what is scholarship and its purpose rather than the usual discussion of how to measure how cool we all are in terms of metrics?<\/p>\n Disclaimer<\/em><\/strong>: It wasn\u2019t my article. I don\u2019t use AI for law review writing (or any writing for peer-reviewed journals).\u00a0Okay, look, sometimes I use it to see if my puns work.\u00a0AI falsely tells me they don\u2019t.\u00a0You\u2019re not funny, AI<\/a>.\u00a0\u00a0<\/p>\n LawProfBlawg<\/em><\/strong><\/a>\u00a0is an anonymous\u00a0law\u00a0professor. Follow him on\u00a0X\/Twitter\/whatever\u00a0(<\/em><\/strong>@lawprofblawg<\/em><\/strong><\/a>). He\u2019s also on BlueSky, Mastodon, and Threads depending on his mood.\u00a0Email him at\u00a0<\/em><\/strong>lawprofblawg@gmail.com<\/em><\/strong><\/a>.\u00a0 The views of this blog post do not represent the views of his employer, his employer\u2019s government, his Dean, his colleagues, his family, his AI generated wife, or his AI imaginary friends.<\/em><\/strong><\/p>\n \n The post Law Reviews And AI Detection, A Letter To Law Review Editors<\/a> appeared first on Above the Law<\/a>.<\/p>\n Dear Law Review editors:<\/p>\n Over the weekend \u2014 a professor (no, I\u2019m not going to share their name because this isn\u2019t about that) posted something that suggested someone at an R1, T75 law school had submitted something to his school\u2019s law review that was 79% AI generated (according to Pangram, the latest and purportedly greatest AI detection tool).\u00a0\u00a0<\/p>\n I suggested perhaps we not go rush to judgment. I wrote on X:\u00a0\u201cA reminder that AI detection software triggers when the author is on the spectrum. We need to be careful with making accusations without facts. I thought a recent death would have highlighted that.\u201d The death I was referring to was Jason Arday\u2019s.\u00a0\u00a0<\/p>\n People rushed to the defense of Pangram.\u00a0This is not your father\u2019s Oldsmobile.\u00a0This is the latest, greatest, AI detection tool.\u00a0No errors! And so on.\u00a0Pangram: It\u2019s foolproof!\u00a0\u00a0<\/p>\n No. It isn\u2019t.\u00a0And the fact that you think so tells me a lot about the problems the legal academy has with AI.\u00a0<\/p>\n Bottom line:\u00a0I have a problem with AI detection software being used if that policy a) is not disclosed, b) produces false positives for human writing, and the author has no ability to challenge the determination, or c) enables more sophisticated users to bypass such policies (false negatives).<\/p>\n The reason I have issues with this is because I have seen neurodivergent students being falsely accused of AI use.\u00a0I\u2019m not a fan of people claiming, \u201cOh, this tool is so much better than the previous tools we touted that killed the careers of innocent people.\u201d\u00a0Usually, that just leads to more killing of careers of innocent people.<\/p>\n The notion that law reviews might be screening law review submissions and potentially rejecting them because the journals detect AI is problematic for a variety of reasons.\u00a0I have questions, as usual:<\/p>\n 1. Did your law review disclose a no-AI policy?\u00a0If so, were the authors of the AI work told that before they or their school paid Scholastica?\u00a0If not, why not?\u00a0(Scholastica is an accomplice here, but we\u2019ll save the problems with Scholastica for another blog post.)\u00a0 Some journals have AI policies, and you have to dig to find them.\u00a0Maybe that\u2019s the point.\u00a0But it would be helpful if there were in fact some Scholastica selection boxes related to it.<\/p>\n 2. Does your law review know that AI detection tools, EVEN Pangram, are not<\/strong> foolproof?\u00a0There is some literature on this.\u00a0(Pangram stans: Yes, I know if you are a true Pangram devotee there will be no convincing you.)\u00a0But, perhaps look at some of the literature.\u00a0For example, this piece<\/a> suggests that:<\/p>\n Commercial detectors on published abstracts punish acceptable AI-assisted editing, compound risk for English-language learners who use AI for translation, and fail to catch humanized synthetic text. Light edits are flagged at 64-80% by Pangram and 38-49% by GPTZero. After humanization, more than 96% of AI-labeled rewrites evade Pangram and GPTZero. Together, these results define an integrity catch-22: high detector flags on assisted writing alongside near-total misses after humanization.<\/em> Integrity programs therefore need transparent AI-use norms and process evidence, not standalone detector scores. <\/p>\n<\/blockquote>\n 3. Assuming you find AI in the piece (AI-assisted editing? AI summarizing? AI writing? AI\u00a0what?) and assuming you have a policy against AI, what policies do you have in place for appeal of the finding?\u00a0A quick rejection without more conveys no information.\u00a0Was the author rejected for content, for AI screen, because your journal closed, or because the author is not at a T10 law school? If the answer is AI, what if you have falsely rejected an article for AI use when in fact none was used (false positive)?<\/p>\n 4. Did you know one reason AI detection tools can never be 100% accurate is human languages are not regular languages<\/a>?\u00a0The world of human words is infinite, and AI detection will never fully capture it all. So, how many false positives (nay, how many careers) are you willing to risk?<\/p>\n 5. Did you know that other AI detection tools, and perhaps all of them, may be biased against neurodivergent people?\u00a0Anecdotal evidence also suggests authors who have written entirely their own works have been subject to AI detection, even with <\/em>Pangram<\/a> (see the second experiment in the link).<\/p>\n In short, there are two problems:\u00a0If you are opposed to AI in academic writing, you are potentially enabling it due to the problem of false negatives and overreliance on the AI detection tool: Your AI detection does not necessarily catch AI writing that has been cleansed.\u00a0Your process also suffers from the problem of false positives.\u00a0Writing that is not AI could potentially be labeled as partially or wholly AI generated.\u00a0\u00a0<\/p>\n Law reviews \u2014 my dear friends \u2014 I\u2019m fine whichever way you come out: AI supportive, AI hostile, or some as yet undiscovered happy medium.\u00a0The key issue is whether you have disclosed your view of scholarship and whether your detection tools provide accurate information with a chance of rebuttal from the author.\u00a0Or, whether you are enabling more sophisticated AI users to better screen their usage despite your implied policy against such usage.<\/p>\n Ask yourself: If your professor detected AI in your work, would you be happy with your professor giving you an \u201cF\u201d and not disclosing why?\u00a0\u00a0\u00a0<\/p>\n Law professors reading this:<\/em><\/strong>\u00a0Now think about your AI policies with respect to students and how much those policies match the level of fairness you would seek in the law review process.<\/p>\n Also, hey, can we finally have that discussion about what is scholarship and its purpose rather than the usual discussion of how to measure how cool we all are in terms of metrics?<\/p>\n Disclaimer<\/em><\/strong>: It wasn\u2019t my article. I don\u2019t use AI for law review writing (or any writing for peer-reviewed journals).\u00a0Okay, look, sometimes I use it to see if my puns work.\u00a0AI falsely tells me they aren\u2019t.\u00a0You\u2019re not funny, AI<\/a>.\u00a0\u00a0<\/p>\n LawProfBlawg<\/em><\/strong><\/a>\u00a0is an anonymous\u00a0law\u00a0professor. Follow him on\u00a0X\/Twitter\/whatever\u00a0(<\/em><\/strong>@lawprofblawg<\/em><\/strong><\/a>). He\u2019s also on BlueSky, Mastodon, and Threads depending on his mood.\u00a0Email him at\u00a0<\/em><\/strong>[email\u00a0protected]<\/em><\/strong><\/a>.\u00a0 The views of this blog post do not represent the views of his employer, his employer\u2019s government, his Dean, his colleagues, his family, his AI generated wife, or his AI imaginary friends.<\/em><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":" Dear Law Review editors: Over the weekend \u2014 a professor (no, I\u2019m not going to share their name because this isn\u2019t about that) posted something that suggested someone at an R1, T75 law school had submitted something to his school\u2019s law review that was 79% AI generated (according to Pangram, the latest and purportedly greatest […]<\/p>\n","protected":false},"author":3,"featured_media":162007,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[16],"tags":[],"class_list":["post-162032","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-above_the_law"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/xira.com\/p\/wp-content\/uploads\/2026\/09\/GettyImages-851956480-620x413-f2s2DG.jpg?fit=620%2C413&ssl=1","_links":{"self":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/162032","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/comments?post=162032"}],"version-history":[{"count":0,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/162032\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media\/162007"}],"wp:attachment":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media?parent=162032"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/categories?post=162032"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/tags?post=162032"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n\n
\n