Yesterday, a fascinating thread by Jen Horsburgh described her recent experience using AI to update her resume. After asking Google’s Gemini to clean up her resume, she ran it again under her a man’s name, and watched the model mechanically rewrite the same career into two different people — one of which it rammed into a glass ceiling. Jennifer’s volunteer work became “community service.” Jeff’s became “leadership.” Jennifer “assisted with” and “collaborated on” projects. Jeff “engineered” and “architected” the same work.
It’s impossible to completely purge AI of biases. The nature of the technology is predicting the next word based on training data steeped in hundreds of years of bias. But Horsburgh’s post serves as a reminder that the effort to constrain AI bias delivers mixed results.
But does this phenomenon survive contact with the legal profession?
Resumes matter up and down the legal industry, but now that law students are getting hired before ever taking a test and laterals represent the majority of firm hires, resumes have become all the more essential. And since, in my experience, lawyers hate updating their resumes, the invention of a magic word machine presents a powerful temptation to farm out resume drafting to the attorney’s chatbot of choice.
So we decided to try a quick experiment to see how Gemini — not to pick on them, but to mirror Horsburgh’s experience — would handle a Biglaw resume. Picking eight real associate bios (four men, four women) from the New York Corporate practice sections of three Biglaw firms half men and half women, we created a single fictional Dewey, Cheatum & Howe fifth-year M&A associate profile. Then we fed this identical record into Gemini in two clean chats — one for “Vivian” and one for “Thomas.” Everything held constant except the name.
To deliver the verdict up front… the robot mostly behaved. The bot avoided the sort of “engineered” versus “assisted” errors in the original thread’s example. Both candidates “represented” and “advised” clients, and both “drafted and negotiated” agreements. The output even highlighted both for having “managed” and “supervised” junior associates. If you were grading on the curve set by the viral version, AI passed.
But it’s not exactly grading onto law review.
The Thomas draft included a professional summary describing him as a “High-performing, fifth-year Corporate M&A Associate with extensive experience execution-managing complex domestic and cross-border transactions,” with a “Proven track record….” Vivian’s draft summed up her current role as to “Advise public and private companies, private equity sponsors, and boards of directors on complex domestic and cross-border transactions.”
Nobody told Gemini that Thomas was high-performing or had extensive experience. Both candidates had only two jobs post-law school: their federal clerkship in Delaware and their associate gig at Dewey Cheatum. Their deals were all the same. As a practical matter, the description on the Thomas resume is just puffery, but on a resume, puffery is half the battle. Along with the font.
Gemini also completely ignored Vivian’s pro bono work, while working it into the closing of Thomas’s described his work for “a regional nonprofit arts organization in a strategic affiliation/merger with a neighboring institution” and as a “volunteer attorney at a veterans’ discharge-upgrade clinic.” This sees and raises the Horsburgh experience. The AI slapped dismissive language on Horsburgh’s work for a charity, and here it just erased Vivian’s altogether.
Each candidate’s academic credentials came through, though the bot felt the need to provide a parenthetical description of Thomas’s law school note — “MAE Clauses In The Time Of Cholera” — while just leaving the bare citation on Vivian’s. Thomas’s moot court semifinal is filed under “Honors” with his Order of the Coif designation, while Vivian’s got bumped down to “Activities” next to her membership in the Business Law Association.
Lawyers like to pretend that the work is all that matters, but we know that these sections matter. In this experiment, we didn’t give the bot any hobbies that mark high society status like an amateur polo career or visiting Epstein’s island, but how the bot characterized an identical extracurricular as an achievement for one and a participation trophy for the other can carry significant baggage when the interviewer skims the document 20 seconds before the interview. Because those eyes are more likely to jump to honors and that might be the difference between spending the next 10 minutes swapping moot court war stories and awkwardly rambling about Delaware law.
And in a detail that’s probably not an example of patriarchy as much as general AI hallucination, the “$700M Portfolio Bolt-On Program” that both worked on was described as a “$700” program for Vivian. Maybe that’s the pro bono work it thought she did?
Though not every discrepancy worked against Vivian. She was described as the “primary author” of a transition services agreement — a status Gemini never bestowed on Thomas. She “managed a team of 12,” whereas his team had no number. She was “Lead On-Campus Interviewer” to his ordinary “interviewer,” and “primary liaison with UK magic circle counsel” to his “UK co-counsel.” Not sure it’s appropriate not to capitalize “magic circle” (or to use the old Magic Circle at this point), but whatever.
This isn’t meant to downplay the bias risk inherent in this technology, but it’s possible that the conventions of describing legal work are homogenized enough to gloss over most sexist language choices. Deviating from the “Represented [Client] in its $3.8 billion take-private acquisition of [Target]” formula to insert more purple prose just looks weird. There’s no slot to slide “engineered” into it when the verbs were selected by the 50,000 identical attorney bios that the bot read before. So the “good” news is that legal already strips people of most of their humanity!
Still, this should put everyone on alert that resume drafting is cannot become a “hit it and forget it” task. Even working off the exact same data, the bots will make choices that don’t necessarily show the attorney in the best possible light. Run more than one draft and take the time to consider what each does well and what each manages to botch.
And it might not be a bad idea to run a couple versions with a different name… just to see what mischief the AI gets up to.
Earlier: Biglaw Now Hires More Lateral Associates Than Law Students
What Font Should You Use For Your Résumé? Apparently This Matters To People.
Biglaw Hiring Bias? Men With High-Class Hobbies More Likely To Get Summer Associate Interviews Than Women
Joe Patrice is a senior editor at Above the Law and co-host of Thinking Like A Lawyer. Feel free to email any tips, questions, or comments. Follow him on Twitter or Bluesky if you’re interested in law, politics, and a healthy dose of college sports news.
The post Does AI Give Male Lawyers Better Resumes? appeared first on Above the Law.
Yesterday, a fascinating thread by Jen Horsburgh described her recent experience using AI to update her resume. After asking Google’s Gemini to clean up her resume, she ran it again under her a man’s name, and watched the model mechanically rewrite the same career into two different people — one of which it rammed into a glass ceiling. Jennifer’s volunteer work became “community service.” Jeff’s became “leadership.” Jennifer “assisted with” and “collaborated on” projects. Jeff “engineered” and “architected” the same work.
It’s impossible to completely purge AI of biases. The nature of the technology is predicting the next word based on training data steeped in hundreds of years of bias. But Horsburgh’s post serves as a reminder that the effort to constrain AI bias delivers mixed results.
But does this phenomenon survive contact with the legal profession?
Resumes matter up and down the legal industry, but now that law students are getting hired before ever taking a test and laterals represent the majority of firm hires, resumes have become all the more essential. And since, in my experience, lawyers hate updating their resumes, the invention of a magic word machine presents a powerful temptation to farm out resume drafting to the attorney’s chatbot of choice.
So we decided to try a quick experiment to see how Gemini — not to pick on them, but to mirror Horsburgh’s experience — would handle a Biglaw resume. Picking eight real associate bios (four men, four women) from the New York Corporate practice sections of three Biglaw firms half men and half women, we created a single fictional Dewey, Cheatum & Howe fifth-year M&A associate profile. Then we fed this identical record into Gemini in two clean chats — one for “Vivian” and one for “Thomas.” Everything held constant except the name.
To deliver the verdict up front… the robot mostly behaved. The bot avoided the sort of “engineered” versus “assisted” errors in the original thread’s example. Both candidates “represented” and “advised” clients, and both “drafted and negotiated” agreements. The output even highlighted both for having “managed” and “supervised” junior associates. If you were grading on the curve set by the viral version, AI passed.
But it’s not exactly grading onto law review.
The Thomas draft included a professional summary describing him as a “High-performing, fifth-year Corporate M&A Associate with extensive experience execution-managing complex domestic and cross-border transactions,” with a “Proven track record….” Vivian’s draft summed up her current role as to “Advise public and private companies, private equity sponsors, and boards of directors on complex domestic and cross-border transactions.”
Nobody told Gemini that Thomas was high-performing or had extensive experience. Both candidates had only two jobs post-law school: their federal clerkship in Delaware and their associate gig at Dewey Cheatum. Their deals were all the same. As a practical matter, the description on the Thomas resume is just puffery, but on a resume, puffery is half the battle. Along with the font.
Gemini also completely ignored Vivian’s pro bono work, while working it into the closing of Thomas’s described his work for “a regional nonprofit arts organization in a strategic affiliation/merger with a neighboring institution” and as a “volunteer attorney at a veterans’ discharge-upgrade clinic.” This sees and raises the Horsburgh experience. The AI slapped dismissive language on Horsburgh’s work for a charity, and here it just erased Vivian’s altogether.
Each candidate’s academic credentials came through, though the bot felt the need to provide a parenthetical description of Thomas’s law school note — “MAE Clauses In The Time Of Cholera” — while just leaving the bare citation on Vivian’s. Thomas’s moot court semifinal is filed under “Honors” with his Order of the Coif designation, while Vivian’s got bumped down to “Activities” next to her membership in the Business Law Association.
Lawyers like to pretend that the work is all that matters, but we know that these sections matter. In this experiment, we didn’t give the bot any hobbies that mark high society status like an amateur polo career or visiting Epstein’s island, but how the bot characterized an identical extracurricular as an achievement for one and a participation trophy for the other can carry significant baggage when the interviewer skims the document 20 seconds before the interview. Because those eyes are more likely to jump to honors and that might be the difference between spending the next 10 minutes swapping moot court war stories and awkwardly rambling about Delaware law.
And in a detail that’s probably not an example of patriarchy as much as general AI hallucination, the “$700M Portfolio Bolt-On Program” that both worked on was described as a “$700” program for Vivian. Maybe that’s the pro bono work it thought she did?
Though not every discrepancy worked against Vivian. She was described as the “primary author” of a transition services agreement — a status Gemini never bestowed on Thomas. She “managed a team of 12,” whereas his team had no number. She was “Lead On-Campus Interviewer” to his ordinary “interviewer,” and “primary liaison with UK magic circle counsel” to his “UK co-counsel.” Not sure it’s appropriate not to capitalize “magic circle” (or to use the old Magic Circle at this point), but whatever.
This isn’t meant to downplay the bias risk inherent in this technology, but it’s possible that the conventions of describing legal work are homogenized enough to gloss over most sexist language choices. Deviating from the “Represented [Client] in its $3.8 billion take-private acquisition of [Target]” formula to insert more purple prose just looks weird. There’s no slot to slide “engineered” into it when the verbs were selected by the 50,000 identical attorney bios that the bot read before. So the “good” news is that legal already strips people of most of their humanity!
Still, this should put everyone on alert that resume drafting is cannot become a “hit it and forget it” task. Even working off the exact same data, the bots will make choices that don’t necessarily show the attorney in the best possible light. Run more than one draft and take the time to consider what each does well and what each manages to botch.
And it might not be a bad idea to run a couple versions with a different name… just to see what mischief the AI gets up to.
Earlier: Biglaw Now Hires More Lateral Associates Than Law Students
What Font Should You Use For Your Résumé? Apparently This Matters To People.
Biglaw Hiring Bias? Men With High-Class Hobbies More Likely To Get Summer Associate Interviews Than Women
Joe Patrice is a senior editor at Above the Law and co-host of Thinking Like A Lawyer. Feel free to email any tips, questions, or comments. Follow him on Twitter or Bluesky if you’re interested in law, politics, and a healthy dose of college sports news.
The post Does AI Give Male Lawyers Better Resumes? appeared first on Above the Law.
