In our enthusiasm to adopt and use GenAI, are we ignoring a critical problem: what are the tools doing to our critical thinking skills? And just why are experienced partners better at using GenAI than younger lawyers?

I attended an interesting panel discussion on Monday at the American Association of Law Librarians (AALL) conference in Portland. It was entitled I Hate Hallucinations: How to Combat Misconceptions Around GenAI Accuracy. It was advertised to be a discussion on the misconception of GenAI inaccuracies, how to reduce hallucinations, how to ask GenAI questions the right way to reduce risk, and what the future holds. The panelists included Andrea Guldalian, Director of Library Services at Duane Morris, Saskia Mehlhorn, Director of Knowledge Services at Norton Rose Fulbright, and Patrick Parsons, the Director of the Law Library at the University of Pittsburgh School of Law.

The Critical Discussion

But despite the session description, the conversation quickly pivoted to something more fundamental: the implicit dangers of GenAI and the problems the profession faces in the training of young lawyers. (I recently discussed this issue in an article on the development of gut instinct in the days of GenAI.) This part of the discussion raised some troublesome and challenging issues.

These issues can be clearly seen in the context of legal research. Here’s the reasoning of the panel. First, the use of GenAI for everyone but particularly law students and young lawyers is tempting because it’s so easy. Parsons summed it up this way: “It’s too easy and really tempting to just rely on what it gives you.” The other problem is that GenAI is really good at doing rote tasks, summarizing and manipulating data, and easing the everyday burdens. But legal research is different.

It’s the Process, Stupid

Parsons believes, and so do I, that legal research is a process. “Legal research is not finding the law. It’s not a one-sentence answer.” Legal research involves finding a case, reading a case, reading similar cases, reading cases that come to an opposite conclusion, and understanding not just the holding of the case but the context and nuance. None of the GenAI tools can do this, at least not yet.

The danger, of course, is that the more younger lawyers — and for that matter, the rest of us rely — on GenAI for the rote tasks, the more we are tempted to use it for more complicated things like legal research to get quick answers. “Legal research is not just finding a quote,” said Parsons.

Certainly, in my years of practice, I saw plenty of lawyers who tried to take the easy way out with research pre-GenAI. These were the ones who relied on headnotes instead of reading the cases. This often led to disastrous results when their opposition (me) actually read their cases. But today, it’s even easier and more tempting than ever.

The Problem is the Problem

Which raises another more fundamental concern. Sound legal research (and solving many other complicated problems) involves thinking through the problem on the front end. There is a well-known adage that has been applied to things from therapy to business management: “the problem…is the problem.”

One of my mentors used to always ask me: what’s the first thing you do in any case? It’s not reading cases or, in today’s parlance, asking ChatGPT for an answer. It’s sitting back and looking out at the Ohio River (our offices overlooked the River). What he meant of course was thinking through the problem was the key. Or as Albert Einstein put it, “If I had an hour to solve a problem, I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.”

But here’s what’s going on now, according to the panel: as Mehlhorn put it, “Critical thinking has left the house. Too often younger lawyers don’t think through the problem, they just want information quickly.”

Senior Lawyers Better at Using GenAI? What?

The erosion of critical thinking skills makes another panelist’s observation particularly telling: an observation that on first blush seems surprising but really shouldn’t be. Mehlhorn observed that it was the more senior lawyers who are the best users of AI products. Why? Mehlhorn says it’s because they know what they are looking for. In other words, they have a better grip on the problem. They also know when an answer looks right. And when it looks wrong.

How do they know these things? I think it’s because of their experience with the research process that Parsons talked about, over and over again, so that they perhaps subconsciously see patterns, as I discussed in my article. It’s this pattern that triggers what I referred to as “gut instinct.” But as someone smarter than me observed after reading my article, it’s not gut instinct but wisdom.

The Real Challenge

But here is the danger and the challenge we as a profession are facing. As we rely more and more on GenAI tools, we risk losing the process that develops the critical thinking skills, the wisdom that more experienced lawyers have.

We have to find some ways to first instill among our young lawyers the respect for critical thinking and the process necessary to solve legal problems. We have to develop ways to ensure that they develop and use those skills and processes.

We have to make sure that with all the benefits of GenAI, we don’t come to expect and even demand that quick answer. That isolated and seemingly reliant quote. We need to demand that they show us that they have thought through the problem, not asked ChatGPT for an easy answer.

The solution isn’t to abandon GenAI. But we need to be intentional about preserving and encouraging the critical thinking skills that make good lawyers. This means building training programs that emphasize process over speed, creating mentorship opportunities where senior lawyers can pass on their pattern recognition skills, and perhaps most importantly, giving young lawyers permission to take the time to think.

Let’s Not Kid Ourselves

As with everything, reliance and use of GenAI comes with a price. We are only starting to see that price. We have to find ways to minimize and control that price else our future lawyers won’t be humans. They will be robots with simplistic non-answers to complicated problems that go nowhere.

Just as it has always been, the future of law depends on lawyers who can think. A GenAI tool that spends 55 minutes thinking about a problem would be a laughingstock today. Let’s not give our profession the same treatment.

Stephen Embry is a lawyer, speaker, blogger and writer. He publishes TechLaw Crossroads (Opens in a new window), a blog devoted to the examination of the tension between technology, the law, and the practice of law.

The post GenAI And Critical Thinking: The Problem Is The Problem appeared first on Above the Law.