“Garbage in, garbage out” has become a cliche in the age of generative AI, but many law firms are still storming ahead with technology investment despite the messy reality of their data governance.
Various versions of documents may be spread across inboxes, file shares, document-management platforms, SharePoint sites, and databases. Key records may exist in the form of emails, transaction data, or even presentations saved on a local hard drive.
Layering decision-making software onto that fragmented information landscape is like repairing the bus while driving it, said Kurt Thies, Global Head of Information Governance Strategy at the legal services consultancy Morae.
“AI hasn’t created the information governance problem,” he said. “But it has finally made the business consequences impossible to ignore.”
Widespread disorder in law firm data coincides with legal professionals being wary of AI — and it may even be a cause of this distrust.
According to Morae’s research, only 33% of senior leaders trust the results of AI-assisted legal work, while 67% worry that the cost of human verification is outweighing AI’s productivity benefits.
To help firms get the most out of their AI investments, Thies outlined four practical steps to prepare the information landscape.
No. 1: Learn What You Have
As a first step, law firms must take an inventory of all of their data — and it’s not as easy as it sounds.
The nature of today’s legal work means that a record does not always take the form of a conventional document such as a contract or court filing. It may exist as data inside a system recognizing a transaction, an email recording a decision, or even a presentation stored on a local hard drive.
“You can’t govern information you don’t know you have,” Thies said. “Start by understanding where it lives, what it contains, and whether it still holds business value.”
Once located, information must be classified according to the risk it creates if mishandled, the obligations governing its retention, and who is authorized to see it.
Certain documents, like wills, are high-risk and high-obligation. The same classification principles apply to, say, an HR file containing Social Security numbers or health information.
While this data may present privacy and cybersecurity risks, regulations may also require the firm to retain a document for a specified period, or a legal hold may prevent its scheduled destruction.
Other concerns apply to items like executed contracts, which must be managed according to factors like renewals, holds, or eligibility for disposition.
“The goal isn’t to govern everything equally,” Thies said. “It’s to identify the data that actually matters and make sure it’s trustworthy and accessible.”
No. 2: Create Systems
Once you’ve completed your data census, the next step is to determine what information is authoritative — and what data the AI tools should be using.
“Think about a contract,” Thies said. “There may be a dozen drafts scattered across email and document management systems, but only one is executed.”
If you want your AI tools to access just the executed, authoritative version, you need an information governance policy that will map these documents onto a reliable system of record.
Many law firms may not be set up for success here, however. According to Morae’s research, one third of them have no information-governance policies at all. Among those that do, only 30% of senior team members are confident that they’re being executed.
Firms that are able to put a policy in place, however, will be well positioned to realize the productivity and other business benefits legal tech can provide. It won’t require paying more for more technology; it will ensure the firm can receive the ROI from its tech investments.
No. 3: Make Execution Repeatable
The next challenge is turning those reinforced systems into recurring workflows that require as few new decisions as possible.
Most organizations have some sort of retention policy, but the challenge is to apply it consistently. If it relies on individuals to remember what to keep or dispose of, the system is set up for failure.
“Rules must be built into repeatable workflows so governance becomes a daily business process, not a periodic cleanup fire drill,” Thies said.
Falling short on this point can also have negative consequences, as applying the same policy differently from one record or matter to another can expose a firm to liability and make it difficult to defend why information was kept or destroyed.
Thies said many organizations have retention policies but do not regularly destroy records that have become eligible. A one-time “destruction day,” when employees choose what to delete, does not solve that problem.
“That’s bad,” Thies said. “You want it to happen because your system says, ‘This HR contract has been sitting in our system for 10 years, and now it’s eligible.’”
The records team should review eligible information on a regular schedule — quarterly, semiannually, or annually, and they should not depend on individual attorneys approving each step.
The process must also be auditable. A firm needs to be able to show what information existed, when it was disposed of, which rule applied at the time, and what action was taken.
No. 4: Set Useful AI Parameters
Once the governance framework is operating, firms can set parameters around their AI tools. They can direct an LLM toward approved repositories and current records while preserving the permissions that protect sensitive information.
Working from that controlled body of information gives the model fewer bad sources to reconcile and improves AI outputs dramatically. The investment begins paying off when extensive human verification no longer consumes the time AI was supposed to save.
“That’s how you get to trust and reliability,” Thies said. “Without these steps, it’s not achievable.”
The post Want To Show Legal Tech ROI? Start By Getting Your Data In Order appeared first on Above the Law.
“Garbage in, garbage out” has become a cliche in the age of generative AI, but many law firms are still storming ahead with technology investment despite the messy reality of their data governance.
Various versions of documents may be spread across inboxes, file shares, document-management platforms, SharePoint sites, and databases. Key records may exist in the form of emails, transaction data, or even presentations saved on a local hard drive.
Layering decision-making software onto that fragmented information landscape is like repairing the bus while driving it, said Kurt Thies, Global Head of Information Governance Strategy at the legal services consultancy Morae.
“AI hasn’t created the information governance problem,” he said. “But it has finally made the business consequences impossible to ignore.”
Widespread disorder in law firm data coincides with legal professionals being wary of AI — and it may even be a cause of this distrust.
According to Morae’s research, only 33% of senior leaders trust the results of AI-assisted legal work, while 67% worry that the cost of human verification is outweighing AI’s productivity benefits.
To help firms get the most out of their AI investments, Thies outlined four practical steps to prepare the information landscape.
No. 1: Learn What You Have
As a first step, law firms must take an inventory of all of their data — and it’s not as easy as it sounds.
The nature of today’s legal work means that a record does not always take the form of a conventional document such as a contract or court filing. It may exist as data inside a system recognizing a transaction, an email recording a decision, or even a presentation stored on a local hard drive.
“You can’t govern information you don’t know you have,” Thies said. “Start by understanding where it lives, what it contains, and whether it still holds business value.”
Once located, information must be classified according to the risk it creates if mishandled, the obligations governing its retention, and who is authorized to see it.
Certain documents, like wills, are high-risk and high-obligation. The same classification principles apply to, say, an HR file containing Social Security numbers or health information.
While this data may present privacy and cybersecurity risks, regulations may also require the firm to retain a document for a specified period, or a legal hold may prevent its scheduled destruction.
Other concerns apply to items like executed contracts, which must be managed according to factors like renewals, holds, or eligibility for disposition.
“The goal isn’t to govern everything equally,” Thies said. “It’s to identify the data that actually matters and make sure it’s trustworthy and accessible.”
No. 2: Create Systems
Once you’ve completed your data census, the next step is to determine what information is authoritative — and what data the AI tools should be using.
“Think about a contract,” Thies said. “There may be a dozen drafts scattered across email and document management systems, but only one is executed.”
If you want your AI tools to access just the executed, authoritative version, you need an information governance policy that will map these documents onto a reliable system of record.
Many law firms may not be set up for success here, however. According to Morae’s research, one third of them have no information-governance policies at all. Among those that do, only 30% of senior team members are confident that they’re being executed.
Firms that are able to put a policy in place, however, will be well positioned to realize the productivity and other business benefits legal tech can provide. It won’t require paying more for more technology; it will ensure the firm can receive the ROI from its tech investments.
No. 3: Make Execution Repeatable
The next challenge is turning those reinforced systems into recurring workflows that require as few new decisions as possible.
Most organizations have some sort of retention policy, but the challenge is to apply it consistently. If it relies on individuals to remember what to keep or dispose of, the system is set up for failure.
“Rules must be built into repeatable workflows so governance becomes a daily business process, not a periodic cleanup fire drill,” Thies said.
Falling short on this point can also have negative consequences, as applying the same policy differently from one record or matter to another can expose a firm to liability and make it difficult to defend why information was kept or destroyed.
Thies said many organizations have retention policies but do not regularly destroy records that have become eligible. A one-time “destruction day,” when employees choose what to delete, does not solve that problem.
“That’s bad,” Thies said. “You want it to happen because your system says, ‘This HR contract has been sitting in our system for 10 years, and now it’s eligible.’”
The records team should review eligible information on a regular schedule — quarterly, semiannually, or annually, and they should not depend on individual attorneys approving each step.
The process must also be auditable. A firm needs to be able to show what information existed, when it was disposed of, which rule applied at the time, and what action was taken.
No. 4: Set Useful AI Parameters
Once the governance framework is operating, firms can set parameters around their AI tools. They can direct an LLM toward approved repositories and current records while preserving the permissions that protect sensitive information.
Working from that controlled body of information gives the model fewer bad sources to reconcile and improves AI outputs dramatically. The investment begins paying off when extensive human verification no longer consumes the time AI was supposed to save.
“That’s how you get to trust and reliability,” Thies said. “Without these steps, it’s not achievable.”
The post Want To Show Legal Tech ROI? Start By Getting Your Data In Order appeared first on Above the Law.
