Privilege review with AI in document production
By the ChamberOne team 5 minute read
Disclosure and production exercises often turn on a single worry: what if a privileged email goes out with the rest? The sheer number of documents makes a complete human read costly. AI-assisted review promises to sort the pile and point reviewers at the risky documents. It can do that, as long as everyone is clear that the tool flags and people decide.
Key points
- Use AI to flag and prioritise, not to decide, which documents are privileged.
- Missed privileged documents are the costliest error, so low scores do not remove the need for human review.
- Build layers: first-level review, senior escalation, quality sampling and partner approval.
- Test the tool on a past set with known answers and agree the pass mark in advance.
- Keep records of the tool, settings, sampling and decisions, and check where the data goes.
What privilege review is, and why errors are costly
Privilege review means identifying documents that a party is entitled to withhold from production because of legal professional privilege or a similar protection, and deciding how to deal with them. The detail of what is protected, and what happens after an accidental disclosure, differs from one jurisdiction to another. Check the rules of the court and the guidance from your regulator or bar.
Whatever the system, a mistake in one direction can waive or endanger a protection that belongs to the client, and a mistake in the other, withholding too much, can lead to challenge and cost. Privilege errors are among the more serious mistakes a litigation team can make, which is why the process around the tool matters more than the tool.
What a first-pass flag can do
A tool can read a large set of documents and indicate which are likely to involve lawyers or legal advice. It can identify the names of known lawyers, the presence of terms such as 'advice', 'counsel' or 'without prejudice', and the sort of language that suggests a request for or a provision of legal advice. It can rank documents so that reviewers see the likeliest first.
This is useful for prioritising. It lets a team put its most experienced reviewers on the documents most likely to be sensitive and avoids spending partner time on circulars and newsletters. It is a sorting aid, not a decision-maker.
Where it goes wrong
Tools are weak where privilege depends on context. An email that looks like business correspondence may attach legal advice. A chat with an in-house lawyer may be partly privileged and partly commercial. A forwarded message may have changed the status of a document. A tool may flag a document because a lawyer's name appears on it, even if the lawyer was only copied for information.
False negatives, where privileged material is not flagged, matter most. A low score is not a reason to skip human review. Treat the sort order as a way to arrange the work, not to decide which documents deserve attention.
- Spotting known lawyers' names
- Spotting terms such as 'advice' or 'without prejudice'
- Ranking documents so the likeliest come first
- Business-looking emails that attach legal advice
- Chats that are part privileged, part commercial
- Flagging a lawyer who was only copied in
- A low score hiding privileged material
A workflow with human review at the second level
A reliable pattern has layers, with people responsible for each decision.
- 1
Set up the list of lawyers, law firms and in-house counsel, with their domains and any alternative names, before the tool runs.
- 2
Run the tool to produce a first-pass flag and score for each document.
- 3
Have first-level reviewers read every flagged document and a defined sample of unflagged ones.
- 4
Escalate doubtful documents to a senior reviewer who decides, and record the reason.
- 5
Run a quality check on a sample of documents marked not privileged, before anything leaves the firm.
- 6
Have a partner approve the final privilege list before production.
Testing the tool on your own set
Before relying on a tool for a real exercise, test it. Take a sample of documents from a past matter in which privilege calls were made and carefully checked. Run the tool and compare. How many privileged documents did it flag? How many did it miss? How many harmless documents did it flag? Note in particular what the misses have in common.
Agree in advance what level of performance would be acceptable and what the fallback is if it does not reach it. The pass mark should be set by the person with responsibility for the exercise, and written down.
Keeping defensible records
If the process is later challenged, you will want to show what was done and why. Keep a record of the tool and version used, the settings, the list of names provided, the sampling and quality checks, the decisions made by reviewers and the approvals. Keep the privilege log in the form the rules and the parties require, with descriptions that identify the basis of the claim without revealing the content.
Be careful with the privilege of the review itself. Notes on why a document was judged privileged may themselves be sensitive. Store them with the same protection as the rest of the file.
Confidentiality and the tool provider
Putting privileged material into a third-party system is itself a decision that needs thought. Ask where the data is stored and processed, whether any of it is used to improve the provider's models, who at the provider can access it and what happens when the project ends. Your client may have views, and some clients will require approval.
Check that the system respects restrictions on who within the firm may see which documents, so that review teams see only what they should.
Cost and proportion
The business case for AI-assisted review rests on proportion. A tool that halves the reading time on a very large set can justify its cost, but the quality-control steps add time that must be counted. Do not promise a client a saving until you have run a pilot on their kind of data and measured the effort of second-level review as well as the first pass.
Explain the approach to the client and to the other side where the rules or a case management direction call for it. Check the guidance from your regulator or bar and the rules of the court on technology-assisted review before you start.
When not to use AI for privilege
For a small set of documents, a complete human read is simpler and safer. For matters involving highly sensitive material, such as an internal investigation, consider whether a third-party tool is appropriate at all. And where the rules of the court or the parties' agreement address technology-assisted review, follow them.
This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.
Read next
AI in Witness Bundle and Proof of Evidence Preparation: What to Automate and What Not To
A practical guide for litigators on which parts of witness bundle and proof of evidence preparation AI can handle, and which parts it must not touch.
6 minute readHallucinated citations: what court sanctions teach about checking AI output
The patterns behind fake authorities reaching courts, and the supervision and checking controls that would have caught them.
5 minute readE-discovery and technology-assisted review explained for non-specialists
How predictive coding and generative review differ, why validation matters and how to document the process.
5 minute read