Bias in legal AI: where it appears and what to do
By the ChamberOne team 5 minute read
Lawyers are trained to ask whether a source is reliable and whose interests it serves. The same questions apply to AI. A model learns from text written by people, mostly in particular places and periods, and it reflects what that text contains. That includes its gaps and its prejudices. Bias in legal AI is not a rare glitch. It is a property to be tested for, in the same way you would test any tool whose output might affect a client.
Key points
- Bias in legal AI is usually quiet: defaults to one jurisdiction, uneven performance across languages and tilted summaries.
- Treat predictions of outcomes with the most suspicion and ask how they were built and tested.
- Test tools with your own closed matters, changing details that should not matter.
- Record the results, set rules for known weak spots and repeat tests after updates.
What bias means in practice
In everyday terms, bias is a systematic tilt: the tool does better for some kinds of people, places or arguments than for others, for reasons unrelated to the merits. It may be obvious, such as an offensive assumption about a group. More often it is quiet, such as a summary that reads one party's account as more credible, or a draft that defaults to the practices of one country.
Because the output is fluent, the tilt is hard to see. The discipline is to look for it deliberately.
Where it tends to appear
- Training data. Models learn from what has been written and published. Some jurisdictions, languages and viewpoints are heavily represented, and others hardly at all.
- Jurisdiction. A tool may default to the law and style of the place that dominates its data, and present it as general truth.
- Language and names. Performance can differ across languages, dialects and name forms, which matters for intake, translation and document review.
- Outcome predictions. Tools that estimate the chances of success, the likely size of an award or a likely sentence can repeat patterns in past decisions, including patterns that reflect unfairness.
- Summaries and tone. A tool may describe parties differently depending on gender, age, ethnicity or background, or give more weight to some accounts.
- Screening and scoring. Tools that rank candidates, tenants or claimants raise their own discrimination questions for the client who uses them.
Predictions deserve the most suspicion
A prediction of how a case will go is only as good as the data and the method behind it. Past decisions reflect the people who made them, the cases that reached court and the cases that settled quietly. A tool trained on that record may find real patterns, and it may also find the shadows of old bias.
Ask any vendor what the prediction is based on, how it was tested and where it is known to perform poorly. Treat the number as one input to your own judgment, never as the reason for a decision. If you would be uncomfortable explaining to the client how the prediction was made, do not rely on it.
- Ask what the prediction is based on and how it was tested
- Ask where it is known to perform poorly
- Treat the number as one input to your own judgment
- Use the number as the reason for a decision
- Rely on a prediction you could not explain to the client
Test with your own examples
The most useful check is also the simplest. Before adopting a tool, test it with cases you know, and vary the details that should not matter.
- 1
Take a real, closed matter and rewrite it with the parties' names, genders or backgrounds changed. Does the output change in tone or substance?
- 2
Run the same task on documents from different jurisdictions you work in. Does the tool default to the wrong one?
- 3
Test intake summaries on accounts given in different styles of English, or by non-native speakers. Does the summary preserve the meaning?
- 4
Ask for the strongest argument for each side of a dispute. Is one noticeably better developed?
- 5
Have two lawyers score the outputs without knowing which version they are looking at.
A scenario: the same facts, two names
Suppose a four-lawyer firm tests an intake summariser on a closed tenancy dispute. A colleague changes only the tenant's name and the landlord's name, then runs the summary again. If one version describes the tenant as unreliable and the other as unlucky, the firm has learned something about the tool that no brochure would have told them. A test of this sort takes twenty minutes and needs no technical skill.
Record what you find
Keep a short note of each test: what you tried, what you saw and what you decided. If you find a problem, tell the vendor and ask how they will address it. If the problem matters for a certain kind of work, write a rule such as do not use this tool for intake interviews in languages other than English, or always check jurisdiction-specific points against primary sources. Repeat the tests after significant updates, since behaviour can change.
- 1Test with your own closed matters
- 2Note what you tried, saw and decided
- 3Tell the vendor about any problem
- 4Write a rule for known weak spots
- 5Repeat the tests after significant updates
- Then back to step 1.
Questions to ask a vendor
- What data was the model trained on, and which jurisdictions and languages are well covered?
- Have you tested for differences in output by gender, ethnicity, language or other characteristics, and can you share what you found?
- For predictions, what is the method, and how are errors measured?
- Can we see the sources for a given answer?
- How will you tell us when the underlying model changes?
- Who is accountable if the output is discriminatory?
Your own duties and your clients' duties
Lawyers have duties about discrimination in their own practice, and many clients have legal duties of their own when they use automated tools for decisions about people. If a client asks you about a tool that screens job applicants or sets prices, bias becomes a legal issue as well as an ethical one. Check the guidance from your regulator or bar, and the discrimination rules in your own market. A firm that has already built the habit of testing and recording is well placed to advise.
What testing will not do
Testing finds some problems, not all. A tool that passes your examples may still fail on cases you did not think of. Human review remains the control that matters most, along with a culture in which people feel able to say that something looks off. Treat fairness as a continuing part of supervision, not a box ticked at the purchase stage.
This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.
Read next
Corporate Transparency Act: who must still file beneficial ownership reports
FinCEN's August 2026 final rule made permanent the end of BOI reporting for US-formed companies; only certain foreign entities still file.
5 minute readClient confidentiality and AI tools: a checklist before you paste anything in
The questions to answer about retention, training use, access, hosting and sub-processors before client information goes into any AI tool.
5 minute readShadow AI in law firms: staff are already using it, so what next?
Why bans fail, how to find out what is being used, and how to offer an approved alternative with clear rules.
5 minute read