ChamberOne

How to check AI legal research before you rely on it

By the ChamberOne team 5 minute read

An AI tool that returns a tidy paragraph with five authorities has not done your research. It has produced something that looks like research and that you now have to test. The work of testing is not glamorous, but it is quick if you do it the same way every time. This article sets out a routine that a busy associate or partner can apply to any AI-assisted research note, whatever tool produced it.

Key points

  • List every authority and the exact proposition it is cited for before you check anything.
  • Confirming a case exists is only half the job. Find the paragraph and the quoted words.
  • Note up each authority and record the date, because AI tools do not reliably show later history.
  • Look for contrary authority yourself. The tool answers only the question it was asked.
  • Scale the checking to how far the output will travel: more for clients and courts, less for orientation.

Treat the output as a lead, not a finding

The most useful mental shift is to read AI research output the way you would read a note from a keen but unsupervised pupil. It may point you to the right place. It may also be confident and wrong. Nothing in the text tells you which.

That means the output has no status until you have checked it. Do not copy a citation into a draft, an email or a skeleton argument because it appeared in a chat window. Copy it into a checklist first.

Step one: list every authority and every proposition

Before you open a single case or statute, write a two-column list. In the left column put each authority the tool named. In the right column put the exact proposition the tool says that authority supports.

This sounds like busywork, but it does two things. It stops you checking only that a case exists while ignoring what it is said to decide. And it shows you how many separate claims you are about to rely on, which is often more than the neat summary suggested.

Step two: find the primary source yourself

For each authority, retrieve the judgment, statute or rule from a source you trust: the court's own site, an official legislation database, or a recognised law report service you already use. Do not accept a link supplied by the tool without checking where it leads.

Check the basics against what the tool told you.

  • The name, court, year and neutral or report citation all match one real document.
  • The judge, panel or issuing body is the one described.
  • The jurisdiction is the one you need. Authorities from another country may be persuasive at best, and tools sometimes mix them.
  • The version of the statute or rule is the one in force at the relevant time.

Step three: read the passage, not the headnote

A citation that exists can still be cited for something it does not say. This is the quieter failure and it is the one that gets past people who have already confirmed the case is real.

Open the judgment and find the paragraph that supports the proposition. If the tool quoted words, search for those exact words. If you cannot find them, treat the quotation as unreliable and do not use it. If the tool paraphrased, read enough of the surrounding text to see whether the court was deciding the point, discussing it in passing, or distinguishing it.

Note the paragraph number next to the authority in your list. A later reader should be able to check your work in a minute.

Do
  • Open the judgment and find the paragraph that supports the proposition.
  • Search for the exact words of any quotation.
  • Note the paragraph number next to the authority.
Don't
  • Stop once you have confirmed that the case exists.
  • Use a quotation you cannot find in the source.
Checking a citation against the judgment

Steps four and five: confirm it is still good law, then look for gaps

AI tools are trained on a snapshot, and even tools connected to live sources may not tell you whether a decision was appealed, overruled, distinguished or superseded by legislation. Use the citator or noting-up service you normally use, or the court's own appeal listings, and record the date you checked.

For statutes and rules, look for amendments, repeals and commencement provisions. A section that was accurate when the model was trained may have changed since.

Even a clean set of real, accurate authorities may be incomplete. The tool answers the question you asked, and it may have missed a leading case that goes the other way.

Spend a few minutes looking for contrary authority yourself. Search for the opposing proposition, check what the other side is likely to cite, and look at textbook or practitioner commentary for the area. If the AI result and your own search disagree, believe neither until you have read the sources.

Record the check, and match effort to risk

Keep the checklist on the file. It shows who verified what and when, and it is useful if a supervising partner or a client later asks how a point was researched.

Not every question needs the same depth. A background question to orient yourself in an unfamiliar area can be lightly checked, provided nothing from it goes into advice or a filing. Anything that will be cited to a court, a client or the other side needs every step above. A short rule helps: the further the output travels from your desk, the more of it you verify.

  • Internal orientation only: confirm the sources exist and skim the key passages.
  • Advice to a client: all five steps, with a second person reviewing the list.
  • Court filings: all five steps, with the supervising lawyer personally reading each cited passage.

Where this routine does not help

The routine catches fabricated and misdescribed authorities. It does not tell you whether the argument is a good one, whether the court will be receptive to it, or whether the question was framed correctly. Those remain matters of legal judgment, and no checklist replaces them.

Check the guidance from your regulator or bar about the use of AI in research and in court documents. Practice rules are changing, and some courts have issued their own directions.

A worked example, clearly hypothetical

Suppose an associate asks a tool whether a landlord can recover a particular category of cost, and gets back a confident paragraph with four authorities. She writes them in her list. Two are easy to find and say roughly what the tool claimed. One cannot be found under that name and court, so she drops it and says why in her note. The fourth exists, but when she reads the paragraph, the court was assuming the point rather than deciding it.

Of four authorities, two survive. The note she hands her partner says so, with paragraph numbers, and flags that the best case for the other side has not been found yet. That is a smaller, less impressive note than the tool produced. It is also one the partner can safely use.

The example shows why the order matters. If she had stopped after confirming that three of the four cases existed, the note would have gone up with a misdescribed authority in it.

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next