ChamberOne

Case-law summaries from AI: what they leave out

By the ChamberOne team 4 minute read

A good case summary is a map of the judgment. It is handy for deciding whether to read on. The danger comes when the map is mistaken for the territory. AI tools produce tidy, confident summaries in seconds, and tidy summaries leave things out. What they leave out is often what a court or a supervising partner cares about most. This article is for associates and students. It lists what typically goes missing and offers a method for deciding when a summary is enough and when it is not.

Key points

  • Summaries often drop procedural posture, dissents, the limits of the holding and later treatment.
  • First confirm the judgment exists and read it in a reliable source.
  • Read in order: stage and question, facts, reasoning, separate opinions, order, later treatment.
  • Skim if a wrong summary costs little; read in full for anything you advise on or cite.

Procedural posture disappears first

A summary will often say 'the court held that...' without saying which court, at what stage and on what question. Was it an appeal on a point of law, a trial finding of fact, an interim application or a strike-out? The same words carry very different weight depending on the answer.

A statement made on an application to dismiss a claim at an early stage may say only that the claim is arguable. A summary that states it as a finding that the claim succeeds is badly misleading. Always find the posture first.

Dissents and separate opinions get lost

Where judges disagree, the summary usually follows the majority and may not mention the rest. But the dissent can show how fragile the rule is, which points were close, and where later courts may go. Where there are several concurring opinions with different reasons, the actual ratio may be narrower than the summary suggests, or hard to identify.

Check how many judges sat, whether they agreed on the result and whether they agreed on the reasons.

The limits of the holding

Judges often say what they are not deciding. They qualify a rule with 'on the facts of this case' or leave a question open. A summary compresses these into a general statement. You then cite the case for a broader rule than the court laid down, and the other side points out the qualification.

Look for passages where the court narrows its own reasoning, and for the facts it treats as important. A case that turned on an unusual fact may not help you if your facts differ.

Other things that tend to go missing

  • Obiter remarks that a later court may adopt, and the difference between those and the ratio.
  • The way the court treated earlier cases: followed, distinguished, doubted or overruled.
  • Whether the case has been appealed, reversed or criticised later.
  • The statutory provision as it stood at the time, which may differ from today's text.
  • Costs, remedies and orders that show how far the court was willing to go.
  • Whether the decision binds the court you are in, or is persuasive only.

Fabrication is a separate risk

Everything above assumes the case exists and the tool is summarising a real text. Tools can also produce a plausible case name, citation and holding that is entirely invented. Before you read a summary, make sure you can find the judgment itself in a reliable source. If you cannot, treat the case as unreal.

Be careful also when you give a tool a case name without the text. It may summarise from memory of other cases and what it thinks the case probably says. Paste in the actual judgment, where your firm's rules allow it, and ask for a summary of that text.

Do
  • Find the judgment itself in a trusted source
  • Paste in the actual judgment where your rules allow
  • Quote from the judgment, with paragraph numbers
Don't
  • Give a tool a case name without the text
  • Quote from the summary
Using AI case summaries safely

A method for reading when it matters

Use the summary to decide where to look, then read the judgment in a deliberate order.

  1. 1

    Find the court, date and stage, and read the headnote or first paragraphs for the question decided.

  2. 2

    Read the facts, noting any that the court stresses.

  3. 3

    Read the reasoning on the point you need, in the judge's words, with the passage open next to your notes.

  4. 4

    Read any separate or dissenting opinions.

  5. 5

    Read the order, to see what the court actually did.

  6. 6

    Check later treatment in a citator or a trusted database.

  7. 7

    Quote from the judgment, with paragraph numbers, never from the summary.

A short scenario

Suppose a trainee is told that a tool says a certain case 'established that a landlord owes a duty to repair in all residential tenancies'. It sounds useful. She opens the judgment and finds that it was an appeal on a narrow question about a particular type of lease, that one judge dissented, and that the court said in terms that it was not deciding the wider point. Her supervisor would have been unhappy to see the broad statement in a letter. Ten minutes of reading saved her from it.

The lesson is not that summaries are useless. It is that the summary told her where to look, and the judgment told her what was true.

When a summary is enough

Not every case needs a full read. A summary is fine for background reading, for sorting a large result set, for working out which five cases of fifty to read properly, or for refreshing your memory of a case you already know. It is not enough for anything you will rely on in advice, quote in a letter or cite to a court.

A simple rule is to ask what happens if the summary is wrong. If the answer is 'nothing much', skim. If the answer is 'I give wrong advice' or 'I mislead the court', read.

A summary is enough
  • Background reading
  • Sorting a large result set
  • Choosing which five of fifty cases to read
  • Refreshing your memory of a case you know
Read the judgment
  • Anything you will rely on in advice
  • Anything you will quote in a letter
  • Anything you will cite to a court
When to skim and when to read the judgment

Make the tool help you read

Better uses exist than asking for a summary. Ask the tool to list the questions the court had to answer, then find each answer yourself. Ask it to point to the paragraphs where the court qualifies its holding. Ask it to produce a list of the facts the court treated as significant. Then check each point. Used this way, the tool speeds up your reading without standing in for it.

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next

AI case law summaries: what they leave out | ChamberOne