ChamberOne

Summarising long documents with AI: risks and a safer method

By the ChamberOne team 4 minute read

Suppose a lawyer is handed a bundle of several hundred pages the night before a call. The temptation is obvious: paste the lot into an AI tool and ask for a summary. The summary arrives in seconds, reads well, and is very hard to check. This article sets out the risks of the single big summary and a method that keeps the speed while giving you something you can verify.

Key points

  • One big summary compresses away the detail that may matter and gives you nothing to check.
  • Summarise section by section, with a page or paragraph link on every point.
  • Verify numbers, dates, names, exceptions and a random sample; read operative clauses in full.
  • Treat scanned pages and tables as drafts, and keep client text out of tools the firm has not assessed.

What goes wrong with one big summary

A single summary of a long document compresses everything into a few paragraphs. Compression means choosing, and the model chooses by what looks typical, not by what matters to your client. A short carve-out buried in an appendix may be exactly what changes the advice, and it is the kind of detail a summary drops.

Long inputs add their own problems. Tools may cut the text off when it exceeds their limit, without saying so. Models can also pay less attention to material in the middle of a long input. And a fluent summary hides its gaps: nothing on the page tells you what was left out.

The deeper difficulty is verification. If the summary has no pointers back to the source, checking it means reading the document, which defeats the purpose. So people skip the check, and the summary becomes the record.

The safer method: summarise in pieces, link each piece

Break the document along its own structure: clauses, headings, chapters, exhibits or date ranges for correspondence. Summarise each piece separately, and attach to each summary the page or paragraph it came from. A reader can then jump from any statement to its source in seconds.

Only after the pieces are summarised should you ask for an overview, built from the piece summaries, not from the raw document. The overview inherits the links. If a sentence in it looks odd, you trace it down a level and read the original.

  • Split by the document's natural sections, not by an arbitrary number of pages.
  • Ask for a neutral summary of what each section says, without commentary or advice.
  • Require a page or paragraph reference for every point.
  • Ask the tool to list anything it could not read, such as images, tables or scanned pages.
  • Build the overview from the section summaries and keep both.
One big summary
  • Compresses everything into a few paragraphs
  • Chooses by what looks typical, not what matters to your client
  • Has no pointers back, so checking means reading the document
Section by section, linked
  • Splits the document along its own structure
  • Gives a page or paragraph reference for every point
  • Builds the overview from the piece summaries
  • Lets you trace an odd sentence down a level
One big summary and a section-by-section method

What to verify

You cannot verify everything, so verify by risk. Start with anything that carries a number, a date or a name, since these are where a small slip does the most damage. Amounts, deadlines, notice periods, party names and defined terms should be compared directly against the source.

Then check negatives and exceptions. Summaries tend to turn 'does not apply where' into a flat rule. Words such as unless, except, subject to and provided that are signals to open the original. Finally, sample: pick a few sections at random and read them against their summaries. If a random sample shows a problem, treat the whole set with suspicion.

What to read in full

Some parts of a document deserve a human reading no matter how good the summary looks. These are the parts where the document's legal effect is concentrated.

  • Operative clauses: payment, termination, liability, indemnity, governing law and dispute resolution.
  • Definitions, because a changed meaning can alter every clause that uses the term.
  • Schedules and appendices that vary or limit the main text.
  • Anything the client has told you worries them.
  • Handwritten notes, stamps and annotations, which text extraction often misses.

Scanned pages and tables

Scanned documents are converted to text by optical character recognition before any model reads them, and that step makes mistakes. A figure with a smudge becomes a different figure. A table may lose its column alignment, so the right number ends up next to the wrong label. If the document came to you as a scan, treat the extracted text as a draft and look at the page image for anything important.

Ask for tables to be reproduced rather than described, and compare them with the page. A narrative description of a table is where errors hide.

Confidentiality before convenience

Summarising a document means sending its text somewhere. Before you do, know where it goes, who can see it, whether it is kept, and whether it can be used to train anyone's model. A client's file should not be pasted into a free public tool. Check your firm's policy, your client's instructions and the guidance from your regulator or bar, and use a tool the firm has assessed.

If a document is too sensitive for any outside tool, summarise it the old way, or redact first. The safer method above works just as well on a redacted copy.

Test the method on a document you know

Before trusting any workflow on a live matter, run it on a document you already know well, such as a lease or agreement from a closed file. Read the summaries with the original in your head. Where did the tool drop a qualification? Did it keep the right party in the right role? Did the page references point to the right place? Ten minutes spent this way tells you more about a tool's habits than any demonstration, and it shows your team what a typical slip looks like.

A worked pattern for a bundle

Suppose the bundle holds a contract, a chain of emails and some invoices. Handle each type differently. For the contract, summarise clause by clause and read the operative clauses yourself. For the emails, summarise in date order with a reference to each message, and pull out decisions and promises. For the invoices, do not summarise at all: extract the amounts and dates into a table and compare them with the source.

Finish with a one-page note that lists what you read in full, what you checked against the source, and what you relied on the summary for. That note is your record if the question is ever asked.

Document analysis in ChamberOne

  • Lets you upload a PDF, Word, Excel, CSV, HTML, text or markdown file to a matter.
  • Lists the document type, parties, key dates, clauses and citations, each linked to the paragraph it came from.
  • Shows risk flags on clauses as suggestions a partner can override.

What it does not do

  • It is produced by an AI model and can miss or misread things, so it needs checking against the document.
  • It does not give legal advice or replace a lawyer's reading of the operative clauses.
  • The document text is sent to the AI provider chosen for the feature; the Security and AI page says what is sent and on what terms.

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next