ChamberOne

Document comparison: AI summary versus a redline compare tool

By the ChamberOne team 5 minute read

The counterparty's lawyer sends back 'a few small changes' to the draft. The file has the same name, a new version number and no tracked changes. Somewhere in its forty pages is a quietly altered indemnity, and it is your job to find it. Two kinds of tool can help, and they do different things. Mixing them up is how changes get missed.

Key points

  • A compare tool is complete and literal; an AI summary explains meaning but can omit or misread changes.
  • Run the redline first, then the summary, then reconcile them and read the key changes yourself.
  • Always compare against your own last version, and again before signing.
  • Check where text goes before using an AI tool on confidential drafts.

Two tools, two jobs

A compare tool, often called a redline or blackline tool, lines up two versions of a document and marks every difference: words added, words deleted, text moved. It does not interpret anything. Its strength is completeness. If a comma has gone, it shows it.

An AI model reads both versions and tells you in words what has changed and what it might mean. It can say that the liability cap now excludes a category of loss, or that a notice period has moved from thirty days to ten. Its strength is meaning. Its weakness is that it may skip a change, or describe it wrongly.

What a compare tool does well

  • Catches every textual change, including the small ones that matter, such as 'and' becoming 'or' or a deleted 'not'.
  • Gives you something to show a client, a colleague or the other side, in a form everyone recognises.
  • Produces the same result every time for the same two documents. It is predictable.
  • Works on the actual text, so it cannot invent a change that is not there.

What a compare tool cannot do

A redline of a heavily reworked document can be a sea of red and blue. Moved paragraphs may show as deleted in one place and inserted in another. Formatting changes and renumbering can swamp the real edits. And no compare tool tells you whether a change helps or hurts your client.

It also only compares text it can read. A scanned PDF with no text layer, or a table pasted in as an image, may not compare properly. Check what the tool did with tables, footnotes, headers and schedules, since those are the places where surprises hide.

What an AI summary does well

  • Turns a long redline into a short list of substantive changes, ordered by importance.
  • Explains in plain words how a change shifts risk, which saves a junior hours.
  • Notices when a change in one clause affects another, such as a new definition that widens an indemnity elsewhere.
  • Can compare a draft with your standard position or playbook, not only with the previous version.

What an AI summary does badly

A model can leave out a change entirely, especially a short one buried in a long document. It can smooth over a difference, calling a change 'minor' when it is not. It can misread a defined term, or state that a clause is unchanged when a single word moved. It tends to sound sure of itself, which makes omissions hard to spot.

A summary is also not a record. If a dispute arises over what was changed and when, you need the redline, not a paragraph that a tool produced.

Compare tool
  • Catches every textual change, including 'and' becoming 'or'
  • Gives the same result every time
  • Cannot invent a change that is not there
  • Gives you a record to show others
AI summary
  • Turns a long redline into a short list of changes
  • Explains how a change shifts risk
  • Can leave out a short change or call it 'minor'
  • Is not a record of what changed
Redline compare and AI summary

A workable order of operations

The two tools work best in sequence. Here is a routine that many lawyers find reliable.

  1. 1

    Run the redline first, between the last version you sent and the version you received. Save it with the date.

  2. 2

    Ask for an AI summary of the changes, giving it both versions, and ask it to list every change it sees, not only the important ones.

  3. 3

    Compare the list to the redline. Anything in the redline that the summary missed is a prompt to look harder.

  4. 4

    Read each substantive change yourself in context, with the surrounding clauses.

  5. 5

    Decide your response on each change, and note the reasons for your client.

  6. 6

    Before signing, run a final compare between the agreed text and the execution version, to catch last-minute edits.

Watch for the changes nobody marks

A counterparty who sends a clean copy rather than tracked changes may have edited things they did not mention. The same risk exists when a document is retyped or converted between formats. Always compare against your own last version, not against the other side's description of what changed.

Check the less obvious parts of the file as well: definitions, schedules, cross-references, footnotes and annexures. A changed cross-reference can quietly redirect a whole clause. Make sure someone has verified that every cross-reference still points to the right place after renumbering.

Confidentiality when comparing drafts

Draft contracts are confidential, and the document you receive may also carry the other side's confidentiality obligations. Before sending them to an AI tool, check the tool's terms on retention and training, and check whether your client or the counterparty has restricted such use. Many compare tools run locally on your own computer, which can be an advantage. Whichever you use, know where the text goes.

A small example

Suppose a lawyer receives a revised services agreement. The redline is long, mostly because the other side reformatted the schedules. The AI summary lists five substantive changes. Reading the redline, the lawyer spots a sixth: the summary did not mention that a termination right had gone from 'either party' to 'the supplier'. The tool missed it. The redline did not. That is why you use both.

Build the habit into the file. Save the redline, the summary and your own notes together under the matter, with dates. If a point is disputed months later, you can show exactly what you saw at each stage, and a junior who takes over the matter can follow the negotiation without starting again.

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next

AI summary or redline compare tool for draft changes