ChamberOne

Human-in-the-loop review: designing AI workflows where a lawyer decides

By the ChamberOne team 5 minute read

Every firm using AI says a lawyer reviews the output. Ask how, and the answers vary. Someone glances at it. A trainee reads it if there is time. Nobody records that it happened. When something goes wrong, the firm discovers that 'a lawyer reviews it' was an intention, not a process. Human-in-the-loop design turns the intention into steps that can be seen, followed and audited.

Key points

  • Make AI output a visible draft until a named person accepts it.
  • Treat accept and reject as deliberate acts with recorded reasons and times.
  • Match the level of review to the risk of the output.
  • Watch for review fatigue and rubber-stamping, and build in time for review.

What the phrase should mean

'Human in the loop' is often used loosely. A useful definition for a firm is this: the AI produces a proposal, a named person examines it, and nothing leaves the firm or changes the record until that person says yes. The decision belongs to the person, and the system remembers who made it.

Three features follow from that definition. Outputs have a status that shows they are not yet approved. Approval is a deliberate act, not the absence of an objection. And the act is recorded with a name and a time.

Draft status as the default

The simplest safeguard is that every AI output starts life as a draft. A draft is visibly different from a finished document: marked, perhaps watermarked, and unable to be sent to a client or filed. Stopping accidental release matters more than it sounds. Most AI mishaps in practice are not dramatic; they are an unreviewed paragraph that made it into a letter because the file looked finished.

If you use several tools, agree one convention across them. A document marked 'AI draft' in the title, or held in a draft folder, works if everyone sticks to it. A system that enforces the status is better than one that relies on goodwill.

Accept and reject as real choices

Review is a decision, so make it a decision with two buttons. Accepting means 'I have read this and I adopt it'. Rejecting means 'I do not, and here is why'. Editing and then accepting is a third path and a common one.

Encourage reviewers to record a short reason when they reject. Over time, the reasons show where the tool is weak. If reviewers repeatedly reject the same kind of output, you have learned something about where to stop using it or how to instruct it better.

Be wary of designs that make acceptance the path of least resistance, such as a single 'Done' button or an automatic approval after a timeout. They invite rubber-stamping.

  1. 1The AI produces a draft
  2. 2A named person examines it
  3. 3Accept, edit then accept, or reject with a short reason
  4. 4The decision is recorded with name and time
  5. 5Rejection reasons show where to adjust instructions or scope
  6. Then back to step 1.
How an AI draft is reviewed

Who reviews what

Review should match the risk. A simple scheme:

  • Low risk, such as an internal summary of a long email chain: the fee earner who requested it reviews it.
  • Medium risk, such as a first draft of a client letter or a clause extraction: the fee earner reviews, and a supervisor spot-checks.
  • High risk, such as anything that states the client's legal position, goes to a court or relates to money: a qualified lawyer reviews in full, and the file records that they did.

Recording the sign-off

A record serves three purposes. It reminds the reviewer that the act matters. It lets a supervisor see what was reviewed and by whom. And it gives the firm something to show a client, an insurer or a regulator if asked how AI output is controlled.

At minimum, record who accepted or rejected, when, and what the output was. Better still, keep the version they saw, so that later edits cannot blur what was approved. If your tools record the model used and the request made, keep those too.

Common failures

  • Review fatigue. When most outputs are fine, reviewers stop looking hard. Rotate who reviews, and periodically seed a known error to see whether it is caught.
  • Authority gradient. A junior who suspects a problem with a senior's AI draft may not say so. Make challenge normal.
  • Hidden edits. If the output is changed silently after approval, the sign-off means nothing. Lock the approved version.
  • Time pressure. Deadlines compress review. Build time for it into the plan, and treat unreviewed AI output as unusable, not as a risk to be absorbed.

None of these is solved by a tool alone. They need habits, supervision and a culture where saying 'I am not sure about this' is welcome.

A short implementation checklist

  1. 1

    List the places AI output enters your work, and assign each a risk level.

  2. 2

    Define who may accept at each level.

  3. 3

    Mark all AI output as draft until accepted.

  4. 4

    Record acceptance with name, time and version.

  5. 5

    Review rejection reasons monthly and adjust instructions or scope.

  6. 6

    Check the guidance from your regulator or bar on supervision and responsibility, and align your policy with it.

A hypothetical example: the client letter

Suppose a fee earner asks an AI tool to draft a letter to a client explaining a settlement offer. The tool produces a clear, polite draft. It also states, with confidence, a timescale the fee earner never mentioned.

In a well-designed workflow, the letter is a draft and cannot be sent. The fee earner reads it, notices the timescale, removes it, edits two sentences and accepts. The record shows who accepted, when and which version. In a poorly designed one, the draft is copied into a word processor, tidied and sent. Nothing is recorded, and the invented timescale reaches the client.

The difference is not the quality of the tool. It is whether the process made the human decision unavoidable.

Well-designed workflow
  • The letter is a draft and cannot be sent
  • The fee earner reads it, removes the invented timescale and accepts
  • The record shows who accepted, when and which version
Poorly designed workflow
  • The draft is copied into a word processor, tidied and sent
  • Nothing is recorded
  • The invented timescale reaches the client
Two ways to handle the same AI draft

Starting small

You do not need to redesign everything at once. Pick one AI use that is already happening, such as summarising long documents, and add the draft status, the accept step and the record to that use alone. Run it for a month and ask the reviewers what slowed them down and what they nearly missed.

Then widen the scheme to the next use. A process that people have seen working in one place is far easier to adopt than a policy handed down in the abstract. Keep the rules short enough that a new starter can remember them after one briefing.

A lawyer decides in ChamberOne

  • Each AI output stays a draft until someone accepts it.
  • Every AI call is recorded in the audit log with the model and token counts.
  • You choose the model for each AI feature, GPT-5 or Claude, or switch AI features off for the firm.

What it does not do

  • AI output can be wrong or incomplete and does not replace a lawyer's review.
  • It does not give legal advice or decide who in your firm should review what.
  • With AI features on, the text needed for a request is sent to the AI provider for that feature, outside the country where ChamberOne is hosted.

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next

Human-in-the-loop review for legal AI workflows | ChamberOne