ChamberOne

Reviewing an associate's AI-assisted draft: a supervisor's routine

By the ChamberOne team 4 minute read

An associate hands you a research note. It reads well, it is organised, and it arrived faster than you expected. A flicker of suspicion is reasonable. Not because the associate did anything wrong, but because fluent text hides weak foundations, whether a tool or a tired person wrote it. Supervising partners and senior associates need a way of reviewing that works whether or not AI was involved, and that keeps responsibility where it belongs. This article offers a routine, with questions to ask.

Key points

  • Make it normal and safe for associates to say how a draft was made.
  • Check the authorities before the prose, and check them all if one is wrong.
  • Test the reasoning as an opponent would, and ask the associate to explain it aloud.
  • Accountability stays with the lawyer who signs, and with you for the standard of review.

Make it safe to say how it was made

The first move is cultural. If the associate fears criticism for using a tool, they will not say so, and you will review blind. Say plainly that you expect people to use approved tools, that you want to know how, and that the lawyer remains responsible either way.

Make the question routine for every piece of work, so that it does not feel like an accusation. 'How did you approach this?' is ordinary supervision.

Questions to ask first

  • Did you use an AI tool, which one and for which parts of the work?
  • What did you give it, and was that permitted by the firm's data rules?
  • What did you ask it, and may I see the prompt and the answer?
  • What did you change, and why?
  • Which authorities did you read yourself, and which did you take from the tool?
  • What are you least sure about?

Check the sources before the prose

Start with the authorities, not the argument. Pick the main authorities and open them. Do they exist? Do they say what the draft says they say? Are they from the right court, in force and relevant to the point? Do quotations match the original?

Check a sample if there are many, and weight it towards the ones the argument depends on. If one citation is wrong, check them all. A single invented or misdescribed source is a sign that the process failed, and the rest may be unsafe.

Test the reasoning

Fluent text can hide gaps. Read the argument as an opponent would. Where does it assume rather than show? Which step would a judge push on? What is the best counter-argument, and does the draft deal with it?

Ask the associate to explain the reasoning aloud without the document. If they cannot, they may have accepted a conclusion without understanding it. That is a teaching moment, not a disciplinary one.

Check the facts and the fit

Compare the draft with the file. Are the facts right, with dates and amounts? Has the tool smoothed over a fact that does not fit? Is the advice for the right client, the right jurisdiction and the right question? A good-looking answer to a question nobody asked is a common failing.

Look at tone and detail too. Generic phrasing, odd formality and padding are signs of unedited tool output. They also suggest the associate may not have thought hard about the case.

Adjust the level of review to the risk

Not every draft needs the same scrutiny. An internal note that will not leave the firm needs less checking than advice going to a client, and that needs less than a document going to court. Match the depth of review to what happens if the work is wrong.

For high-risk work, review the tool use itself: ask to see the prompt and the output, compare them with the final draft, and check what was changed. For lower-risk work, a conversation and a check of the key points may be enough. Tell associates which level applies, so they know what to expect.

Be especially alert when the work is outside the associate's usual field. A person who knows an area will spot a tool's error at once, and one who does not may not notice anything wrong.

Associate new to the field →
Be alert

A person new to an area may not notice a tool's error

Most scrutiny

Review the prompt and output and be especially alert

Lighter review

A conversation and a check of the key points may be enough

Check the tool use

Compare the prompt and output with the final draft

Risk if the work is wrong →
Matching review to risk and familiarity

Keep accountability where it belongs

The lawyer who signs or sends the work is responsible for it. Supervisors share that responsibility for the standard of review. Make this explicit: a draft is not 'the AI's' and a mistake is not 'the tool's fault'. Say it kindly and clearly, and say it before anything goes wrong.

Record what you did: a short file note saying the draft was reviewed, the sources checked and the changes made. If a question comes later, you will have a record.

Teach as you review

Use each review to build the associate's skill. Show them how you check a citation, where you look for later treatment and what you notice in a clause. Ask them to write short check notes on future work. Share anonymised examples of good and bad tool output with the team.

Over time, associates who use tools well will ask better questions and check more carefully. They should be rewarded for it, since checking is the skill that matters most.

  1. 1Ask how it was made
  2. 2Check the sources first
  3. 3Test the reasoning
  4. 4Check the facts and the fit
  5. 5Record what you did
  6. 6Teach as you review
  7. Then back to step 1.
A supervisor's routine for each draft

A short scenario

Suppose a senior associate reviews a junior's memo on a limitation point. She asks how it was made. The junior says she used a tool to outline the issues and then wrote the memo. The senior opens the three key authorities and finds that one does not say what the memo claims. She does not scold. She asks the junior to find the right authority, explains how she would have checked, and asks for a short note on the process. The memo improves, and so does the junior's habit.

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next

Reviewing an associate's AI-assisted draft | ChamberOne