ChamberOne

Legal operations metrics worth tracking in the AI era

By the ChamberOne team 4 minute read

Practice managers have always counted things: hours recorded, invoices sent, matters opened. When AI enters the work, the temptation is to add a new dashboard of AI measures, such as prompts sent or documents summarised. Most of those are activity counts, and activity is not the point. A firm needs to know whether work is getting to clients sooner, with fewer mistakes, at a fair price, with people used well. This article looks at four familiar measures and how AI changes their meaning.

Key points

  • Turnaround, rework, realisation and utilisation still matter, and AI changes what each one means.
  • Measure total turnaround including review, and look at the slowest cases as well as the average.
  • Watch rework closely and define it clearly without punishing people for reporting it.
  • Read measures together, check the work mix, and avoid counting activity as achievement.

Turnaround time

Turnaround is the time from instruction to delivery, for a given type of task. Pick tasks that repeat, such as a first review of a contract or a response to a new enquiry, and measure from the same start and end points each time.

AI may shorten the first draft but lengthen the review, so watch the total. Also watch the spread. An average that hides a few very late items does not tell you how clients experience your service. Report the slowest tenth of cases as well as the middle.

Rework rate

Rework is how often a piece of work comes back for correction, and how much correction. Record it when a partner changes an associate's draft materially, when a client asks for a redo, or when an error is found after delivery.

This is the measure to watch most closely when AI is introduced. A fast draft that needs heavy rework is not saving time, and a rise in errors found after delivery is a warning. Define 'material' in advance so people count it the same way. Do not turn the number into a stick for juniors. If people fear it, they will stop reporting.

Realisation

Realisation compares what you billed with what the time was worth at standard rates, and what you collected with what you billed. It shows how much work turns into money.

AI can affect it in several ways. Faster work billed by the hour reduces billable time. Clients may push back on bills for tasks they know are partly automated. Write-offs may rise if narratives look generic. Track realisation by work type so you can see where the pattern changes, and look at write-off reasons, not just totals.

Utilisation

Utilisation compares billable time with available time. If AI shortens routine tasks, a lawyer's recorded time may fall without the lawyer being any less busy. They may be reviewing, supervising or doing work that is not easily billed.

Read utilisation together with other measures. Falling utilisation with steady turnaround and quality could mean the tools are doing what you hoped, and that you need to use the freed time well. Falling utilisation with growing rework could mean the opposite.

Measures to add, sparingly

  • Time spent on review and correction of AI-assisted work, as a separate category in the time records.
  • Share of matters where AI features were used, to see where use concentrates.
  • Client questions or complaints that mention AI or narrative quality.
  • Cost per task of tools and software, compared with the same measure for people.
  • Training completion and policy acknowledgement rates.

Pitfalls when reading the numbers

Do not compare periods without checking that the work mix has not changed. A firm that takes on a few large matters will see utilisation and realisation move for reasons unrelated to AI. Do not assume that a gain in one measure came from a tool. Look for the plain explanation first.

Avoid counting activity as achievement. The number of prompts sent says nothing about value. And watch for measures that people can game. If you reward short turnaround alone, quality will slip.

Do
  • Check the work mix has not changed between periods
  • Look for the plain explanation first
  • Define 'material' rework in advance
Don't
  • Assume a gain came from a tool
  • Count prompts sent as achievement
  • Reward short turnaround alone
Reading legal operations numbers

A worked example: reading a month's figures

Suppose a five-partner firm sees turnaround on routine contract reviews fall, rework stay steady and realisation on that work dip slightly. On its own, each number is ambiguous. Put together, they suggest that the work is quicker, quality has held, and clients are paying a little less for it, perhaps because the narratives are shorter or write-offs have grown.

The practice manager checks the write-off reasons and finds that two bills were reduced after clients asked why a short task cost so much. She asks the partners whether a fixed price would suit those clients. That one question comes straight from reading the measures together, and none of them would have prompted it alone.

Notice also what she did not do. She did not credit the whole change to AI, and she did not treat the realisation dip as a failure. She asked what was behind it.

Who owns the numbers

Give each measure an owner. A practice manager may own turnaround and utilisation, the head of finance realisation, and a risk partner rework. Owners should be able to explain a movement in their number and to say what they would do about it.

Be clear about definitions and write them on one page. When the definition of 'material rework' changes, note the date, so that a jump in the figure is not mistaken for a change in performance. Consistency over a year is worth more than a clever new measure.

A routine for reviewing

Hold a short monthly review with the heads of each practice group. Look at the four measures by work type, compared with the same months last year where possible. Pick one change to investigate, ask the people involved what happened, and write down the answer. Keep the pack to a single page so that people read it.

Share a summary with the whole firm. People accept new tools more readily when they see that the firm is counting the right things and listening to what they say.

  1. 1Look at the four measures by work type
  2. 2Pick one change to investigate
  3. 3Ask the people involved what happened
  4. 4Write down the answer
  5. 5Share a summary with the firm
  6. Then back to step 1.
The monthly metrics review

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next

Legal operations metrics worth tracking in the AI era