Measuring AI return on investment in a law firm
By the ChamberOne team 4 minute read
At some point a partner will ask, 'Is this AI thing worth what we pay for it?' The easy answers are the unreliable ones: a vendor's claim of hours saved, a keen associate's impression, a guess multiplied by the number of fee earners. A firm that wants a real answer has to do a little unglamorous work. It needs a baseline, a measure of the whole task including checking, and a willingness to hear that some uses do not pay. This article sets out how, for partners and finance managers.
Key points
- Decide what you are trying to improve and pick a few repeatable tasks to measure.
- Record a baseline for time and errors before starting.
- Count prompt writing, checking and correction as part of the task with AI.
- Make narrow claims you can trace, and repeat the measurement each quarter.
Decide what you are trying to improve
Return on investment means different things to different firms. For some it is lawyer time saved on routine work. For others it is faster turnaround for clients, fewer errors, or the ability to take on work that was previously too costly. For a firm that bills by the hour, saved time may even reduce revenue, so you must be clear what you want.
Pick one or two goals for the first measurement. Trying to measure everything at once produces numbers nobody trusts.
Choose a baseline before you start
You cannot say a task got quicker without knowing how long it took before. Pick a small set of repeatable tasks, such as reviewing a standard lease, summarising a bundle or preparing a first draft of a routine letter. Record how long they take now, who does them and how often they need correction.
Use your time records if they are reliable, and sample if they are not. Be honest about how rough the baseline is. A rough baseline measured consistently is still more useful than a precise guess.
Count review time as a cost
This is where many calculations fail. A tool may produce a draft in seconds, but a lawyer must read it, check it against the source, correct it and sometimes redo parts. That time belongs in the cost of the task with AI.
A fair comparison is the total time from start to approved output. Include prompt writing, checking citations, fixing errors, and any time spent explaining the tool to colleagues. Include also the cost of the tool itself and of training. If the total is lower than the baseline, you have a gain. If not, you do not, however fast the draft appeared.
- A draft in seconds
- A quick first pass
- Prompt writing and checking citations
- Fixing errors and redoing parts
- Explaining the tool to colleagues
- The cost of the tool and of training
A simple method per task
- 1
Pick the task and write down what a finished version looks like.
- 2
Record baseline time and error rate on five to ten recent examples done without the tool.
- 3
Do a similar set with the tool, recording total time including checking and correction.
- 4
Have a senior lawyer compare the quality of both sets without knowing which is which.
- 5
Compare total time and quality, and note anything the tool made worse.
- 6
Decide whether to use it for that task, use it with limits, or stop.
Measure what changed, not what you hoped
Look beyond hours. Did the turnaround time to the client improve? Did the number of corrections from partners change? Did a junior take on work they could not have done before? Did clients comment? These are harder to count, but they may be where the real value is.
Also record the costs that sit outside the tool: more time spent supervising juniors, new risk checks, extra training. If the gain is real, it will survive being counted properly.
Avoid claims you cannot trace
Avoid statements such as 'AI saves each lawyer an hour a day' unless you can show where the hour came from. Such claims spread, get quoted to clients and prospective hires, and then become embarrassing. Prefer narrow statements: 'on routine lease reviews in our property team over the last quarter, total time fell, and partners rated quality as similar'.
If you quote results to clients or in marketing, be sure your regulator's advertising rules are satisfied. Check the guidance from your regulator or bar.
A worked comparison
Suppose a four-partner property firm wants to test AI on residential lease reviews. Over a month it records the time on eight recent reviews done by hand. It then does eight similar reviews with a tool. The tool's first pass is quick, but the associates spend time checking every extracted date and clause against the lease, and two reviews need a partial redo because a clause was misread.
When the firm adds the checking and redo time, the total per review is lower than before, but by less than the headline speed suggested. Partners compare the finished reviews and see no drop in quality. The firm concludes that the tool pays on standard leases and not on unusual ones. That is a modest, useful, traceable result, and it tells the firm where to use the tool and where not to.
Notice what the firm did not do. It did not multiply a minutes-saved figure by every fee earner. It did not count the draft as the finished product. And it did not hide the two reviews that went wrong.
Costs people forget
- Subscription or usage charges, including growth as more people use it.
- Time spent choosing the tool, testing it and writing the policy.
- Training and the time lawyers spend learning to write good instructions.
- Extra supervision of juniors who now produce more drafts to review.
- Any change to insurance, security review or client approval processes.
Reviewing over time
Results change. A tool improves, a team becomes better at using it, or its provider changes behaviour. Repeat the measurement each quarter for the tasks that matter, and keep the records. Share the results with the team, including the failures. A firm that talks openly about what did not work will find it easier to adopt what does.
- 1Measure the tasks that matter
- 2Keep the records
- 3Share results, failures included
- 4Repeat the measurement each quarter
- Then back to step 1.
This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.
Read next
How a small law firm can start with AI without a big project
Pick two low-risk uses, set review rules and measure before expanding, with a list of what to postpone.
5 minute readA 90-day AI pilot plan for a law firm
A week-by-week plan to choose a use case, set measures, run a small trial, review errors and decide.
5 minute readChoosing AI models and firm-level controls: what a practice should decide
Model choice, an off switch and a usage record are three decisions a firm should make deliberately.
4 minute read