ChamberOne

Why law firm AI projects fail, and what to do differently

By the ChamberOne team 4 minute read

Few AI projects in law firms fail dramatically. Most fade. A tool is bought, a launch email goes out, a handful of enthusiasts use it for a month, and then it becomes another login that nobody opens. The cause is rarely the technology. It is usually something duller: nobody agreed what the project was for, nobody owned it, the data was a mess, or fee earners had no time to learn. This article describes those four failure modes and what a partner or sponsor can do to avoid them.

Key points

  • Name a task, a team and a measure; a vague goal cannot succeed or fail.
  • Give the project an owner with time and a partner sponsor with visible commitment.
  • Tidy the data the tool will rely on before launch, and decide what must not be processed.
  • Protect learning time and teach through real tasks rather than a single training day.
  • Run a short pilot that can fail, sample the output for errors and schedule a later review.

Failure one: the goal is vague

'Use AI to be more efficient' is not a goal. It cannot be measured, and nobody can tell whether it has been met. Projects with fuzzy aims attract fuzzy tools and fuzzy enthusiasm.

A useful goal names a task, a group of people and a way of judging success. For example: 'Associates in the property team will use AI-assisted lease summaries as the first step on every new lease review, and the team will compare the time spent against the previous quarter.' That can be tested, and it can fail visibly, which is a strength.

Failure two: nobody owns it

Projects owned by 'the firm' are owned by nobody. Someone needs the job of making it work, with enough authority to change a process and enough time to do so. Often that is a senior associate or practice manager with a partner as sponsor.

The sponsor's role is different from the owner's. The sponsor removes obstacles, speaks for the project at partner meetings and uses the tool visibly. The owner runs the pilot, collects feedback and fixes problems. If one person tries to be both, one of the jobs usually gets dropped.

Failure three: the data is not ready

AI tools work on what they are given. If your precedents are scattered across personal drives, if documents are named inconsistently, or if matter details are half-recorded, the tool will reflect that. A search tool cannot find a clause in a scanned page it cannot read. A summary tool cannot reflect a version of the contract that was never uploaded.

  • Decide where the master copy of each document type lives.
  • Fix naming and matter-reference habits before you ask a tool to rely on them.
  • Check scanned documents are readable and not just images of text.
  • Clean up the most-used templates before connecting AI to them.
  • Be clear about which documents may not be processed at all, for confidentiality reasons.

Failure four: no time to learn

Fee earners are measured on billable hours, and learning a tool is not billable. If the only time for learning is lunch or evenings, only the keenest will do it. The tool then becomes the property of three enthusiasts, and everyone else concludes that it is not for them.

Set aside protected time. An hour a week for a month, written into the project plan, is more effective than an all-day training session followed by silence. Pair people, so that the person who has learned something teaches a colleague on a real task.

Do
  • Set aside protected time, such as an hour a week for a month
  • Pair people so one teaches a colleague on a real task
Don't
  • Leave learning to lunchtimes and evenings
  • Rely on one all-day training session followed by silence
Making time for fee earners to learn a tool

Starting small and on purpose

Choose one workflow, one team and one measure. Suppose a ten-lawyer firm starts with first-pass document summaries in its disputes team. For six weeks, the team uses the tool on every new bundle, records how long a summary took before and after review, and notes errors found by the reviewing lawyer. At the end, the team meets and decides whether to continue, change the process or stop.

Stopping is a legitimate outcome. A pilot designed to be able to fail is more honest than one designed to succeed, and it spares the firm the expense of rolling out something that did not work.

Questions to ask before you start

  • What specific task will this change, and for whom?
  • How will we know after six weeks whether it has helped?
  • Who owns it, and how many hours a week do they have?
  • Who is the partner sponsor and what will they do?
  • What data does it need, and is that data in good order?
  • What are the confidentiality terms and who has checked them?
  • What will we do if it does not work?

Review, not blind trust

A pilot should include a check on output quality. Pick a sample each week and have a lawyer compare the AI output to the source. Record the kinds of errors found, such as omissions, wrong dates or misread clauses. Those records are more useful than satisfaction scores, because people tend to like a tool that saves them effort even when it makes mistakes they do not notice.

Share the findings with the team honestly, including the failures. A culture where errors are discussed openly is better protected against the occasional serious one.

  1. 1Pick a sample of outputs
  2. 2A lawyer compares each with the source
  3. 3Record the kinds of errors found
  4. 4Share the findings with the team
  5. 5Repeat next week
  6. Then back to step 1.
The weekly output check during a pilot

Handling the sceptics

Every firm has a senior lawyer who thinks the project is a distraction. Do not try to win that person with enthusiasm. Ask them to test the tool on a task they know well and to report the errors. Sceptical review is a free quality check, and a sceptic who has found the limits is a more credible advocate than someone who has never looked.

Equally, watch for the partner who wants everything at once. Agree the scope in writing and defend it.

When the project works

Success is quiet. People stop talking about the tool and just use it. The process documents are updated, new joiners are taught it as part of the usual way of working, and the owner still reviews outputs and gathers feedback. Resist the urge to declare victory early and move on. Many projects slip back after the sponsor's attention shifts, so schedule a review a few months later to check that the use continues.

This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.

Read next

Why law firm AI projects fail and how to prevent it