Semantic search versus keyword search for legal documents
By the ChamberOne team 5 minute read
Search for 'termination' in a pile of contracts and you will find every document that uses the word. You will miss the one that says 'the agreement shall come to an end', and the one that talks about 'ceasing to be bound'. Semantic search tries to close that gap by matching meaning instead of spelling. It helps, but not everywhere, and a lawyer who understands the difference gets better results from both methods.
Key points
- Keyword search is literal and predictable; semantic search matches meaning and is flexible but fuzzy.
- Use semantic search to explore a concept, then learn the actual words and search for them exactly.
- Use exact search for names, numbers, citations and defined terms, and where missing a document is costly.
- Test any tool on questions you already know the answer to before relying on it.
- Check that access limits apply to search results and summaries.
What each method actually does
Keyword search looks for the characters you typed. Good systems add refinements: exact phrases, wildcards, proximity operators and Boolean logic, so that you can look for 'indemnif*' within ten words of 'third party'. The method is predictable. If a document contains the term, it will be found, and if it does not, it will not.
Semantic search converts text into numerical representations that capture something of its meaning, and then looks for passages whose meaning is close to your query. You can ask a question in ordinary words and get passages that answer it, even if they share no words with the question. The method is less predictable. It returns the nearest matches, whether or not any is truly relevant.
Where meaning-based search wins
It is strongest when you know the idea but not the wording. Lawyers rarely know how a drafter phrased a concept in a document written ten years ago.
- Finding a clause by its function: 'which of our precedents let a buyer walk away if a key person leaves?'
- Searching across documents drafted by different people, firms or countries, where the same concept is expressed in different words.
- Looking for earlier advice on a problem described in everyday language, in a firm knowledge bank.
- Locating the part of a long document that deals with a topic, when you do not know its heading.
- Handling spelling variations, synonyms and the loose language of emails and attendance notes.
Where exact terms still win
Law is full of words that must be matched exactly. A statute section number, a defined term, a party's name, a case reference, a precise phrase from an order: if you are looking for those, a keyword search is the right tool, and semantic search may bury the exact match under looser ones.
Exact search also has the advantage of being checkable. When a keyword search returns nothing, you know that the term does not appear. When a semantic search returns ten passages, you cannot be sure that the eleventh, which you did not see, was not better. For disclosure exercises and conflict checks, where missing a document matters, that difference is serious.
The trade-offs in plain terms
Neither method is better in general. Keyword search is precise but literal, and semantic search is flexible but fuzzy. A search that returns too few results suggests you need to widen it, and one that returns too many suggests narrowing it. Each method fails in a different way, and the skill is in noticing which kind of failure you are facing.
Semantic systems also depend on how documents are broken into passages, how the meaning is calculated and what the system was built on. A model trained on general text may understand common-law contract language well and local statutory terms badly. Test it on your own material before you trust it.
- Looks for the characters you typed
- Precise but literal
- Checkable: no results means the term is absent
- Right for section numbers, defined terms and names
- Looks for passages close in meaning
- Flexible but fuzzy
- Lets you ask in ordinary words
- You cannot be sure what it missed
Combining the two in practice
Many modern tools offer both, sometimes blended in a single result list. A sensible workflow uses each for what it does well.
- 1
Start with a semantic query to explore: describe the concept in plain words and look at what comes back.
- 2
Note the actual words used in the best results, then run exact keyword searches on those terms to find more documents with the same wording.
- 3
Use exact searches for names, numbers, citations and defined terms.
- 4
Repeat the search with a different phrasing and compare. If the results differ widely, your question may be unclear.
- 5
Read the passages themselves. Do not rely on a summary of the results.
- 1Explore with a semantic query
- 2Note the words used in the best results
- 3Run exact keyword searches on those terms
- 4Rephrase and compare the results
- Then back to step 1.
Testing a search tool before relying on it
Pick ten questions where you already know the answer: a clause you know exists in a certain document, an old advice note on a certain topic, a letter containing a certain term. Run each through the tool and see whether it finds what you know is there, how high it ranks, and what junk comes with it.
Then add a few trick queries: a concept described in unusual words, a rare name, a misspelling. The aim is to learn the tool's habits. Does it find the clause when the wording differs? Does it miss a document that includes the exact name? A short test like this teaches more than a demonstration.
Access, confidentiality and knowledge management
Any search over firm documents must respect confidentiality. If a matter is restricted to a certain team, results from it should not appear for anyone else, whether the search is by keyword or by meaning. Ask a vendor how access limits apply to the index and to any summaries, and test it with a restricted document and a user who should not see it.
For knowledge managers, the quality of the index depends on the quality of the documents. Old drafts, duplicate versions and unlabelled precedents will all be found. A search tool is a good prompt to clean up what the firm calls its precedent bank, and to mark the documents that are approved models.
A small example
Suppose a knowledge manager at a mid-sized firm wants to find every precedent that deals with a seller's duty to disclose defects. A keyword search for 'disclose' returns hundreds of documents, most about something else. A semantic query phrased as a question surfaces the five most relevant clauses, two of which never use the word. The manager reads them, notes that one uses 'make known', and runs an exact search on that phrase to find the rest.
A rule of thumb
If you know the words, search for the words. If you know only the idea, search for the idea, then learn the words and search again. If missing a document would cost you something, use more than one method, and write down what you searched for so that the work can be repeated.
This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.
Read next
How to check AI legal research before you rely on it
A routine for tracing every AI-suggested authority to its source, confirming it is still good law and testing the proposition it is cited for.
5 minute readGeneral chatbots versus legal research tools: what differs and why it matters
How general chatbots and legal research tools differ on sourcing, links, jurisdiction coverage and currency, and when a general model will do.
5 minute readRetrieval-augmented generation explained for lawyers
A plain explanation of how AI answers from a document set, why it reduces but does not remove fabricated content, and what to ask vendors.
5 minute read