Cross-jurisdiction research with AI: when a model mixes up the law
By the ChamberOne team 5 minute read
Ask an AI assistant a question about limitation periods, privilege or the effect of a contractual penalty clause, and you will often get a clear, well-organised answer. Look closely and you may find it describes the law of a different country. The English, the American and the local position get folded into one paragraph, with no seam showing. For lawyers who advise across borders, or who work in a jurisdiction whose law is a small part of what the model has read, this is the central risk.
Key points
- Models blend legal systems and may answer about the wrong country without saying so.
- State the jurisdiction, area and date in every prompt, and ask for the source of each point.
- Confirm each proposition in a local primary source, with the version date.
- For cross-border matters, work out the governing law for each issue before asking.
Why models blur jurisdictions
A language model learns patterns from a very large body of text. Some legal systems are far better represented in that text than others. American material is plentiful and English material nearly so. Commonwealth systems share a great deal of vocabulary and history, so an answer about one reads convincingly like an answer about another. Smaller jurisdictions are thinly covered.
The model does not hold a separate shelf for each country. When your question does not pin the jurisdiction, or when local material is sparse, it fills the gap with whatever pattern is strongest. The result is not random. It is plausible, which makes it more dangerous.
What the errors look like
Most mistakes fall into a few recognisable types.
- Wrong system. A rule from one country presented as the rule in yours, such as a federal American doctrine described as if it applied in a Commonwealth court.
- Right idea, wrong detail. The general principle exists locally, but the time limit, threshold or formality differs.
- Out-of-date law. The answer reflects a version of the law before an amendment, because older material outnumbers newer.
- Borrowed terminology. A term of art from one system used for a similar but different concept in another.
- Invented local authority. The model supplies a plausible-sounding local case or section because it was asked for one.
Prompting for jurisdiction
You cannot prompt your way to reliability, but you can make errors less likely and easier to spot.
- 1
Name the jurisdiction in the first line, and the area of law. Name sub-jurisdictions where they matter, such as a state, province or court.
- 2
State the date as at which you need the law to apply.
- 3
Ask the model to say which jurisdiction each point comes from, and to flag any point that is general or borrowed from elsewhere.
- 4
Ask what it is unsure about. Models can sometimes identify the weak parts of an answer when asked directly.
- 5
Tell it not to cite authority it cannot supply from text you provide, and say that you will check every reference.
- 6
If you have the statute or judgment, paste the relevant text in and ask questions about that text only.
Cross-checking with local sources
Treat every proposition as unverified until you have seen it in a local primary source: the current legislation on the official site, the judgment on a recognised law report service or court website, or a practitioner text for your jurisdiction. Then confirm the date of the version you are reading and whether later amendments exist.
A useful discipline is the two-source rule. Do not rely on a point until you have found it in a primary source and, where practical, in a reputable secondary one written for that system. If the two disagree, go with the primary source and work out why they differ.
Keep the model's answer next to the sources, and mark each statement as confirmed, corrected or unresolved. That marked-up note is more valuable to the next reader than the AI text itself.
- Treat every proposition as unverified until seen in a local primary source
- Confirm the date of the version you are reading
- Look for a reputable secondary source written for that system
- Mark each statement confirmed, corrected or unresolved
- Rely on the model's answer alone
- Prefer a secondary source when it disagrees with the primary source
- Let the AI text stand in for the sources
Special care for comparative and cross-border work
Cross-border matters create a particular trap. A client's contract may be governed by one law and enforced in another. Different issues within the same dispute can be governed by different laws. A model asked about the dispute generally may blend them.
Break the question up. Identify the governing law for each issue first, using the contract and conflict-of-laws rules, which is a human task. Then ask separate, jurisdiction-specific questions. Where you instruct local counsel, give them the AI output only as a lead and ask them to confirm it from their own sources.
A hypothetical example
Suppose a Singapore-based associate asks an assistant whether a non-compete clause in a services agreement is likely to be enforceable. The answer cites a mix of English and American reasoning, uses a test from one of them as though it were settled local law, and mentions a statutory provision that, on checking, belongs to another jurisdiction entirely.
The associate has not wasted the time. The answer listed useful factors to consider. But none of it could go into advice until each factor was found in local authority. Had the answer gone straight into a memo, the client would have been told something about the wrong country's law.
Making it a firm habit
Add a line to your research template: 'Jurisdiction and date checked against primary source: yes or no.' Add a similar field in your AI policy. Train juniors to ask 'whose law is this?' of every answer. Where a firm practises in several countries, keep a list of recommended local sources for each, so that the cross-check is quick.
Check the guidance from your regulator or bar on competence and on using technology in research. A firm that can show it verified AI output against local sources is better placed than one that cannot.
Questions to ask a vendor
If a firm is buying a research tool rather than using a general assistant, ask the vendor in writing which jurisdictions it covers, from which sources, and how often those sources are updated. Ask whether answers link to the primary text, and whether the tool says so when it has no local material rather than answering from another system.
Ask for a demonstration on a question from your own jurisdiction where you already know the answer, including a question on a recent amendment. How the tool copes with something you can verify is the best guide to how it will cope with something you cannot.
This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.
Read next
How to check AI legal research before you rely on it
A routine for tracing every AI-suggested authority to its source, confirming it is still good law and testing the proposition it is cited for.
5 minute readGeneral chatbots versus legal research tools: what differs and why it matters
How general chatbots and legal research tools differ on sourcing, links, jurisdiction coverage and currency, and when a general model will do.
5 minute readRetrieval-augmented generation explained for lawyers
A plain explanation of how AI answers from a document set, why it reduces but does not remove fabricated content, and what to ask vendors.
5 minute read