Tax research and AI: where it goes wrong
By the ChamberOne team 5 minute read
Ask a chatbot for a tax rate and you will get an answer in a second, delivered with complete calm. It may be right. It may be the rate from three years ago, the rate from a different country or a blend of the two. Tax is the area of law where the cost of a small factual slip is easiest to measure, and where general AI tools are least reliable on exactly the details that matter.
Key points
- Tax rules change often and differ by place, so general AI answers go stale or go wrong.
- Confirm every rate, threshold, deadline and reference at an official source for the right period and place.
- Use AI to explain, summarise official text you supply, and draft structure, not to supply figures.
- Record the source and date for every figure in the file.
Why tax is a hard subject for a language model
A language model learns from text collected up to a certain point. Tax legislation, guidance and rulings change frequently, and often at fixed times of the year. A model can therefore describe a rule that has since been amended, repealed or replaced, and it has no built-in way to know that.
Tax is also full of numbers: rates, bands, thresholds, allowances, filing dates and penalty amounts. Models are weaker at exact numbers than at general explanations. They may blend figures from different years or different places, and the result reads naturally, so the mistake does not announce itself.
Five typical failure patterns
- Stale rule: the answer was correct once, and has since been changed.
- Wrong place: the answer describes another country, state or province. Local tax systems often share terminology while differing in substance.
- Wrong period: a threshold or relief that applied to one tax year is given as if it applied to another.
- Missed condition: the rule is stated correctly, but the exceptions and anti-avoidance provisions are left out.
- Invented authority: a section number, ruling or case that sounds right and does not exist, or does not say what is claimed.
Regional differences within one country
Many places have layers: national, regional and local taxes, with different authorities collecting each. Stamp or transfer duties, property taxes, payroll taxes and indirect taxes can all vary between regions. A general answer about 'the' rule may be correct for one region and wrong for the one where your client lives.
When you ask a question, name the place, the tax, the tax period and the type of taxpayer. Then check that the answer reflects them. If it answers in general terms, treat that as a warning.
What AI is useful for in tax work
None of this means the tools are useless. Used for the right tasks, they save time.
- Explaining a concept in plain language so you can brief a client or a junior.
- Summarising a long piece of official guidance that you have supplied, so that you can decide what to read in full.
- Producing a first-draft checklist of questions to ask a client at the start of a matter.
- Drafting an email or an advice note structure, which you then fill in with checked facts.
- Spotting inconsistencies between a client's documents, such as dates or amounts that do not match.
A verification routine for every number and date
Adopt a rule that no rate, threshold, deadline or penalty enters advice because an AI tool said it. Each is confirmed at an official source. The routine is quick once it becomes habit.
- 1
Note every figure, date and legal reference in the draft.
- 2
Find the primary source: the legislation as currently in force, and the tax authority's current guidance.
- 3
Check the version date of the source, and check that it covers the tax period in question.
- 4
Look for later amendments, transitional rules and any announcement that has not yet taken effect.
- 5
Record the source and date in the file note, so that someone can retrace it later.
- 1Note every figure, date and reference
- 2Find the primary source
- 3Check the version date and tax period
- 4Look for later amendments
- 5Record the source and date
- Then back to step 1.
Use the tool on your own material
The safest way to use AI on tax is to give it the official text and ask it to work on that. If you paste in the current guidance and ask for a summary, the model works from what is in front of it rather than from memory. This does not remove the need to read the source, but it makes the output far more reliable, and it keeps the answer tied to a document you can cite.
Be wary of tools that claim to have 'up to date' tax knowledge without saying where it comes from. Ask how the tool is updated, how often, and from which sources. If the vendor cannot say, assume it cannot be relied on for current figures.
- Learned from text up to a certain point
- Weaker at exact numbers
- 'Up to date' claims need a named source
- Works from what is in front of it
- Output is far more reliable
- Answer stays tied to a document you can cite
- You still read the source
Professional responsibility and client expectations
Advice on tax carries consequences for clients: penalties, interest, missed reliefs and sometimes allegations of avoidance. The adviser is responsible for the advice, whatever tools were used. Check the guidance from your regulator or bar, and from any professional tax body you belong to, about technology and competence.
It also helps to be open with clients. A short line in the engagement letter about how technology is used, and that every figure is checked against official sources, tells clients what to expect and sets a standard you can keep to.
A short case to illustrate
Suppose a small advisory firm asks an AI assistant about the tax treatment of a share transfer for a client who has recently moved between two countries. The answer is clear, tidy and gives a figure. A careful associate checks it, and finds that the answer applies to a resident taxpayer, that a relief mentioned had changed, and that the figure belonged to a previous period. None of that was obvious from the text. The associate corrects the note and tells the partner what happened.
That is the right outcome, and it shows the right attitude: the tool is a quick first draft of the thinking, and the checking is where the professional work lies.
One more habit helps: keep a short list of the sources your team trusts for each tax in your market, with links, and share it with new joiners. When everyone checks against the same pages, disagreements about a figure are settled in minutes, and the firm builds a shared sense of which sources are current and which are only commentary.
This article is general information about the law, legal practice and technology, current when it was published. It is not legal advice. Laws and guidance change: check the sources linked above and take advice on your own circumstances from a lawyer qualified where you are.
Read next
How to check AI legal research before you rely on it
A routine for tracing every AI-suggested authority to its source, confirming it is still good law and testing the proposition it is cited for.
5 minute readGeneral chatbots versus legal research tools: what differs and why it matters
How general chatbots and legal research tools differ on sourcing, links, jurisdiction coverage and currency, and when a general model will do.
5 minute readRetrieval-augmented generation explained for lawyers
A plain explanation of how AI answers from a document set, why it reduces but does not remove fabricated content, and what to ask vendors.
5 minute read