The fear is well founded and the examples are real. AI systems can produce text that is fluent, confident and entirely invented — a case that does not exist, a section number that is wrong, a date that was never true. In a profession where a fabricated citation is a serious matter, this is not a small concern.
But the risk is frequently misunderstood, and the misunderstanding leads solicitors either to reject AI wholesale or to use it in exactly the way most likely to cause a problem.
Why it happens
A language model generates text by predicting what should come next. It is extraordinarily good at producing writing that reads correctly. It has no independent mechanism for checking whether what it has produced is true.
The consequence matters: fluency and accuracy are separate properties. A confident, well-structured, professionally-worded paragraph carries no more guarantee of being right than a hesitant one. Human writing does not work this way, which is why the failure mode catches people out. We are trained to read confidence as a signal of competence.
AI is reliable at transforming information you give it, and unreliable at recalling information you did not. Almost every practical control follows from that one distinction.
Where the risk is high — and where it is not
Not all AI tasks carry the same exposure. It is worth separating them.
Higher risk: asking it to recall
“What does section 62 say?” “Find me a case on this point.” “What is the current threshold?” These invite the model to produce specifics from memory, and memory is exactly what it does not reliably have. This is where fabricated citations come from.
Lower risk: asking it to transform
“Summarise this document I have given you.” “Turn these notes into a draft attendance.” “Restructure this letter to follow our house format.” “Extract every date mentioned in this contract.” The source material is in front of it. The task is transformation, not recall. This is where the reliable time savings live, and it is most of the admin burden in a practice.
A great deal of the anxiety about hallucination comes from people trying the first category, being burned, and concluding the technology is unusable — when the second category is where the value was all along.
The controls that contain it
- Supply the source material. If the answer should come from a document, give it the document. Do not rely on recall.
- Verify every specific. Citations, section numbers, dates, figures and names get checked against the primary source. Always, without exception.
- Prefer transformation tasks. Build workflows around summarising, restructuring, extracting and drafting from supplied material.
- Treat output as a trainee’s first draft. Useful, fast, and never sent unread.
- Ask it to cite its source from the supplied text. If it cannot point to where in the document something came from, that is a flag.
The comparison that puts it in proportion
A capable trainee produces a first draft quickly and sometimes gets something wrong. That is not an argument against trainees. It is an argument for review — which every practice already does, as a matter of routine, without regarding it as a burden.
The same discipline applied to AI output is sufficient. What is not sufficient is treating AI output as finished work because it reads like finished work.
The short version
AI makes things up when asked to recall. It is dependable when asked to transform material you have supplied. Build your workflows around the second, verify every specific in the first, and keep a solicitor reviewing before anything leaves the office. The risk becomes manageable — not because the technology stopped hallucinating, but because your process assumes it might.
See where the hours are going
A free 15-minute AI admin audit. We look at your actual week and tell you which tasks are worth automating — and which aren’t.
Book Your Free Audit →