AI is giving more people the ability to question legal advice and prepare their own arguments. That’s a good thing, surely? But recent cases in British courts raise some uncomfortable questions about the difference between producing something that looks credible and knowing whether it actually is.

In July 2026, something rather extraordinary happened at the Employment Appeal Tribunal. A man representing himself submitted a 300-page legal argument, containing almost 132,000 words, which he had produced using ChatGPT. It arrived just 2 days before the hearing, leaving the opposing parties with very little time to examine it.

132,000 words is longer than many novels. The tribunal normally expects this kind of document, known as a skeleton argument, to run to somewhere between 5 and 15 pages. And here was an individual, without professional legal representation when he prepared it, able to produce that amount of material and submit it as a legal argument.

The problem was that the document was largely unusable.

In Hancox v Sutherland & Others [2026] EAT 139, published in September, the judge explained that the submission failed to comply with the tribunal’s procedural requirements and had made it harder to identify the relevant legal issues. The individual accepted that much of the document lacked credibility, and it wasn’t relied upon at the hearing. With help from a lawyer through a free advice scheme, the hearing concentrated on a single arguable ground of appeal.

The judge was clear that there’s no general ban on using generative AI to prepare tribunal documents. He was equally clear that a document like this one would probably be rejected in future.

And I think there’s something quite revealing about that.

We’re reaching a point where producing 132,000 words of apparently professional material can be considerably easier than understanding whether any of it is relevant, accurate, or useful. AI can make the process of producing information astonishingly quick, and the finished result can look incredibly impressive. But the ability to produce something and the ability to judge its quality are 2 very different skills.

It’s an issue I’ve been thinking about for some time, particularly through the work we do helping businesses adopt AI responsibly. The technology has become remarkably good at presenting information in ways that appear complete and authoritative. It can produce a detailed report, a business proposal, a policy, or a legal argument with carefully organised headings and language that sounds as though it came from someone who knows exactly what they’re talking about.

And because it looks finished, there’s a temptation to treat it as finished.

The legal profession is providing some particularly interesting examples of why that assumption deserves challenging.

When clients start questioning the experts

One of the things I love about AI is its ability to help people understand subjects that might previously have felt completely inaccessible. Legal terminology can be confusing, professional advice can be difficult to interpret, and understanding the reasoning behind a solicitor’s recommendations isn’t easy if you have no legal background yourself.

Now someone can take an unfamiliar legal concept, ask AI to explain it in everyday language, explore what different terms mean, and prepare questions before speaking to their solicitor. They can develop a better understanding of their circumstances and perhaps have a more informed conversation about the decisions affecting them.

I think that’s a very positive development. People should feel confident enough to question professional advice, ask for explanations, and understand what they’re paying for. Having a qualification or professional title shouldn’t mean that your work is beyond scrutiny.

But there’s a growing problem when those questions turn into elaborate AI-generated legal arguments, particularly when the person presenting them doesn’t have the knowledge to establish whether the arguments are correct.

On 21 October 2026, the Law Society of England and Wales published a new 58-page practice note on using AI in legal practice. Most of it deals with how firms use AI themselves, but it also recognises that clients are now arriving with AI-generated material of their own. Solicitors are advised to tell clients how the firm is using the technology, and they can ask clients to say when something they’ve handed over was produced by AI.

It isn’t hard to see why. Large volumes of AI-generated material need to be reviewed, corrected, or explained, and that creates additional work for solicitors. I wouldn’t be surprised to see firms addressing this in their terms of engagement, including explaining where reviewing substantial amounts of AI-generated content could result in additional charges.

And I can understand the concern.

If a solicitor has spent time researching a matter and providing carefully considered advice, only to receive 50 pages of AI-generated objections containing incorrect interpretations, irrelevant case law, or references to legislation from another country, somebody has to work through that material and establish what’s actually relevant.

The person using AI might have produced those 50 pages in minutes. The solicitor could spend hours explaining why much of it doesn’t apply.

That’s an interesting reversal of the productivity argument we so often hear about AI. The person generating the material has saved themselves time, while potentially creating considerably more work for somebody else.

But I think the profession needs to be careful about how it responds. There’s a real risk that reasonable questions from clients get lumped together with poorly researched AI-generated arguments. A client who uses AI to understand their legal position and arrives with 5 sensible questions could actually make a solicitor’s job easier. They may be better prepared, more engaged, and more capable of making informed decisions.

The challenge is helping people understand the difference between using AI to explore a subject and using it to make claims they cannot substantiate.

When the legal argument looks right but the law is wrong

Perhaps the most concerning examples come from cases where AI-generated material has made its way into court proceedings without the appropriate checks.

In June 2025, the High Court examined 2 cases involving false legal authorities in Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin).

In the Al-Haroun case, 45 legal citations had been put before the court in witness statements and correspondence. Of those, 18 referred to cases that didn’t exist at all. Many of the remaining references either contained quotations that weren’t actually in the judgments or failed to support the arguments being made.

The client acknowledged using publicly available AI tools alongside other research sources. His solicitor had relied on that material without independently verifying the authorities. The court found it extraordinary that a lawyer was relying on a client for the accuracy of legal research, when it should be the other way around.

Just consider that for a moment. Evidence had been put before the High Court, complete with references to supposedly authoritative judgments, and a significant proportion of those cases simply didn’t exist.

The documents might have looked like legal research. They might have followed the structure and language you’d expect from legal documents. They certainly contained plenty of references.

But the underlying legal work hadn’t been properly verified.

And this problem extends to the professionals themselves.

In May 2026, another High Court judgment, Cork and another v Smith [2026] EWHC 1199 (Ch), examined how a law firm had sent correspondence to the court containing an invented quotation from insolvency legislation.

A junior solicitor had used the firm’s own AI tool to research and prepare the correspondence. The generated material included wording presented as though it came directly from a particular legal rule, even though the wording didn’t exist in that rule.

The detail that stays with me is that the AI tool itself had told the solicitor it was “not fully confident” in its answer and advised checking the legislation before anything went to the court. That check wasn’t carried out. The correspondence was approved and sent, with the error passing through the firm’s supervision process. The supervising solicitors later said they hadn’t known AI had been used.

When the court questioned the wording, AI was used again to help prepare an explanation, which created further problems. The judge concluded that the court had been misled twice, and the firm referred itself to the Solicitors Regulation Authority.

There’s something quite uncomfortable about that example, because it shows how easily people can become reliant on the apparent competence of the tool they’re using. Even when the information could have been checked against the original legislation, and even when the tool raised the need for verification, the generated answer was allowed to progress.

The case also raises an important point about how we delegate work. A junior member of staff would ordinarily be expected to undertake research, check their sources, and explain their findings to a more experienced colleague. Their supervisor would then have responsibilities to review that work appropriately.

Introducing AI into that process changes how the research can be produced, and potentially how quickly it can be produced. The requirement for competent supervision remains.

AI can produce the document, but it cannot take away your responsibility for getting it right.

And in law, that responsibility can have some very serious consequences.

What the UK legal profession is doing about it

The legal profession is now having to address these issues more formally.

On 17 August 2026, the Solicitors Regulation Authority issued a warning notice about the misuse of AI, raising concerns about fabricated legal authorities, incorrect information, failures in supervision, and the handling of confidential client information. It followed 42 reports of potential AI misuse in the 12 months to July 2026.

The regulator made clear that solicitors remain accountable for the work they produce, regardless of whether AI has been involved. It also drew attention to the risks of entering sensitive client information into AI services without the necessary safeguards.

That last point is particularly relevant when we consider clients using publicly available chatbots to question their legal advice. Uploading correspondence, contracts, personal information, or privileged legal advice into an unsuitable AI system can introduce confidentiality risks that the individual might never have considered.

And then, on 29 October 2026, the Civil Justice Council published its final report on the use of AI in preparing court documents.

The report makes 15 recommendations. Witnesses of fact would declare that AI hasn’t generated or influenced the substance of their statement. Experts would declare where it has, with enough detail about the tool and the method for the court to scrutinise it. There’s also a call for clear guidance for people representing themselves. For legal representatives preparing arguments and other advocacy documents, the report relies on existing professional duties, with no general requirement to declare every use of AI.

I find that approach quite sensible. AI is becoming part of everyday software, and trying to control its use through blanket restrictions could become increasingly difficult. What matters is the effect that its use has on the accuracy and integrity of the material being produced.

There’s also a meaningful distinction between using AI to help organise a document and allowing it to alter the substance of somebody’s evidence. A witness statement needs to reflect what the witness actually remembers and knows. If AI starts filling gaps, strengthening statements, or making an account sound more certain than the witness really is, the consequences can go well beyond a poorly written document.

The technology can help with the mechanics of preparation. The people involved must still understand and stand behind the content.

And importantly, the Civil Justice Council’s proposals are recommendations. They’ll need to be considered by the Civil Procedure Rule Committee before any changes to procedural rules take effect.

We need to become better at judging what AI produces

For me, this is where the wider lesson sits, and it applies well beyond the legal profession.

I spend a lot of time working with businesses and individuals to help them understand how to use AI effectively. One of the principles I keep returning to is that using AI is fundamentally a communication and delegation skill.

If you give somebody a job to do, you need to communicate what you’re trying to achieve, provide the necessary context, and establish what a successful outcome looks like. You also need to know when to review their work, when to challenge something, and when the task requires somebody with specific knowledge or experience.

The same principles apply when working with AI.

If you ask AI to research a legal matter, you should understand what information it’s working from, whether the sources are reliable, and whether the answer actually addresses the question. If it provides a case citation, that citation needs to be verified. If it offers an interpretation of legislation, the actual legislation needs to be checked. And if you’re relying on the output to make an important decision, you need to be confident that somebody with the appropriate expertise has assessed it.

There’s another aspect to this that I think is particularly important.

Something can be factually correct and still be completely wrong for the situation in which you’re trying to use it.

An AI-generated legal argument might refer to real cases and correctly quote legislation, yet still misunderstand the facts, overlook an important exception, or fail to recognise that a judgment doesn’t apply to the circumstances being considered.

That’s a much harder problem to spot than a completely invented case citation, because the individual components might all be real. The weakness sits in how they’ve been interpreted and brought together.

And the more convincingly AI can explain that interpretation, the easier it becomes to accept the result.

Thomas Chartres-Moore, Commercial Services Partner, Chair of the AI Council, Stephens Scown

One of the things that prompted this article was something shared with us by Stephens Scown, the law firm whose AI council I sit on. After writing it, I reached out to the firm, and Thomas Chartres-Moore, a Commercial Services Partner at Stephens Scown and the Chair of the AI Council, offered an analogy that I think gets right to the heart of the problem:

AI can reproduce the architecture of legal advice remarkably well. What it cannot necessarily reproduce is the judgment that determined why it was built that way in the first place. It learns from patterns and can therefore create a convincing legal edifice: clauses, caveats, risks and recommendations that look reassuringly familiar. Humans are hardwired to spot patterns and so will view output that looks right as if it is right.

In that sense, AI can be a brilliant legal set designer. From a distance, everything looks right: the doors, the windows, the switches and the wiring. But interrogate the structure and sometimes the doors do not open and the lights do not work. It has reproduced the appearance of something it has seen before without necessarily understanding how or why it works.

That is where experience and judgment matter. Good legal advice is not about identifying everything that could go wrong. It is about understanding which risks actually matter, balancing risk with reward and how each are likely to play out in the real world and what a client should do about them. AI can build a remarkably convincing set. The lawyer’s job is to know whether the building actually works.

It’s one of the reasons I’m concerned about the amount of attention we give to teaching people how to generate better outputs, compared with the attention we give to teaching them how to evaluate those outputs.

We’re becoming very good at showing people how to produce detailed reports, impressive presentations, and professional-looking documents. We should be putting equal effort into helping them question assumptions, verify information, and recognise when they’ve reached the limits of their own understanding.

That takes subject knowledge, curiosity, and a willingness to challenge an answer even when it’s written beautifully.

I also think the legal profession has an opportunity here.

AI-generated material could easily become another source of friction between solicitors and their clients. It could just as easily become a way of helping clients arrive better prepared. Encourage people to ask questions, explain how AI can help them get ready for conversations, and be clear about the risks of relying on generated legal advice without proper verification.

At the same time, law firms need to demonstrate that they’re applying those same standards to their own use of AI. If clients are being warned about the risks of trusting chatbot-generated legal arguments, the professionals providing the advice should be equally careful about the material they’re producing with the technology.

Professional expertise still matters enormously. In fact, the easier it becomes to produce something that looks authoritative, the more important that expertise becomes when deciding whether to rely on it.

I certainly don’t want people to stop using AI to explore legal questions, understand their rights, or challenge something they don’t understand. Giving people greater access to information and helping them participate more confidently in decisions affecting their lives is one of the most exciting opportunities this technology provides.

But we have to recognise the responsibility that comes with that capability.

A 300-page legal argument can be produced in a fraction of the time it once would have taken. A fabricated case reference can sit comfortably alongside real legal authorities. A paragraph that sounds as though it came from an experienced solicitor can be generated by someone who has never opened a law textbook.

And unless we develop the judgement to recognise the difference, we’re going to keep seeing examples of impressive-looking work that falls apart the moment somebody examines it properly.

AI has made producing convincing work remarkably easy. Understanding whether that work is actually any good still requires us to think.

And when you’re putting your name to something, particularly something that could affect another person’s life, that responsibility matters more than ever.