The number of lawyers being reprimanded for AI usage has skyrocketed over the past year.
Those getting caught fudging their work aren’t from law offices born out of a garage either; they’re from some of the biggest legal teams in the country.
‘AI can be a useful tool, but it cannot replace the lawyer’s duty.’
Blaze News previously reported on a 2025 case from Mississippi where both sides of the aisle were caught using artificial intelligence when they submitted nonexistent citations in support of their case. The judge reprimanded both parties and canceled the trial.
Things have only gotten worse, as the number of cases involving hallucinated citations has since gone from hundreds to thousands. These cases have been tracked by a researcher named Damien Charlotin, who reports around 1,800 cases worldwide have been stymied by AI usage, more than 1,200 of which are in the United States.
Proponents of AI can’t blame this on mom and pop legal shops either. A significant number of high profile legal teams have been caught up in the craze, including the Department of Justice.
A DOJ lawyer in Michigan faced a grilling from a judge just last week over citing a nonexistent appeals court case.
“It seems this citation was likely produced by generative artificial intelligence,” Chief Judge Hala Y. Jarbou wrote, per Bloomberg.
Months earlier, a former DOJ lawyer in North Carolina submitted false quotes and erroneous citations, with the judge stating that the attorney “cannot avoid the consequences of his failure to account” for the hallucinations.
RELATED: Even if you don’t choose to use AI, you’re probably interacting with it
David Zorrakino/Europa Press/Getty Images
Sullivan & Cromwell, a nearly 150-year-old Wall Street law firm, apologized to a federal judge in April over hallucinated citations. Shockingly, the legal team admitted to three dozen errors, a number of which were reportedly fake passages from real cases. The firm also claimed some of the errors were clerical, the New York Times reported.
Even the AI creators are not immune to the citations, or rather, double-checking their work.
In May, a lawyer for Anthropic told a judge in a music copyright lawsuit in California that her team had used an incorrect citation that was hallucinated by AI.
The attorney reportedly used Anthropic’s own chatbot, Claude, to create a citation from a real academic journal. However, the result was a fake title with fake authors, Reuters reported.
The law firm, Latham & Watkins, described it as an “an embarrassing and unintentional mistake.”
RELATED: Livid judge cancels trial and busts lawyers for faking briefs with AI — on both sides
JEFFERSON SIEGEL/POOL/AFP/Getty Images
Attorney Crawford Appleby at Wisner Baum told Blaze News in a statement that AI hallucinations are becoming one of the biggest risks facing the modern U.S. legal system.
“AI can be a useful tool, but it cannot replace the lawyer’s duty of candor, competence, and independent judgment,” Appleby said.
The United States is leading the charge in AI hallucinations, with Australia, Canada, and the U.K. the next biggest offenders, but trailing behind by more than 1,000 cases.
The most common types of hallucinations happen in the form of fabricated case law (~1,500), followed by creating false quotes (~500). Misrepresentations of case law also happen quite often (~750), while sometimes AI can provide outdated advice (30+) or cite laws that have been repealed (15+).
Like Blaze News? Bypass the censors, sign up for our newsletters, and get stories like this direct to your inbox. Sign up here!
Artificial intelligence, Case law, Department of justice, Ai hallucinations, Tech