What was hallucinated

False Quotes: Case Law | Plaintiff quoted Gideon v. Wainwright as stating, "[c]onstitutional rights would be of little value if they could be indirectly denied," but the court found this quote does not appear in Gideon on the cited page or anywhere; a similar statement appears in Smith v. Allwright. || Misrepresented: Case Law | Court noted several instances where cases cited by plaintiff did not remotely match the propositions for which they were cited.

Details

"Artificial intelligence may ultimately prove a helpful tool to assist pro se litigants in bringing meritorious cases to the courts. In that way, artificial intelligence has the potential to contribute to the cause of justice. However, accessing any beneficial use of artificial intelligence requires carefully understanding its limitations. For example, if merely asked to write an opposition to an opposing party’s motion or brief, or to respond to a court order, an artificial intelligence program is likely to generate such a response, regardless of whether the response actually has an arguable basis in the law. Where the court or opposing party was correct on the law, the program will very likely generate a response or brief that includes a false statement of the law. And because artificial intelligence synthesizes many sources with varying degrees of trustworthiness, reliance on artificial intelligence without independent verification renders litigants unable to represent to the Court that the information in their filings is truthful."

Sanction teardown · D. Connecticut, USA · 2025-06-20

Reilly v. Conn. Interlocal Risk Mgmt. Agency

What happened

In D. Connecticut, USA, a filing relied on an unnamed/unconfirmed AI tool to help draft legal argument. The court identified the following problems with the citations in that filing:

False Quotes (Case Law)

Plaintiff quoted Gideon v. Wainwright as stating, "[c]onstitutional rights would be of little value if they could be indirectly denied," but the court found this quote does not appear in Gideon on the cited page or anywhere; a similar statement appears in Smith v. Allwright.
Misrepresented (Case Law)

Court noted several instances where cases cited by plaintiff did not remotely match the propositions for which they were cited.

Which AI tool

an unnamed/unconfirmed AI tool. Note: Charlotin's public database records tool attribution only where a court order, brief, or reporting on the matter states it explicitly; "unidentified" or "implied" means the record indicates AI use but does not name a specific product — we do not guess.

Outcome

Warning

Additional detail

"Artificial intelligence may ultimately prove a helpful tool to assist pro se litigants in bringing meritorious cases to the courts. In that way, artificial intelligence has the potential to contribute to the cause of justice. However, accessing any beneficial use of artificial intelligence requires carefully understanding its limitations. For example, if merely asked to write an opposition to an opposing party’s motion or brief, or to respond to a court order, an artificial intelligence program is likely to generate such a response, regardless of whether the response actually has an arguable basis in the law. Where the court or opposing party was correct on the law, the program will very likely generate a response or brief that includes a false statement of the law. And because artificial intelligence synthesizes many sources with varying degrees of trustworthiness, reliance on artificial intelligence without independent verification renders litigants unable to represent to the Court that the information in their filings is truthful."

How Citation Safe would have caught this

Citation Safe runs three deterministic layers before a brief is filed: (1) does the citation exist against CourtListener's database of published opinions, (2) if quoted, does that exact language appear in the source, (3) does the cited case actually support the proposition it is cited for. Fabricated case citations fail Layer 1. Fabricated or misattributed quotations fail Layer 2 even when the underlying case is real. Misrepresented holdings — a real case cited for a proposition it does not support — are the target of Layer 3. None of these checks involve asking another language model whether the citation looks right; they are lookups and text-matches against the actual source, which is why a hallucinated citation has to survive a direct lookup against the authoritative source — not another model's opinion — to earn a VERIFIED stamp; our measured false-verify rate is published live at /quality.

Check a brief before you file it → · See our live false-verify rate

Source: https://www.govinfo.gov/content/pkg/USCOURTS-ctd-3_25-cv-00640/pdf/USCOURTS-ctd-3_25-cv-00640-0.pdf, via Damien Charlotin's public AI Hallucination Cases Database (CC0).