What was hallucinated

Fabricated: Case Law | Plaintiff cited and relied on non-existent case law; Defendants flagged this and the Court was unable to locate the cited cases, warning that such conduct could lead to sanctions.

Details

AI UseThe plaintiff, proceeding pro se, submitted filings citing multiple nonexistent cases. The court noted patterns typical of ChatGPT hallucinations, referencing studies and prior cases involving AI errors, though the plaintiff did not admit using AI.Hallucination DetailsSeveral fake citations identified, including invented federal cases and misquoted Supreme Court opinions. Defendants flagged these to the court, and the court independently confirmed they were fictitious.Ruling/SanctionNo sanctions imposed at this stage, citing special solicitude for pro se litigants. However, the court issued a formal warning: further false citations would lead to sanctions without additional leniency.Key Judicial ReasoningThe court emphasized that even pro se parties must comply with procedural and substantive law, including truthfulness in court filings. Cited Mata v. Avianca and Park v. Kim as established examples where AI-generated hallucinations resulted in sanctions for attorneys, underscoring the seriousness of the misconduct.

Sanction teardown · S.D. New York, USA · 2024-07-18

Anonymous v. NYC Department of Education

What happened

In S.D. New York, USA, a filing relied on an unnamed/unconfirmed AI tool to help draft legal argument. The court identified the following problems with the citations in that filing:

Fabricated (Case Law)

Plaintiff cited and relied on non-existent case law; Defendants flagged this and the Court was unable to locate the cited cases, warning that such conduct could lead to sanctions.

Which AI tool

an unnamed/unconfirmed AI tool. Note: Charlotin's public database records tool attribution only where a court order, brief, or reporting on the matter states it explicitly; "unidentified" or "implied" means the record indicates AI use but does not name a specific product — we do not guess.

Outcome

No sanction; Formal Warning Issued

Additional detail

AI UseThe plaintiff, proceeding pro se, submitted filings citing multiple nonexistent cases. The court noted patterns typical of ChatGPT hallucinations, referencing studies and prior cases involving AI errors, though the plaintiff did not admit using AI.Hallucination DetailsSeveral fake citations identified, including invented federal cases and misquoted Supreme Court opinions. Defendants flagged these to the court, and the court independently confirmed they were fictitious.Ruling/SanctionNo sanctions imposed at this stage, citing special solicitude for pro se litigants. However, the court issued a formal warning: further false citations would lead to sanctions without additional leniency.Key Judicial ReasoningThe court emphasized that even pro se parties must comply with procedural and substantive law, including truthfulness in court filings. Cited Mata v. Avianca and Park v. Kim as established examples where AI-generated hallucinations resulted in sanctions for attorneys, underscoring the seriousness of the misconduct.

How Citation Safe would have caught this

Citation Safe runs three deterministic layers before a brief is filed: (1) does the citation exist against CourtListener's database of published opinions, (2) if quoted, does that exact language appear in the source, (3) does the cited case actually support the proposition it is cited for. Fabricated case citations fail Layer 1. Fabricated or misattributed quotations fail Layer 2 even when the underlying case is real. Misrepresented holdings — a real case cited for a proposition it does not support — are the target of Layer 3. None of these checks involve asking another language model whether the citation looks right; they are lookups and text-matches against the actual source, which is why a hallucinated citation has to survive a direct lookup against the authoritative source — not another model's opinion — to earn a VERIFIED stamp; our measured false-verify rate is published live at /quality.

Check a brief before you file it → · See our live false-verify rate

Source: https://www.damiencharlotin.com/documents/240/Anonymous_v._NYC_Dept_of_Education_US_DC_SDNY_July_18_2024.pdf, via Damien Charlotin's public AI Hallucination Cases Database (CC0).