Legal AI Accuracy Index · Q3 2026 · v0
Which legal AI tools publish their citation error rate?
This is the launch edition of a quarterly public index tracking one specific, narrow question across legal AI tools: does the vendor publish a self-measured, methodology-disclosed citation error rate, and if not, does an independent third-party study exist? We are not scoring these tools on features, price, or overall quality here — other pages and other publications do that. This index exists because, as far as we could determine researching this list, almost none of them answer the one question that matters most after a citation-fabrication sanction: “how do you know your error rate, and what is it?”
| Tool | Category | Error rate | Status |
|---|---|---|---|
| Citation Safe | Citation verification (existence + quote + proposition) | Live per-layer FVR, updated weekly Source: citationsafe.com/scorecard (self) | Self-published |
| Lexis+ AI | AI-assisted legal research | ~17% hallucination rate (third-party study, not vendor-published) Source: Stanford RegLab/HAI, Magesh et al. 2024 | Third-party study only |
| Westlaw AI-Assisted Research | AI-assisted legal research | ~33% hallucination rate (third-party study, not vendor-published) Source: Stanford RegLab/HAI, Magesh et al. 2024 | Third-party study only |
| Casetext (CoCounsel, now Thomson Reuters) | AI legal assistant — historical | Not published Source: Acquired by Thomson Reuters 2023; brand folded into CoCounsel | Historical |
| CoCounsel (Thomson Reuters) | AI legal assistant | Not published (vendor-commissioned Forrester ROI study only; not an error rate) Source: legal.thomsonreuters.com | Not published |
| CiteCheck AI (LawDroid) | Citation existence checker | Not published Source: citecheck.ai Existence-only check against CourtListener/similar sources; no quote or proposition layer as far as public materials describe. | Not published |
| JurisCheck | Bluebook-native citation checker | Not published Source: jurischeck.com | Not published |
| Clearbrief | Word add-in: hallucination check + fact-to-record support | Not published (third-party bar-association review score exists, not an error rate) Source: clearbrief.com/pricing; state bar review PDFs | Not published |
| BriefCatch | Legal writing quality tool (not a citation checker) | Not applicable — different product category Source: briefcatch.com Included for completeness since it is frequently mentioned alongside AI legal-writing tools; it targets prose quality, not citation fabrication. | Not published |
| Harvey | General-purpose legal AI assistant | Not published Source: harvey.ai | Not published |
| Paxton AI | General-purpose legal AI assistant | Not published Source: paxton.ai | Not published |
| Spellbook | Contract drafting/review AI (not primarily a citation checker) | Not published Source: spellbook.legal | Not published |
| Robin AI | Contract AI — status uncertain | Not published Source: robinai.com Included per our tracking list; we have not independently confirmed current operating status and do not assert it either way here. | Not published |
| General-purpose chatbots (e.g. ChatGPT, Claude, Gemini) used ad hoc for legal drafting | Not a legal-specific product | Not published by any vendor as a legal-citation-specific figure Source: N/A — not a purpose-built legal tool The Stanford RegLab/HAI study and the underlying Charlotin sanctions database both indicate general-purpose chatbot use produces the fabricated-citation pattern most often seen in sanctioned filings, but we are not citing a specific numeric rate here because we have not independently verified one for this index. Treat this row as a caution, not a data point. | Not published |
Methodology (v0)
For each tool, we checked the vendor’s own marketing and product pages for a self-published, methodology- disclosed error or hallucination rate. Where none existed, we checked whether an independent, citable third-party study (peer-reviewed or from a recognized research institution) had measured one. Rows marked “third-party study only” means the vendor itself has not published the figure — the number comes entirely from outside research, most often the Stanford RegLab/HAI study cited throughout this index. Rows marked “not published” mean we found no vendor-published figure and no independent study we could cite as of this edition; it does not mean the tool performs poorly, only that its error rate is currently unknowable to a buyer from public information. We will update this index quarterly and correct any row on request if a vendor publishes new data — send corrections via /contact.
Why this index exists
We built this because we think it should be the reference journalists, bar associations, and buyers cite when the question “how accurate is this AI legal tool, really” comes up — not because we expect every reader to take our word for our own row. Our own figure is the one row on this table you can check yourself, live, right now, at /scorecard. That is the entire premise of the Argus Standard: a published number beats a confident claim, every time, and we would rather be graded on ours than continue a market norm where almost nobody publishes one at all.