Accuracy — live, not claimed

This page is generated directly from our nightly benchmark against a held-out set of confirmed-fake citations (from published AI-sanction orders) and confirmed-real citations (from published opinions). We track False-Verify Rate (FVR) — the rate at which we stamp VERIFIED on something that is actually fake or wrong — as the one metric that matters. The inverse error (flagging a real citation UNCONFIRMED) is allowed to run far higher; a lawyer double-checking a real citation is a Tuesday, a lawyer sanctioned while holding our green checkmark is not.

Layer 1 — Existence

within target

Accuracy

100.0% (target ≥99.5%)

False-Verify Rate

0.000% (target <0.1%)

Eval set size

176 citations

Last measured

7/13/2026, 4:50:48 PM

Layer 2 — Quote match

measurement pending

This layer does not yet have a persisted eval run in the live database. See our build log for the most recent manual eval snapshot.

Layer 3 — Proposition support

within target

Accuracy

100.0% (target ≥85.0%)

False-Verify Rate

0.000% (target <5.0%)

Eval set size

40 citations

Last measured

7/13/2026, 12:00:00 AM

Disputes

Total disputes filed

Currently open

Avg. resolution time

—

Target: <24h automated resolution, <72h if escalated.

Volume

39 documents verified to date.

Coverage map

Existence and quote checks run against CourtListener's database of published opinions: federal courts (all circuits + Supreme Court), and state courts to the extent CourtListener has ingested them (coverage varies by state and year, heaviest for appellate/supreme court opinions post-1950). Outside coverage today: statutes and regulations, unpublished BIA/EOIR immigration decisions, tribal courts, and most trial-court-level state filings. Citations in these categories are labeled UNCONFIRMED — OUTSIDE COVERAGE, never silently treated as NOT FOUND.

Methodology

Eval set: confirmed-fake citations drawn from published AI-hallucination sanction orders, paired with an equal count of confirmed-real citations from published opinions (the negative class matters — a checker that flags everything catches all fakes and is useless). FVR per layer = fake/wrong citations stamped VERIFIED at that layer ÷ total citations stamped VERIFIED at that layer. Full methodology and the eval-set generation scripts are open in our repository.