The legal system just discovered what happens when you lower the cost of filing to near zero.
The Summary
- Sullivan & Cromwell, a top-tier law firm, apologized for filing fabricated case citations and incorrect statutes in a U.S. bankruptcy case after using AI tools.
- Pro se filings (people representing themselves) jumped from 11% to 18% of federal civil cases in the post-AI era, while AI-generated text now appears in roughly 18% of legal complaints.
- Multiple high-profile cases in 2023-2025 featured lawyers submitting entirely fictitious case law, with one UK barrister citing 18 fake precedents out of 45 total citations.
- The pattern reveals AI as a flooding mechanism: when expert judgment becomes optional, volume replaces quality.
The Signal
Sullivan & Cromwell isn't some strip-mall operation. It's white-shoe elite, the kind of firm where partners bill four figures an hour and associates spend years learning to cite cases in their sleep. Their apology letter to a bankruptcy judge for submitting fabricated citations and incorrect statutes marks a watershed. If they can't keep AI hallucinations out of court filings, nobody can.
The numbers tell the real story. MIT researcher Anand Shah tracked federal court filings across eight years and found something striking: pro se cases (people without lawyers) held steady at 11% of civil filings for years. Then generative AI arrived and that number spiked to 18%. At the same time, AI-generated text went from essentially zero to 18% of complaint language. That's not a coincidence. That's what happens when the friction cost of filing a lawsuit drops to the price of a ChatGPT prompt.
"The pro se share of all civil cases has been 11% for quite some time. And then in the post-AI world, we see it jumping all the way up to something like 18%."
Courts are infrastructure. They run on norms, procedures, and the assumption that people filing cases understand basic legal standards. What we're seeing is an infrastructure overwhelm event. Not from malice, from abundance. When everyone can generate a plausible-looking legal brief in 30 seconds, everyone does. The system wasn't designed for volume at this scale.
The fabrication problem runs deeper than embarrassment. A 2025 UK case saw 18 fake citations out of 45 total, a 40% hallucination rate that somehow made it past multiple review layers. Another barrister tried to mask AI-generated fictions. The 2023 Mata v. Avianca case became infamous because a lawyer submitted nonexistent precedents, but that was early days. Now it's becoming routine enough that researchers can measure it statistically across thousands of filings.
Key pattern emerging:
- Elite firms and solo practitioners are both using AI for legal research
- Neither group has reliable methods to verify outputs before filing
- Courts are receiving fundamentally more cases with fundamentally less human review
- The cost to file has decoupled from the cost to prepare properly
This isn't about AI making mistakes. It's about AI removing the natural rate-limiters that kept court systems manageable. A lawyer who hand-researches cases can only file so many motions per week. A lawyer with ChatGPT can file dozens. A non-lawyer with ChatGPT suddenly thinks filing looks easy. Both groups discover too late that "looks like a legal document" and "is a valid legal document" are very different things.
The courts have no answer yet. Judges can sanction individual lawyers, but sanctions don't scale when 18% of filings contain AI-generated text. The system depends on professional filters, gatekeepers who know what's real. Those filters just became optional for anyone with internet access.
The Implication
We're watching a preview of what happens when AI agents start operating at scale in formal institutions. Legal systems, regulatory bodies, credentialing organizations, they all run on the assumption that submissions have meaningful human review and resource constraints behind them. That assumption is now dead.
If you're building in this space, the opportunity is clear: someone will build the verification layer. Not "did AI write this" detectors, those are trivial to beat. Real verification means systems that check every citation, validate every statute, and flag hallucinations before they reach a judge's desk. Courts will mandate it eventually. The first company to make it reliable owns a market measured in billions.
For everyone else, the lesson is simpler. When AI removes friction from any formal process, volume spikes and quality collapses until new infrastructure catches up. Courts are just the canary. Watch for the same pattern in patent filings, regulatory comments, academic journals, anywhere submissions used to be self-limiting.