When a 140-year-old white-shoe law firm has to apologize to a federal judge for AI hallucinations, we've crossed into a new era of accountability for delegation to machines.
The Summary
- Sullivan & Cromwell partner Andrew Dietderich apologized to a bankruptcy judge for a court filing containing fabricated case citations, misquotes, and non-existent legal sources generated by AI
- The errors were caught by opposing counsel at Boies Schiller Flexner, not internal review, despite Sullivan & Cromwell's "comprehensive policies" on AI use
- This marks the first major AI hallucination scandal at an elite Wall Street firm with over 1,000 attorneys, signaling the problem has jumped from solo practitioners to institutional players
The Signal
Sullivan & Cromwell isn't some two-person shop running ChatGPT on the side. This is a 140-year-old institution that represents Fortune 500 companies and governments. Andrew Dietderich, who co-heads their Global Finance & Restructuring practice, had to file an apology letter explaining to a federal bankruptcy judge that his firm's court motion contained AI-generated fiction masquerading as legal precedent.
The filing in the Prince Global Holdings bankruptcy case included incorrect case names, wrong case numbers, and fabricated quotes from real cases. Dietderich's letter defined hallucinations for the judge: "instances in which artificial intelligence tools fabricate case citations, misquote authorities, or generate non-existent legal sources." A chart attached to the apology catalogued each error.
"We deeply regret that this has occurred."
Here's what makes this different from the dozen other AI hallucination cases we've seen since 2023. Sullivan & Cromwell claims to have comprehensive AI policies in place. They have review processes. They have institutional knowledge about technology risk. And the hallucinations still made it through. The errors weren't caught internally. They were caught by opposing counsel at Boies Schiller Flexner, who presumably ran the citations and found them bogus.
This is the institutional failure version of the problem. Early AI hallucination cases involved solo practitioners or small firms who didn't know better. They pasted ChatGPT output directly into filings. Fake legal citations are on the rise across courts, but this case proves that policies, procedures, and billion-dollar reputations don't prevent the problem when humans assume the machine got it right.
Key questions raised:
- If Sullivan & Cromwell's review process failed, what does that say about less-resourced firms?
- Who actually used the AI tool: a partner, an associate, a paralegal?
- What happens when the judge decides an apology isn't enough?
The Implication
Law firms are learning what every company using AI agents will learn: you can't solve hallucination risk with a policy document. You need verification systems that assume the agent is wrong until proven otherwise. For legal work, that means citation checking as a separate workflow, not a trust-and-verify afterthought. For other domains, it means building validation layers into every agent output before it touches the real world.
Watch for courts to start requiring disclosure of AI use in filings, or mandating human certification of every factual claim. The trust model is breaking. Sullivan & Cromwell just proved that embarrassment scales with reputation.