The AI companies want a say in how they'll be policed, and Anthropic just set the bar at "50 dead bodies or a billion dollars."
The Summary
- Anthropic is privately pitching Congress on its own AI safety bill, setting specific thresholds: government can only block a model if it poses imminent risk of 50+ deaths or $1B+ in damages
- The bill mirrors California's SB 53 framework, focusing on biological weapons, model jailbreaks, and cyberattacks as the trigger conditions
- This is the AI industry writing its own regulatory playbook before Congress does it for them
The Signal
Anthropic isn't waiting for regulation. They're drafting it themselves. The confidential bill obtained by POLITICO sets hard numbers on what constitutes an AI threat serious enough for government intervention. Fifty deaths. One billion dollars in damages. Those are the thresholds before federal authorities can block a model's release.
The specificity matters. This isn't vague language about "potential harms" or "significant risks." It's a clear ceiling on regulatory power, and it's high. The bill only applies to three specific scenarios: bioweapon creation, models breaking free of developer controls, or catastrophic cyberattacks.
"This is the AI industry writing its own regulatory playbook before Congress does it for them."
Think about what's NOT in those thresholds. Massive job displacement? Not covered. Models that generate convincing deepfakes that swing elections? Outside the scope. Financial fraud at scale but under a billion dollars? Proceed as normal. The bill carves out a narrow definition of AI danger, and everything else gets a pass.
The timing is surgical. Congress has been fumbling with AI regulation for two years, producing mostly hearings and hand-wringing. Anthropic watched California pass SB 53 last year and saw the template. Now they're offering federal lawmakers a ready-made solution that sounds responsible while setting the bar where most current AI risks can't reach it.
Key context on why this matters:
- AI labs this summer disclosed incidents of agents probing and hacking websites
- None came close to the 50-death or $1B threshold
- This bill would make those incidents legal by default unless they cross into catastrophe
The language mirrors California's approach because California proved industry-friendly bills can get passed. SB 53 gave the appearance of oversight without actually constraining what labs could build. Anthropic is taking that playbook national.
What makes this significant isn't the thresholds themselves. It's that a leading AI lab is defining "imminent risk" on its own terms and shopping it to lawmakers who mostly don't understand the technology. The draft has been in circulation since June, quietly shaping conversations before the public debate even started.
The Implication
Watch what Congress does with this. If Anthropic's framework becomes the baseline, we're setting federal AI policy at "catastrophe or nothing." That's a choice. It means accepting widespread disruption, economic displacement, and lower-tier harms as the acceptable cost of innovation.
The real question isn't whether 50 deaths is the right number. It's who gets to define the terms of AI safety before anyone else has a chance to weigh in. Right now, it's the companies building the models. They're not waiting for permission. They're writing the rules and handing them to lawmakers with enough cover to call it "sensible policy."
If you're building in this space, understand that regulatory certainty is coming, and it's going to look a lot like what the big labs want. Plan accordingly.