The world's most advanced AI models just got hit by the largest documented knowledge theft in machine learning history, and the playbook is embarrassingly simple.
The Summary
- Anthropic accused Alibaba-affiliated operators of using nearly 25,000 fraudulent accounts to generate 28.8 million Claude exchanges, executing what may be the largest AI distillation attack on record.
- The company is urging Congress to strengthen AI export controls to prevent unauthorized knowledge transfer from US models to foreign competitors.
- This isn't theoretical risk anymore. Distillation attacks, where cheaper models learn from expensive ones through massive query campaigns, are now happening at industrial scale.
The Signal
Anthropic caught Alibaba-affiliated operators running 25,000 fake accounts that fired off 28.8 million queries to Claude. That's not exploratory research. That's systematic knowledge extraction. The goal: train a cheaper Chinese model on the responses from one of the world's most capable AI systems without paying for the underlying research, compute, or safety work that went into it.
Model distillation isn't new. Researchers have used it for years to compress large models into smaller, faster versions. But this is different. This is competitive intelligence at datacenter scale, dressed up as normal API usage.
"The playbook is embarrassingly simple: create thousands of accounts, ask millions of questions, feed the answers into your own model."
Here's why it works. Foundation models like Claude cost hundreds of millions to train. But once they're behind an API, anyone with a credit card can query them. Ask enough questions, capture enough responses, and you can train a derivative model that mimics the original's behavior without replicating the original's training process. You skip the hard part. You skip the cost.
Anthropic is pushing Congress to close this gap through stronger AI export controls. The argument: if US models represent national security assets, then letting foreign competitors drain their knowledge through API access is a strategic failure. It's not about blocking access to AI broadly. It's about preventing the wholesale transfer of billions of dollars in R&D through what looks like normal usage.
The challenge is enforcement. How do you distinguish between a researcher asking 1,000 questions and a distillation attack asking 28 million? Rate limits help but sophisticated actors spread queries across thousands of accounts. Usage pattern analysis catches some of it, but not fast enough to stop large-scale campaigns before significant knowledge transfer happens.
Key tensions this exposes:
- Open API access vs. strategic model protection
- Commercial incentives (more API calls = more revenue) vs. national security concerns
- The fiction that model weights are the only thing worth protecting when query access gives you 80% of the value
The Implication
If you're building on frontier models through APIs, assume your usage patterns are being monitored and your competitors are watching what prompts work best. If you're running an AI company with a public API, distillation attacks are now a cost of doing business, and you need detection systems that can spot coordinated campaigns before they drain millions in compute.
Anthropic's call for stronger export controls signals a new phase: AI companies realizing that open APIs and global competition don't mix cleanly when your model represents a strategic asset. Expect more companies to follow with tiered access, country-specific restrictions, and usage caps tied to vetting processes. The era of "anyone can call our API" is ending.