The AI companies promised they'd let independent researchers watch them build superintelligence — then Anthropic showed them what's happening and redacted most of the evidence.
The Summary
- METR researchers spent 11 weeks inside Anthropic and found the company using AI to build better AI, with model improvement rates doubling year-over-year
- Anthropic is stockpiling three unreleased models that are more powerful than what the public can access, and employees use better systems internally
- The researchers say Anthropic heavily redacted their findings, removing "a majority of the evidence we found most informative"
- This is the first report from the embedded safety program that gave researchers "unprecedented access" to monitor Slack channels, interview staff, and test unreleased models
The Signal
METR's investigation revealed that Anthropic's AI development velocity is accelerating dramatically. The company's models are improving roughly twice as fast as they were a year ago, with no signs of the slowdown that some researchers predicted would hit as training data ran out. The reason: Anthropic is now using its own AI systems to help build the next generation of AI systems.
This is recursive improvement in practice, not theory. Six METR researchers embedded at Anthropic's San Francisco headquarters for nearly three months, monitoring internal communications and testing models on company hardware. They found that Anthropic employees are working with AI capabilities that aren't available to the public yet.
"Anthropic workers are using better AI models than the rest of us."
The company has completed three new models that remain unreleased. This stockpiling approach gives Anthropic two advantages:
- Internal teams can push capabilities further before safety constraints bind
- The company can assess risks before deployment without public pressure to ship
- Competitors stay in the dark about Anthropic's true capability level
But here's the problem: METR says Anthropic redacted most of their evidence. The researchers could share "highest-level conclusions about risk" but not the specific findings that led to those conclusions. This creates a strange dynamic where independent safety research exists in name, but the public can't verify the evidence.
The timing matters. This report comes after Anthropic launched its embedded researcher program following high-profile security incidents over the summer. The company positioned the program as radical transparency — a way for outside experts to see inside the black box. But transparency with redactions is just controlled messaging with extra steps.
The Implication
The recursive improvement loop is here. When AI systems can meaningfully accelerate their own development, the timeline to transformative capabilities compresses. Anthropic's doubling of improvement velocity in one year suggests we're past the point where human researchers are the bottleneck.
Watch what happens in the next 12 months. If this acceleration continues, the gap between what Anthropic's internal teams can do and what the rest of the AI industry understands will widen fast. The stockpiled models are the real signal — companies don't sit on finished products unless those products pose deployment risks they're still figuring out how to manage.