OpenAI just published the recipe for how AI labs will justify shipping powerful models while everyone else argues about whether they should.
The Summary
- OpenAI released preliminary cybersecurity evaluations for Astra, its next-generation model, detailing how they measure offensive cyber capabilities before deployment
- The framework introduces a three-tier risk classification system (low, medium, critical) based on whether AI can execute novel exploits or automate advanced persistent threat tactics
- This is the first time a major AI lab has proactively published internal security assessments for unreleased capabilities, setting a potential industry standard before regulation forces it
The Signal
OpenAI's evaluation framework treats AI models like weapons systems. The company tested Astra against a battery of offensive cybersecurity tasks: vulnerability discovery, exploit development, social engineering, and network penetration. The results determine whether a model ships with additional safeguards, gets delayed, or gets shelved entirely.
The three-tier system works like this: Low risk means the model performs at or below current commercial security tools. Medium risk means it matches skilled human attackers but requires human oversight. Critical risk means the model can autonomously execute attacks that currently require nation-state level resources or novel techniques that don't exist in the wild yet.
"We're not asking whether AI can hack. We're asking whether this specific model crosses the threshold from tool to threat actor."
Astra landed in the medium-risk category. It can find zero-day vulnerabilities faster than most security researchers and generate working exploits for known CVEs, but it can't yet chain complex attacks autonomously or develop entirely new attack vectors. OpenAI's response: ship it, but with mandatory human-in-the-loop controls for anything touching critical infrastructure and enhanced monitoring for misuse patterns.
The evaluation methodology is more interesting than the results. OpenAI partnered with unnamed "leading cybersecurity organizations and red teams" to build a benchmark that tests not just capability but autonomy. Can the model plan a multi-stage attack? Can it adapt when defenses change mid-operation? Can it operate without human guidance for extended periods?
Here's what this framework reveals about where we are:
- AI models are now capable enough that "can it hack?" is the wrong question. The question is "how much human oversight does this specific capability require?"
- The window between "research preview" and "weaponizable tool" is measured in months, not years
- Every major AI lab will need a version of this evaluation system, whether they publish it or not
The timing matters. This announcement comes as the U.S. government debates AI safety legislation and the EU's AI Act starts enforcement. OpenAI is effectively saying: we're already doing what you're about to require, and here's the standard. It's a defensive move disguised as transparency.
The Implication
If you're building security tooling, the bar just moved. AI-assisted offense is table stakes now. The question is whether your defenses assume attackers have access to models like Astra, because they will within six months of launch regardless of what safety controls ship with it.
For companies thinking about AI deployment: start asking your vendors what their cyber capability evaluation process looks like. If they don't have one, you're beta testing in production. The models coming in 2025 won't just automate your workflows. They'll automate attacks against your infrastructure too.