When the upstarts start sounding like trustbusters, the power map has already shifted.
The Summary
- Cohere CEO Aidan Gomez is warning against letting dominant AI labs like OpenAI and Anthropic set industry safety standards, calling it a potential "cartel"
- He supports stronger safeguards and independent testing, but wants rulemaking opened beyond the frontier labs
- The framing reveals how quickly "AI safety" has become a competitive weapon, not just a technical challenge
The Signal
Gomez isn't arguing against AI safety measures. He's arguing about who gets to write them. That distinction matters more than the safety debate itself right now.
Cohere, one of the better-funded challengers to OpenAI, is watching Anthropic and OpenAI push for industry-wide coordination on safety standards. On the surface, that sounds responsible. In practice, it looks like regulatory capture before the regulations even exist.
"When incumbents volunteer to help write the rules, they're not being civic-minded. They're being strategic."
Here's the playbook Gomez sees coming: frontier labs establish "best practices" for safety testing, compute thresholds, and deployment standards. Those practices get codified into law or industry norms. Suddenly, only companies with OpenAI-scale resources can meet the bar. Startups and open-source projects get priced out or slowed down. The leaders stay leaders.
This isn't theoretical. We've seen it in every regulated industry from finance to pharmaceuticals. The question isn't whether safety standards are needed. It's whether the companies with the most to lose from competition should be the ones designing the moat.
Key tensions emerging:
- Safety as genuine concern vs. safety as barrier to entry
- Centralized standard-setting vs. distributed governance
- Speed of innovation vs. thoroughness of safeguards
Gomez wants independent testing and broader stakeholder input. That means researchers, smaller labs, governments, and civil society, not just the four companies that can afford $100 million training runs. It also means accepting that safety standards will be messier and slower to converge.
The alternative is what we're drifting toward: a gentlemen's agreement among frontier labs that doubles as a competitive fortress. They set the standards. They have the resources to meet them. Everyone else has to catch up or partner up.
The Implication
Watch who shapes AI governance over the next 18 months. If it's mostly Anthropic, OpenAI, Google, and Microsoft at the table, that's not safety policy. That's industry consolidation with a safety label.
For builders outside the frontier labs, this is the moment to get loud about governance structure, not just technical specs. The rules being written now will determine whether the agent economy has 100 meaningful players or five incumbents licensing to everyone else.