Two AI executives walked into a Bloomberg studio to talk safety — and neither one tried to sell you optimism.

The Summary

The Signal

When two AI executives show up to talk about safety and control instead of capability benchmarks, pay attention. Dan Lahav from Irregular and Joëlle Pineau from Cohere sat down with Bloomberg's Parmy Olson at Bloomberg Tech 2026 in London to discuss what happens when the models get smarter than the safety rails.

The framing matters. Not "if we need guardrails" but "what credible guardrails look like" and who's actually accountable when an AI agent does something its maker didn't intend. These aren't theoretical questions anymore. Every company deploying agents at scale is running into the same wall: you can't audit what you can't observe, and you can't control what you don't understand.

"Credible guardrails and accountability as frontier models advance."

Irregular is building tools for teams trying to wrangle AI systems in production. Cohere is one of the enterprise LLM providers competing with OpenAI and Anthropic. Both have business models that depend on customers trusting AI enough to give it real work. That trust is fragile.

The emerging risk isn't rogue superintelligence. It's the gap between what a model can do and what a company can monitor, explain, or roll back. Right now, most AI safety is post-hoc: run the system, watch what breaks, patch it. That works until the stakes get higher than "wrote a weird email" and closer to "moved money" or "changed code in production."

Here's what the industry is converging on:

  • Observability layers that log every decision an agent makes, not just outputs
  • Third-party audits for high-stakes deployments, the way finance has SOC 2 compliance
  • Kill switches that actually work when an agent starts doing something unexpected

The shift from "move fast and break things" to "prove it's safe before you ship it" is happening quietly. Not because regulators forced it. Because customers started asking harder questions.

The Implication

If you're building with AI agents, expect safety infrastructure to become table stakes in 2027. Enterprises won't deploy agents that can't be audited. Investors won't fund AI companies that can't explain their control mechanisms. The companies that figure out credible third-party verification first will have a moat that capability alone can't match.

Watch for: new categories of tooling around AI observability, red-teaming as a service, and insurance products for AI liability. The safety layer is becoming its own market.

Sources

Bloomberg Tech