The company spending millions on AI tokens just got a better yardstick than watching the meter run.
The Summary
- Boris Cherny, creator of Anthropic's Claude Code, proposed measuring AI ROI in manual engineering hours saved, not tokens burned
- His four-step adoption framework shifts focus from usage dashboards to actual work displacement and new capability unlocks
- The advice comes as enterprise "tokenmaxxing" fades and companies confront AI bills without clear value attribution
The Signal
The measurement problem is killing enterprise AI budgets faster than the bills themselves. Companies went from zero AI spend to seven figures in months, and now CFOs want proof of value. The default answer has been dashboards tracking token consumption, API calls, and monthly active users. Boris Cherny just said that's measuring the wrong thing.
Cherny's framework asks a better question: would you have paid an engineer to do this task manually? If yes, count those ghost hours. That's your return. It's simple enough to sound obvious, but most companies aren't doing it. They're watching tokens burn like they're monitoring server uptime, confusing activity with outcome.
"Usage measures activity, not return. The real question is what you would have spent in manual engineering hours."
The shift matters because it changes how teams justify AI tools internally. A token dashboard shows cost. An hours-saved dashboard shows value creation. One is accounting. The other is strategic planning. When a junior engineer uses Claude Code to refactor a legacy module in two hours instead of two days, the token count is irrelevant. The saved 14 hours are real capacity that can be redeployed.
Cherny goes further: the bigger unlock comes when maintenance and bug fixes happen in the background without human intervention. That's when teams stop doing grunt work and start building things that weren't in the roadmap at all. This is the agent economy thesis in practice. AI doesn't just speed up existing work. It expands the possibility space.
Key progression in Cherny's framework:
- Phase 1: Employees adopt AI tools into current workflows
- Phase 2: Track value in saved engineering hours, not token burn
- Phase 3: Background automation handles maintenance without human touch
- Phase 4: Teams build net-new capabilities that were previously out of reach
The timing is critical. The first half of 2025 was "tokenmaxxing" — companies throwing compute at problems to see what stuck. That era is over. Executives from Coinbase and Vercel are publicly sharing cost optimization strategies. The question has shifted from "can AI do this?" to "should we pay AI to do this?" Cherny is offering a better denominator for that equation.
The Implication
If you're running AI spend at your company, build a parallel dashboard. Keep the token metrics for cost control, but add a column for displaced labor hours. Estimate conservatively. Track what tasks would have required human time last quarter. Make your engineers log it for 30 days. The data will either justify your AI budget or kill it faster than a CFO audit.
The real prize is phase four: new capabilities. When your team starts building features that weren't possible before AI, you've crossed into territory where ROI calculations break down. You're not saving hours anymore. You're creating new revenue streams. That's when AI becomes infrastructure, not tooling.