When your storage layer needs to handle 22 million requests per second, you're not optimizing infrastructure — you're building the pipes for the agent economy.
The Summary
- OpenAI scaled Habitat from a Python library to a distributed storage platform serving 1 billion ChatGPT users at 22M requests/second
- The infrastructure now handles user conversations, memory, and context across the entire ChatGPT ecosystem — the data backbone for AI agents at planetary scale
- This is less about impressive engineering and more about what it signals: the agent economy is already here, it's massive, and it's only getting started
The Signal
A billion users. 22 million requests per second. Those aren't just vanity metrics. They're evidence that AI agents have crossed from novelty to infrastructure — the kind of infrastructure that needs to remember who you are, what you said three weeks ago, and what you're trying to build right now.
OpenAI's engineering team rebuilt Habitat from a simple Python library into a globally distributed storage system because the old approach couldn't keep up. When ChatGPT started, storage was simple: save a conversation, retrieve it later. But as usage exploded and agents got more capable, the requirements changed. Now the system needs to serve memory, context, tool outputs, and multi-turn conversations across billions of interactions daily.
"At 22 million requests per second, you're not running a chat app. You're running the memory layer for digital workers."
The technical challenges are telling. Global distribution means users in Tokyo and Toronto both need sub-100ms access to their data. Consistency matters because an agent that forgets what you told it ten seconds ago is worse than useless. And scale means the system needs to handle both casual users checking in once a week and power users running dozens of agents simultaneously.
Here's what OpenAI's infrastructure team had to solve:
- Horizontal scaling across data centers on multiple continents
- Strong consistency guarantees so agents don't hallucinate based on stale state
- Query patterns that support both simple retrieval and complex agent workflows
This isn't just OpenAI showing off their engineering chops. It's a preview of what every company building agent infrastructure will face. When agents become coworkers — when they're drafting your emails, analyzing your data, managing your calendar — they need memory that works like human memory. Fast, reliable, contextual, and always available.
The bigger signal is in the request volume. 22 million per second doesn't come from people typing questions into a chatbot. It comes from agents checking context, retrieving files, updating state, coordinating across tools. That's the signature of agents doing work, not just answering questions.
The Implication
If you're building anything in the agent space, storage is your bottleneck. Not model capabilities. Not UX. Storage. Because agents that can't remember are just expensive autocomplete.
Watch for the infrastructure layer beneath the agent economy to become the next big technical battleground. The companies that solve state management, memory, and context at scale will own the rails. OpenAI just showed their hand. The question is who else is building pipes strong enough to handle a billion digital workers.