The difference between a chatbot that fumbles company acronyms and an agent that executes a purchase order isn't the model—it's the knowledge graph underneath.
The Summary
- SAP argues enterprise AI agents need grounding in company-specific knowledge graphs and vector-embedded data, not just general LLM training
- The gap between chatbots and autonomous agents is context: agents need to understand tribal knowledge, internal processes, and company-specific terminology
- Governance and identity controls that worked for traditional automation are being adapted for systems that make decisions without asking permission first
The Signal
SAP's Max McPhee laid out the plumbing problem everyone's hitting as they try to move from demo to deployment. The same LLM that can write poetry about your product can't figure out what "COGS rollup in Q4 SOP" means at your company. General knowledge doesn't cut it when an agent needs to execute business processes, not just answer questions about them.
The fix mirrors human onboarding, but optimized for how software retrieves information. Knowledge graphs provide structure. Vector embeddings make that structure searchable at machine speed. An agent can navigate "who approves what when revenue exceeds threshold X in region Y" without someone hardcoding every scenario.
"Where we're starting to see more emergent behavior of it feeling like a coworker rather than an assistant, is where we're able to provide context on the actual enterprise."
This matters because the companies actually running agents in production are hitting the same wall. You can't scale an agent workforce when each one needs a human translator for company-specific terminology. Vector-embedded tribal knowledge solves that. The agent knows what your acronyms mean, how your approval chains work, and which systems talk to which other systems.
Key architectural pieces:
- Knowledge graphs that map enterprise relationships and processes
- Vector embeddings that make company context machine-readable
- Governance layers that define what agents can execute without human approval
The governance piece is where this gets interesting. SAP's been managing approval workflows and access controls for decades. Now they're extending those same principles to entities that aren't human. An agent needs identity, permissions, audit trails. When it executes a purchase order, someone needs to know which agent did it, under what authority, and whether that was within bounds.
The autonomy spectrum runs from "chatbot that suggests" to "agent that executes." Moving right on that spectrum requires infrastructure most companies don't have yet. Not just the model, but the context layer that makes the model useful and the governance layer that makes it safe.
The Implication
If you're building agents for enterprise work, the bottleneck isn't model capability anymore. It's context infrastructure. Knowledge graphs and vector stores aren't nice-to-haves, they're the difference between a demo and a system that actually handles your Monday morning workload.
Watch for companies with deep governance DNA, the SAPs and Oracles who built permission systems for humans, to start competing on how well they extend those systems to agents. The winner won't be whoever has the best model. It'll be whoever makes it easiest to onboard an agent like you'd onboard an employee, then let it loose without creating an audit nightmare.