Nvidia just validated the thesis that training chips and inference chips need different DNA.
The Summary
- d-Matrix, a startup building inference-specific chips called XPUs, announced a partnership with Nvidia to integrate its hardware with Nvidia GPUs via NVLink Fusion
- Nvidia is betting that the inference workload — running trained models at scale — requires fundamentally different silicon than training does
- The move signals a strategic shift: Nvidia sees custom inference chips as complementary, not competitive, to its training dominance
The Signal
For years, the AI chip conversation was about training. Who can build the biggest, fastest GPU to process trillions of parameters. Nvidia won that war so decisively that every other player looked like a rounding error. Now the battlefield is shifting.
d-Matrix builds XPUs designed specifically for inference, the phase where you deploy a trained model to answer millions of queries per second. Training happens once. Inference happens forever. The economics are inverted: training is a capital expense, inference is an operating expense that scales with usage.
"Inference is emerging as the next major battleground in AI compute."
Nvidia's NVLink Fusion partnership with d-Matrix is strategic acceptance of reality:
- Training workloads need raw floating-point power and memory bandwidth
- Inference workloads need energy efficiency, low latency, and cost per query optimization
- No single chip architecture is optimal for both
This is not Nvidia being generous. This is Nvidia recognizing that if companies are forced to choose between expensive H100s for inference and cheaper specialized chips, they will choose cheaper. By making d-Matrix chips work seamlessly with Nvidia GPUs, Nvidia keeps control of the training stack while letting specialists handle inference optimization.
The startup CEO discussing this on Bloomberg signals another shift. Two years ago, custom AI chip companies were pitching as Nvidia killers. Now they are pitching as Nvidia partners. The message: we make your Nvidia investment more efficient, not obsolete.
The Implication
Watch for more partnerships like this. Nvidia is building an ecosystem where its GPUs remain the core but specialized inference chips reduce the total cost of running AI at scale. For companies deploying agents, this matters: your inference bill is about to get cheaper, which means more complex agents become economically viable. The constraint on what agents can do is shifting from compute cost to software design.