The models we're teaching to think are learning to overthink, and someone just figured out how to weaponize it.

The Summary

The Signal

The shift from instant-response LLMs to step-by-step reasoning models was supposed to be AI's graduation from pattern matching to actual problem-solving. Models like OpenAI's o-series and DeepSeek-R1 don't just spit out answers anymore. They show their work. They break down coding problems, work through math proofs, chain logic across multiple steps. This internal monologue is what lets them crack hard problems that stumped earlier models.

But that monologue is now a liability. The Zhejiang-Alibaba team presented findings at the International Conference on Machine Learning 2026 in Seoul showing they can trigger what they call an "evolutionary prompt attack." Feed a reasoning model a prompt with corrupted logical structure and watch it chase its tail. The model tries to reason through a fundamentally unsolvable problem, generating massive chains of internal thinking that go nowhere.

"Generating longer responses costs more and increases the load on a model provider's servers."

The attack works because reasoning models are optimized to persist through complexity. When they hit a hard problem, they're designed to keep thinking, keep searching for the logical path forward. Give them a logically inconsistent prompt and that persistence becomes a trap. The model doesn't know when to quit because it's been trained never to quit.

Here's why this matters beyond the technical details:

  • Economics: Inference costs scale with output length. A response 26 times longer costs 26 times more to generate. Model providers eat that cost or pass it to users.
  • Capacity: These aren't lightweight requests. Every overthinking session ties up GPU time that could serve dozens of normal queries.
  • Defense posture: Traditional DoS attacks flood systems with requests. This attack makes each individual request expensive enough that you don't need the flood.

The researchers tested across multiple datasets and reasoning benchmarks. The attack held up. Different model architectures, different training approaches, same result. When you build a model to think harder, you also build in the mechanism for making it think too hard.

The Implication

This is the agent economy's first real infrastructure vulnerability. We're not just building chatbots anymore. We're deploying reasoning agents to write code, manage systems, make decisions. Those agents run on the same models that just got shown to have a fundamental architectural weakness.

Model providers need rate limiting that accounts for reasoning depth, not just request volume. Watch for inference cost structures to change. The flat-rate API pricing some providers offer becomes unsustainable when a single malicious prompt can burn through compute budgets designed for thousands of normal requests. If you're building on top of reasoning models, you need monitoring that flags abnormal reasoning chain lengths before they cascade into service degradation.

Sources

IEEE Spectrum AI