The first global data on AI in classrooms just arrived, and it's not the productivity paradise Silicon Valley promised.
The Summary
- Students using AI for schoolwork are underperforming compared to those who don't, according to the OECD's PISA report—the first global educational assessment since AI went mainstream
- The data covers 15-year-olds worldwide tested in math, science, and reading, providing the widest evidence base yet on AI's classroom impact
- Critical caveat: Some types of AI use boost performance, especially when students are taught to assess AI tool quality
- This isn't about banning AI. It's about figuring out which kinds of AI use build skills versus which kinds atrophy them.
The Signal
The OECD's Programme for International Student Assessment runs every few years, testing teenagers across dozens of countries. The 2025 data collection happened right as ChatGPT and its competitors became standard teenage tools. This is the first PISA study conducted after AI use went truly mainstream, making it the cleanest natural experiment we have on what happens when millions of students suddenly have access to instant answers.
The headline finding is stark: students using AI chatbots score worse than their non-using peers. But the mechanism matters more than the correlation. Students aren't getting dumber because they touched an LLM. They're likely offloading cognitive work that builds foundational skills. When you let the AI do the thinking, you don't learn to think.
"Some types of AI use boost performance, especially when students are taught to assess AI tool quality."
Here's the nuance that keeps this from being a simple "AI bad" story. The OECD data shows a split. Certain AI applications, paired with critical assessment training, actually help. The difference isn't the tool. It's whether students are using AI as a crutch or as a sparring partner. One atrophies skill. The other builds it.
Think about what "critical assessment of AI tool performance" actually means in practice:
- Students evaluate whether the AI's answer is correct or hallucinated
- They compare AI outputs to their own reasoning
- They use AI to check their work, not replace it
This maps directly to the distinction between Web2 passivity and Web4 agency. If AI is something that happens to you, a black box spitting answers you accept wholesale, you're the product. If AI is something you direct, interrogate, and verify, you're the builder. The OECD data suggests schools are producing more of the former than the latter.
Bloomberg frames this as "some of the widest-ranging evidence yet that the technology could be hurting children's learning." That's technically accurate but misses the design problem. AI isn't hurting learning. Uncritical, passive AI use is. The tool didn't fail. The implementation did.
We're watching a generational experiment in real time. The kids using AI as autocomplete for thinking are building habits that will define their capacity to work with agents later. If you never learn to evaluate an AI's output, you'll never be effective at directing one. The students learning critical assessment now are the ones who'll thrive in an agent economy. The ones outsourcing their cognition entirely are training themselves for irrelevance.
The Implication
Schools need to stop debating whether to allow AI and start teaching students how to use it critically. That means assignments designed around AI, not against it. Teach students to prompt, verify, and iterate. Make them explain why an AI answer is right or wrong. Build exercises where the AI is deliberately wrong and students have to catch it.
For everyone building agent products: this data is a warning shot. If your tool makes users passive consumers of output, you're creating dependency, not capability. The AI products that will matter in five years are the ones that make users better thinkers, not the ones that let them stop thinking altogether.