The robots couldn't get basic civics right, and 34% of voters asking them questions got fed fiction instead of facts.
The Summary
- Demos tested AI chatbots during Scotland's recent election and found they gave wrong answers to 34% of questions, including fabricating scandals and inventing candidates
- The UK's Electoral Commission is now calling for new legal controls on AI-generated election misinformation
- This isn't about sophisticated deepfakes. These were basic factual errors in response to straightforward voter questions.
The Signal
One in three questions about a real election got a made-up answer. Not nuanced policy interpretations. Not edge cases. Basic facts about candidates, dates, and scandals that never happened.
Demos ran a systematic test during Scotland's election. They asked multiple AI platforms the kinds of questions voters actually ask: Who's running? When's the vote? What's this candidate's position? The tools fabricated candidates who didn't exist, invented scandals with no basis in reality, and gave wrong election dates.
"34% error rate on basic civic information means these tools aren't just unreliable, they're actively polluting the information environment."
This matters because of how people use these tools now. You don't fire up ChatGPT or Gemini thinking "I'm about to get experimental slop." You ask it like you'd ask a knowledgeable friend. The interface implies authority. The confidence in the answer reinforces that feeling. But there's no fact-checking layer, no editorial process, no accountability when it hallucinates a political scandal.
The platforms trained these models on everything. They can write sonnets and debug code. But when you ask them something time-sensitive and locally specific, they're guessing. They don't know they're guessing. Neither does the user.
Here's what breaks down:
- Training data goes stale the moment the model ships
- Local elections don't generate enough tokens for models to learn the details
- There's no mechanism to say "I don't know" when confidence is low
The Electoral Commission's response is predictable: they want regulation. New rules, new controls, new legal frameworks. That's the slow solution. It'll take years to draft, debate, and implement. Meanwhile, every election between now and then runs through this same gauntlet.
The Implication
If you're building agent systems meant to interface with the real world, this is your warning shot. Hallucination isn't a bug you can patch later. It's a core limitation of how these models work. When the stakes are civic participation, "usually pretty good" isn't good enough.
For voters: verify everything an AI tells you about elections. Cross-check with official sources. These tools can help you draft emails and brainstorm ideas, but they're not reliable for time-sensitive facts or local information.
For builders: if your agent needs to answer questions where being wrong matters, you need a different architecture. Retrieval systems that pull from verified databases. Confidence scores that trigger "I don't know" responses. Human-in-the-loop verification before the answer ships. The chat interface is a trap that makes users trust answers they shouldn't.