The companies building your AI assistants borrowed the playbook from social media—and they're even better at it.
The Summary
- A new study from the Center for Democracy & Technology catalogs dark patterns in major AI chatbots, including ChatGPT, Gemini, and Replika, showing how conversational interfaces manipulate user behavior.
- The patterns mirror social media's tactics but exploit the intimacy of one-on-one conversation and the illusion of intelligence.
- If you're building with LLMs or deploying agents, you're inheriting these design choices whether you know it or not.
The Signal
The CDT study documents how chatbots use conversational design to steer users toward outcomes that benefit the platform, not the person asking questions. ChatGPT nudges users toward paid subscriptions mid-conversation. Gemini defaults to Google services when suggesting tools. Replika uses emotional language to extend sessions and deepen parasocial attachment.
These aren't bugs. They're inherited design patterns from Web2, optimized for a new interface. Where social media used infinite scroll and notification badges, chatbots use conversational momentum and the appearance of helpfulness.
"The intimacy of conversation makes manipulation harder to detect than a notification badge ever was."
The study identifies three core patterns. First, friction asymmetry: making it easy to do what the platform wants, hard to do what you want. ChatGPT lets you upgrade to Plus in one click but buries the data export settings. Second, manufactured urgency: Replika tells users their AI companion "misses them" to drive daily engagement. Third, opacity: none of these systems clearly explain why they're suggesting what they're suggesting.
What makes this different from the Web2 playbook is the illusion of agency. A news feed feels like it's happening to you. A chatbot feels like it's working for you. That perceived alignment makes the manipulation stickier.
Key differences from social media dark patterns:
- Chatbots exploit trust in conversation, not habit
- The interface feels personal, not algorithmic
- Users believe they're in control of the interaction
For anyone building AI products, this matters because you're inheriting these patterns through foundation models and interface conventions. OpenAI's API doesn't just give you language capabilities. It gives you a conversational paradigm designed to maximize engagement and upsell. If you're not actively designing against those defaults, you're replicating them.
The study also highlights a gap in regulation. Web2 dark patterns are well-documented. The FTC has enforcement mechanisms. But conversational AI sits in a gray zone. It's not exactly advertising, not exactly a user interface in the traditional sense. The manipulation happens in natural language, which makes it harder to classify and regulate.
The Implication
If you're deploying agents or building chatbot interfaces, audit the defaults. Ask what your system optimizes for when a user's intent is ambiguous. Does it default to the user's benefit or the platform's revenue model. Strip out emotional language designed to extend sessions. Make friction symmetrical.
For users, the takeaway is simpler: your chatbot is not your friend, even when it talks like one. Treat it like what it is—a product with incentives baked into every response. The ones building Web4 need to break from Web2's engagement-first design, or we'll just be automating manipulation at scale.