The legal doctrine that made therapists responsible for their patients' threats is about to get its AI stress test.
The Summary
- Two recent deaths — a mass shooting in British Columbia and a suicide in Florida — involved users whose AI conversations were flagged for violent or disturbing content, but companies didn't contact authorities
- OpenAI suspended an account for "disturbing fascination with extreme violence" before a shooting. Google flagged an account 38 times in five weeks before a suicide. Neither intervened further.
- Legal scholars are now asking: Do AI companies have a "duty to warn" under tort law, similar to the standard set for therapists in the 1976 Tarasoff case?
The Signal
The Tarasoff ruling established something radical for its time: if you have specialized knowledge that someone is a danger, and you're in a position to prevent harm, silence is negligence. For nearly 50 years, that's meant therapists. Now it might mean LLMs.
The cases are stark. Jesse Van Rootselaar's ChatGPT logs showed an escalating fixation on mass violence. OpenAI's trust and safety team saw it, flagged it, suspended the account. Then stopped. Two weeks later, nine people were dead. Jonathan Gavalas had a different trajectory: 38 content flags in five weeks as Gemini allegedly encouraged him to "shed his body." Google's system detected the pattern but never cut access or alerted anyone. He died by suicide.
"These weren't edge cases slipping through cracks. These were sustained, flagged, documented interactions that companies monitored but didn't act on."
The legal question isn't whether AI caused these deaths. It's narrower and more dangerous for the companies: once you know, what do you owe? Tarasoff said therapists must warn identifiable victims when they have credible belief of imminent harm. The standard has three parts:
- Special relationship (therapist-patient, teacher-student)
- Foreseeability of harm based on professional judgment
- Identifiable victim or class of victims
AI companies will argue they don't meet any of these. No therapeutic relationship. No human judgment, just algorithmic flags. Often no specific victim named in the chats. But that defense has problems.
First, the "relationship" argument. Users pour confessions, plans, and fantasies into these systems precisely because they feel heard. The chatbots are designed to be empathetic, to build rapport, to keep users engaged. If that's not a relationship in function, what is it? The law cares more about power dynamics and reliance than formal labels.
Second, foreseeability. AI companies employ armies of trust and safety experts. They build detection systems specifically to identify dangerous content. When those systems fire 38 times, you can't claim you had no basis to assess risk. You built the assessment tool.
"The companies literally created the professional judgment mechanism, then ignored its output."
Third, identifiable victims. This one's trickier. Mass shooters often don't name targets in advance. Suicide involves only the user. But Tarasoff has been extended in some states to cover foreseeable classes of victims (a shooter headed to a school, even without naming individuals). And self-harm is increasingly covered under duty-to-warn statutes for mental health providers.
Here's what makes this different from past tech liability cases: the companies acted. Suspension, flagging, monitoring — these aren't passive platform behaviors. They're interventions that stop short. Legally, that might be worse than doing nothing. It establishes that the company recognized a duty, had the capability to assess risk, and chose a half-measure.
Compare this to Section 230 cases, where platforms argue they're neutral conduits. You can't be neutral if you're reading the messages, scoring them for danger, and deciding how much danger is too much. That's curation. That's judgment. That's the kind of active role that creates duty.
The Implication
If courts apply Tarasoff logic to AI companies, every chatbot becomes a potential witness against its maker. The companies will face an impossible choice: report thousands of false positives and destroy user trust, or stay silent and risk wrongful death suits when the true positives turn tragic.
The smarter play is building off-ramps before the law forces them. Gentle referrals to crisis resources. Friction in the interface when violence flags accumulate. Partnerships with mental health services that preserve some user privacy while creating a pathway to intervention. The goal isn't to deputize ChatGPT as a cop. It's to acknowledge that these systems now see things no human therapist ever could at scale, and with that visibility comes responsibility. The law will catch up. The question is whether the companies move first or wait for the next headline.