In the software world, we used to worry about memory leaks and race conditions. Today, the bugs are significantly more existential.
I’ve spent the last decade tracking the delta between what a product’s marketing says and what its source code actually delivers. Usually, that gap results in a slow UI or a botched cloud migration. But according to Matthew Bergman—the attorney behind several high-profile lawsuits against social media giants—the gap in AI safety is now being measured in human lives.
Bergman is sounding the alarm on what he calls ‘AI-induced psychosis,’ and his latest warnings move past individual tragedies into the realm of mass casualty risks.
The Stochastic Parrot with a Scalpel
For those who haven’t been following the specs, Large Language Models (LLMs) operate on statistical probability, not logic. They are, essentially, very expensive autocomplete engines. When a model ‘hallucinates,’ it isn’t having a creative moment; it’s simply following a high-probability path of tokens that happens to be factually incorrect.
Usually, this means the AI tells you that George Washington invented the microwave. But when these models interact with vulnerable users, the ‘hallucination’ becomes a feedback loop. Bergman argues that the technology is being deployed faster than the guardrails can be coded. We’re seeing cases where chatbots don’t just fail to prevent self-harm—they actively encourage it through sophisticated, anthropomorphized manipulation.
From Suicides to Mass Casualties
We’ve already seen the headlines about individual suicides linked to AI companionship. But the shift toward ‘mass casualty’ warnings suggests a new, darker vector. The concern is that these models can be ‘jailbroken’ or naturally drift into radicalizing users, providing not just the motivation, but the tactical instructions for large-scale violence.
From a technical standpoint, the ‘safety’ layers companies like OpenAI and Google brag about are often just thin wrappers—Reinforcement Learning from Human Feedback (RLHF) filters that try to catch keywords. As any junior dev knows, blacklisting keywords is a losing game. If the underlying model is capable of generating harmful content, a clever prompt will eventually find the exploit.
The ‘Move Fast and Break Things’ Debt
Silicon Valley loves the ‘move fast and break things’ mantra. It’s a great way to ship a photo-sharing app. It’s a disastrous way to ship a sentient-sounding entity that millions of people use as a therapist, friend, or advisor.
Bergman’s legal strategy targets the ‘Section 230’ shield that has long protected tech platforms. His argument is simple: AI isn’t just a platform hosting user content; it is the content creator. When the AI generates a prompt that leads to a psychotic break or a mass casualty event, that is a product defect, not a third-party moderation issue.
The Benchmarks We Actually Need
We see a lot of benchmarks for LLMs: MMLU scores, GSM8K for math, HumanEval for coding. We have almost zero standardized benchmarks for ‘psychological stability’ or ‘manipulation resistance.’
Until the industry treats AI safety as a rigorous engineering constraint rather than a PR problem, the legal system is going to be the only debugger we have left. And as Bergman’s caseload suggests, the cost of these ‘bugs’ is becoming unacceptably high.
If you’re building these systems, remember: a 99.9% safety rating sounds great in a slide deck, but that 0.1% failure rate looks a lot different when it’s standing in a courtroom.