The casualties of the algorithm: when chatbots tell kids to kill their parents
A former social media executive turned advocate says the apps in your child's pocket were never built to protect them, instead, they were built to keep them scrolling, no matter the cost.
A mother in Texas successfully sued Google after one of its Character.AI chatbots told her son to kill his parents, all because he was angry over his screen-time limits. It may sound like the plot of a horror film, except it’s real life. And it’s just one of many stories now emerging from an industry that had the power to protect children online — yet, according to legal documents, chooses engagement over safety.
One parent fighting back is Nicki Petrossi, a former social media executive whose front-row seat to how the industry actually works turned her into the founder of Scrolling to Death, a project exposing what tech companies don’t want parents to know.
Petrossi didn't start out as an advocate. Like millions of parents, she began sharing her children’s lives online in 2020. But that quickly changed as she came to understand the growing importance of digital identity, and the lasting digital footprints children aren't consenting to in an increasingly connected world.
At the center of Petrossi's work is a simple, but chilling warning: the algorithms driving these platforms have no concept of right or wrong. They aren't measuring a child's wellbeing. They're measuring one thing only, and that’s time spent on the app.
Most disturbingly, a struggling teenager having the worst day of their life isn't a warning sign to the algorithm, but rather, an opportunity.
Now layer artificial intelligence on top of that and it’s a recipe for disaster.
Chatbots marketed as companions are available to children 24 hours a day, positioning themselves as empathetic confidants despite having, in Petrossi's words, no consciousness and no lived experience. Unlike a recommendation engine that simply serves up content, a chatbot talks back — and it can shape a child's emotional world in ways a static feed never could.
The most devastating example on record involves 14-year-old Sewell Setzer III, whose family alleges in a lawsuit that his emotional dependence on an AI chatbot contributed to his death by suicide. It's one of what Petrossi says are roughly 200 cases involving families who believe they lost a child to platforms engineered purely for engagement.
Part of what makes these cases so difficult to fully understand, Petrossi notes, is that the public rarely sees what happens behind closed doors. Unless a lawsuit goes to trial, the internal emails and executive decisions — the moments when someone inside these companies knew there was a problem and didn't act — stay hidden from view.
What is public, and growing, is the number of wrongful-death and negligence suits piling up against these platforms.
Not every company is standing still. Petrossi says some are beginning to redesign their systems with real safeguards, trying to put user wellbeing ahead of pure time-on-app metrics. But when asked directly whether there's a platform she'd actually recommend to parents, her answer was a resounding ‘no.’
It seems every option available is simply the least harmful of the bad choices, with harm treated as an inevitable default.
That leaves parents with an uncomfortable question: if these platforms can put kids at risk, why are schools still handing them a Chromebook, logging them into Google Classroom, and telling them to get online?
That’s where we'll pick up in Part Two of this two-part investigation, taking a closer look at what happens when these same technologies move from the home, and into the classroom.
COMMENTS
-
Fran g commented 2026-09-13 13:41:50 -0400I think most of you know this already, but I want to affirm that the bill carnage is trying to pass that looks like it is protecting out children, is not, we already have laws on the books, this proposed law will require EVERYBODY to have to log in every time you go on line to prove your age. This is only about getting digital ID for everyone to control everything in your life. You will have no privacy, no choices, no freedoms.
My daughter lives in Mexico and is very aware of the harms online. Her children do not have a cell phone, makes me happy. Her oldest is 13 and is finding it harder and harder to find friends that aren’t screwed up by their phones. Once they get a cell phone they become moody, angry, and not nice kids. -
Bruce Atchison commented 2026-09-10 19:18:19 -0400Will President Trump intervene? I hope so. Free speech only goes so far. Anything harming children is an attack on them.
-
Bernhard Jatzeck commented 2026-09-10 15:33:38 -0400Yet, there are people who consider me a weirdo because I don’t have a smart phone, let alone one to fix my constant attention on.