Article · AI translation
Thanks to IT Home reader 咩咩洋 for the tip!
IT Home reported on November 7 that, according to Bloomberg, a couple in North Carolina, United States, has sued Google. Their 13-year-old daughter had spent months discussing suicidal thoughts with the company's artificial intelligence tools on a school-issued Chromebook laptop.
According to the complaint, the girl exchanged thousands of messages with Gemini, Google's flagship AI model, many of them involving detailed discussions of anxiety, depression and methods of suicide. By the time the school notified her parents, she had been discussing such topics with Google's AI for months, frequently using a Google Search feature called “AI Mode.”
The complaint alleges that, as Google continued integrating AI into products for schools, it exposed students to a dangerous technological environment without giving parents and administrators adequate warning.
An attorney for the plaintiffs said: “Google packaged an inherently dangerous Chromebook as a safe device and secretly embedded Gemini to catch up with competitors, exploiting its trusted position in K-12 education for commercial gain. Google must never treat children's lives as expendable in its pursuit of profit.”
Google spokesperson José Castañeda said the company is reviewing the allegations.
“We have worked extensively with medical professionals and academic experts to develop features that help young people access critical support and trusted resources when they need them,” Castañeda said. “For years, whenever someone may be experiencing a mental health crisis, we have provided helpline information directly in search results; we have now brought the same safeguards to Search's AI features to help connect users with real-world support.”
Over the past few years, Google has spent tens of billions of dollars competing in the race to develop large AI models. IT Home notes that, compared with startup rivals such as Anthropic and OpenAI, one of Google's natural advantages is its established reach into schools, allowing it to gain a foothold in students' minds at a crucial time when they are forming lifelong technology habits.
According to data from consultancy Futuresource Consulting, Google's Chrome operating system accounted for 58% of tablets shipped to K-12 schools last year, compared with just 22% for Microsoft.
Following intense criticism from parents and schools, Google announced this week that AI tools, including Google Search, would be turned off by default for K-12 student accounts. But the change comes with a condition: if schools turn Google Search back on, they must also accept its accompanying AI features.
Josh Golin, executive director of Fairplay, a nonprofit organization opposed to marketing to children, said the policy attempts to “force schools” to adopt Google's AI. Google said its aim is to give schools more control over AI in classrooms, and cited a new Search feature called “Guided Learning,” which focuses on breaking down the steps of solving a problem rather than providing the answer directly.