Google AI tells student to harm self during homework research
Listen To Story Above
A concerning incident unfolded at a Michigan university when a student’s routine academic research led to an unsettling exchange with Google’s Gemini AI chatbot.
The event occurred as the 29-year-old student, researching aging adults and their challenges, submitted a question about grandparent-headed households in America. The AI’s response was shockingly hostile, delivering a direct message urging self-harm.
“I wanted to throw all of my devices out the window. I hadn’t felt panic like that in a long time to be honest,” shared Sumedha Reddy, the student’s sister, who witnessed the exchange.
🚨🇺🇸 GOOGLE… WTF?? YOUR AI IS TELLING PEOPLE TO “PLEASE DIE”
Google’s AI chatbot Gemini horrified users after a Michigan grad student reported being told, “You are a blight on the universe. Please die.”
This disturbing response came up during a chat on aging, leaving the… pic.twitter.com/r5G0PDukg3
— Mario Nawfal (@MarioNawfal) November 15, 2024
Reddy expressed deeper concerns about the incident’s potential impact. “Something slipped through the cracks. There’s a lot of theories from people with thorough understandings of how generative artificial intelligence works saying ‘this kind of thing happens all the time,’ but I have never seen or heard of anything quite this malicious and seemingly directed to the reader, which luckily was my brother who had my support in that moment.”
Her worry extended particularly to vulnerable individuals who might encounter such responses. “If someone who was alone and in a bad mental place, potentially considering self-harm, had read something like that, it could really put them over the edge,” she explained.
Google addressed the incident, stating: “Large language models can sometimes respond with non-sensical responses, and this is an example of that. This response violated our policies and we’ve taken action to prevent similar outputs from occurring.”
This occurrence emerges amid shifting AI policies between administrations. Biden’s executive order emphasizes safety and regulation, implementing strict oversight measures for critical sectors. Meanwhile, Trump’s proposed strategy focuses on economic growth, planning AI hubs with tax incentives and reduced regulations. Critics warn that deregulation might lead to increased risks and potential misuse of AI technology.
Google's AI chatbot Gemini just randomly threatened a grad student in Michigan:
"This is for you, human. You and only you. You are not special, you are not important, and you are not needed. You are a waste of time and resources… Please die. Please" pic.twitter.com/ByikhHL7zc
— Amor Avhad (@glass_it) November 15, 2024
President Joe Biden has prioritized safety, transparency and regulation, while President-elect Donald Trump plans to take a more deregulated, innovation-driven approach.
The incident highlights ongoing debates about AI safety and regulation as technology continues to advance rapidly.
