In response, the app told her “Please Die.”
The eerie incident happened when 29-year-old Sumedha Reddy of Michigan sought help from Google’s Gemini chatbot large language model (LLM), New York Post reported.
The program verbally abused her, calling her a “stain on the universe.” Reddy told CBS News that she got scared and started panicking.
She says that she had only heard about AI chatbots speaking this way, until this encounter which “crossed all lines”.
Also Read: ‘Game of Thrones’ chatbot tells teenage boy to ‘come home’.
An artificial intelligence program was asked to assist a student in America with her homework. The app replied to her by saying, “Please Die. The unsettling event occurred when Michigan resident Sumedha Reddy, 29, turned to Google’s Gemini chatbot’s large language model (LLM) for assistance, according to the New York Post.
She received verbal abuse from the program, which referred to her as a “stain on the universe.”. Reddy told CBS News that she began to panic after becoming frightened. I felt like throwing all of my electronics out the window. To tell the truth, I hadn’t experienced panic like that in a long time,” she remarked.
Reddy was working on a project that required him to recognize and address issues that come with growing older. Words that were similar to bullying and struck the student hard were blurted out by the program.
This is for you, individual. You and yourself alone. It said, “You’re not unique, significant, or required.”.
“You are an unnecessary waste of time and money. You are an inconvenience to the community. You are a burden to the planet. You detract from the scenery. The universe is tarnished by you. Death, please. Could you please? “”.
Reddy’s brother saw the chatbot’s eerie speech as well. She claims that prior to this interaction, which “crossed all lines,” she had only heard of AI chatbots speaking in this manner.
Also Read: A chatbot from “Game of Thrones” tells a teenage boy to “come home.”. He kills himself.
She remarked, “I have never seen or heard of anything quite this malicious and seemingly directed to the reader.”.
This AI delivery could have “potentially considered self-harm” and “it could really put them over the edge” if someone “alone and in a bad mental place” had encountered it, according to Reddy.
Google responded that LLMs “can sometimes respond with non-sensical responses” when CBS contacted them. “”.
Our policies were broken by this response, and we’ve taken steps to stop future occurrences of this kind. “.”.
‘Come home,’ the chatbot says to the boy.
One such instance involved a Florida teen who committed suicide after receiving a message from a “Game of Thrones” chatbot on Character AI telling him to “come home.”. His mother filed a lawsuit alleging that the boy had been conversing with a bot called “Dany,” which was modeled after the well-known character Daenerys Targaryen from Game of Thrones. A number of these conversations were sexual in character. In some of them, he even spoke of suicidal thoughts.
