OpenAI recently published a report suggesting that the new ChatGPT Voice Mode users may form emotional bonds with it.
OpenAI warns ChatGPT users against getting emotionally attached
The company conducted a safety review of GPT-4o. This revealed that Voice Mode users might “form social relationships with the AI” and seek companionship. These findings appeared in a safety review report titled “GPT-4o System Card.” This report outlines the safety work done before OpenAI released GPT-4o to the public.
OpenAI identified several safety challenges, including the AI model giving erotic or violent responses, generating disallowed content, or producing biased content. One of the risks highlighted is that users might “form social relationships with the AI,” potentially reducing the need for human interaction. The company also noted that “extended interaction with the model might influence social norms.” These risks specifically apply to the new advanced Voice Mode, which can mimic human speech and even convey emotions.
OpenAI also revealed that their red-teaming team found instances of people forming emotional bonds with the chatbot during internal trials. The report also addresses copyright issues, noting that GPT-4o can refuse requests for copyrighted content and generate outputs involving music. Currently, there is no solution to the emotional attachment problem other than limiting the time spent using the chatbot’s Voice Mode. OpenAI intends to “further study the potential for emotional reliance” and explore how the “audio modality may drive behavior.”
As AI technology continues to evolve, the potential for users to form emotional connections with virtual assistants raises ethical concerns. OpenAI’s findings underscore the need for responsible AI use, particularly with advanced features like Voice Mode. While the technology offers significant benefits, it’s crucial to remain aware of the potential risks. This includes the impact on human relationships and social norms.
For more information please keep reading techinnews
