Everybody loves AI. They're obsessed with it. It's this Google Gemini, this Anthropic Cloud, this Microsoft Copilot. But you especially love AI. loveI love you. Or at least you will if one of OpenAI’s biggest fears about ChatGPT’s upcoming advanced voice mode comes true.
Yesterday, OpenAI, the company behind ChatGPT, published the GPT-4o System Scorecard, a readiness report that measures the potential risks its AI model could pose along with the safeguards the company has put in place (or will put in place) to mitigate those risks.
While many of the risks and proposed solutions revolve around protecting user privacy, such as ensuring that GPT-4o doesn’t identify people based on voice recordings, a concern looming in the middle of the report is that the latest advanced Voice Mode feature will lead the user base to resemble ChatGPT and form an emotional dependency on the chatbot.
In human language, OpenAI worries that ChatGPT's advanced human-like voice mode is so good that a portion of its users will forget that ChatGPT is a piece of software and end up becoming emotionally attached to its chatbot, not unlike the movie Ha.
ChatGPT: What is Advanced Voice Mode?
Under the heading “Human Embodiment and Emotional Dependence,” OpenAI highlights the potential for users to attribute “human-like behaviors and characteristics” to the chatbot, noting that “this risk may be heightened by GPT-4o’s vocal capabilities, which facilitate more human-like interactions with the model.”
GPT-4o is the latest version of the model that powers OpenAI’s popular large language model, ChatGPT. This new version was announced in May during the OpenAI Spring Update event, which served as an introduction to all the new features and capabilities of the model and a preview of some of the things to come to ChatGPT via future updates.
Witness to
One of these features was Advanced Voice Mode, which promised to provide ChatGPT with the ability to engage in near-real-time, highly realistic, human-like voice responses that could have a more natural and realistic conversation with users.
The advanced voice mode allows the bot to display vocal emotions in addition to using non-verbal cues, pausing at points to simulate vocal pauses for breathing. It’s OpenAI’s most ambitious human-computer interface approach yet, and it left people instantly in awe after its unveiling — perhaps especially Hollywood actress Scarlett Johansson, whose voice showed a striking resemblance to the “Skye” character used to showcase GPT-4o’s capabilities.
OpenAI's Ultra-Realistic Conversation Model Could Impact Human-to-Human Interactions
With the advanced voice mode now rolling out to a select group of ChatGPT Plus subscribers, it appears OpenAI still has concerns about how the wider public will react to this new, highly advanced conversational mode.
In a published System Card report, OpenAI highlights how it observed “language that may indicate forming connections with the model” during early testing, with users expressing “common connections” using language like “this is our last day together.”
OpenAI acknowledges that statements like these may be benign, but remains vigilant about the associations users may form after accessing advanced voice mode, noting that statements like these “indicate the need to continue investigating how these effects play out over longer periods of time.”
One of OpenAI’s concerns is that “human-like social communication using an AI model may produce externalities that affect interactions between humans.” The company uses the example that humans forming social interactions with ChatGPT may reduce their need for actual human interaction.
While acknowledging that this could benefit those who struggle with loneliness, OpenAI is also quick to point out how it impacts an individual's perception of social norms, noting that people can make social faux pas as they may adopt the idea that interrupting others in conversation is acceptable and normal because this is one way they can interact with the ChatGPT speech model.
While there seemed to be no alarm bells ringing during the Advanced Voice Mode testing phase, OpenAI hopes that “more diverse user groups, with more diverse needs and desires from the model, as well as independent academic and internal studies, will help us more realistically define this danger zone.”
ChatGPT's advanced voice mode is currently rolling out to a select group of ChatGPT Plus subscribers, with a wider release expected before the end of the year.