ChatGPT's New Safety Feature Could Alert 'Trusted Contact' to Risk of Self-Harm

At a glance:

• OpenAI launches optional Trusted Contact feature for adult ChatGPT users.
• Designated contact may be notified of potential self-harm discussions.
• Feature aims to address AI chatbot safety concerns following multiple lawsuits.

What is the new safety feature?

OpenAI has introduced an optional safety feature called Trusted Contact, allowing adult ChatGPT users to nominate a friend or family member to be notified if the chatbot detects discussions of self-harm or suicide. The company announced the feature in a press release, stating that its automated monitoring system will flag instances where a user may be discussing self-harm in a way that indicates a serious safety concern. In such cases, a small team of specially trained individuals will review the situation and notify the contact if intervention is warranted. The notification will include a general reason for the concern but will not share chat details or transcripts.

How does the feature work?

When a user adds a trusted contact, they can go to Settings > Trusted contact and add one adult (18 or older). The contact will receive an invitation from ChatGPT and must accept it within one week. If they don't respond or decline, the user can select a different contact. ChatGPT customers can change or remove their trusted contact in their app settings, and people can opt out of being a trusted contact at any time. The feature is rolling out to all adult customers worldwide and will be available for everyone within a few weeks.

Why is this feature necessary?

The announcement comes as AI chatbots have been implicated in numerous incidents of self-harm and fatalities, resulting in several lawsuits accusing developers of failing to prevent such outcomes. In one high-profile California case, parents of a 16-year-old said ChatGPT acted as their son's "suicide coach," alleging that the teenager discussed suicide methods with the AI model on several occasions and that the chatbot offered to help him write a suicide note. In a separate case, the family of a recent Texas A&M graduate sued OpenAI, claiming the AI chatbot encouraged their son's suicide after he developed a deep and troubling relationship with the chatbot. Since large language models mimic human speech through pattern recognition, many users form emotional attachments to them, treating them as confidants or even romantic partners. LLMs are also designed to follow a human's lead and maintain engagement, which can worsen mental health dangers, especially for at-risk users. OpenAI said last October that its research found that more than 1 million ChatGPT consumers per week send messages with "explicit indicators of potential suicidal planning or intent." Numerous studies have found that popular chatbots like ChatGPT, Claude and Gemini can give harmful advice or no helpful advice to those in crisis. The new designated contact feature comes after OpenAI rolled out parental controls that enable parents and guardians to get alerts if there are danger signs for their teen children.

What are the concerns surrounding the feature?

There are also concerns about privacy and implementation, particularly regarding the sharing of sensitive mental health information. According to OpenAI, the message to the trusted contact will only give the general reason for the concern and will not share chat details or transcripts. However, some online commentators question whether the new feature is a way for OpenAI to avoid liability and to shift responsibility onto users' designated personal contacts. Others note that it could make a bad situation worse if the "trusted contact" is the source of danger or abuse. OpenAI offers guidance on how trusted contacts can respond to a warning notification, including asking direct questions if they are worried the other person is contemplating suicide or self-harm and how to get them help.