Meta AI can now alert parents when teens discuss self-harm

Explore with AI:

Meta has introduced a new safety feature that can notify parents when a supervised teen’s conversation with Meta AI suggests they may be thinking about suicide or self-harm. The feature applies to Meta AI conversations across Instagram, Facebook, and Meta Horizon.

To receive these alerts, parents or guardians must enable Meta’s Family Supervision tools and link their teen’s account. The feature is currently available in the United States, Canada, the United Kingdom, and Australia.

Flagged Meta AI conversations are manually reviewed

Meta AI already responds to conversations involving suicide or self-harm by encouraging teens to contact a trusted adult and providing information about crisis support resources.

With this update, Meta may also notify the parent supervising the account if the conversation indicates the teen could be at risk.

The company says it worked with mental health experts to identify the signals used by the system, including subtle language and indirect references that may indicate thoughts of suicide or self-harm.

Before a notification is sent, every flagged conversation is reviewed manually.

Meta says it added this human review process because of the sensitive nature of these alerts and the possibility that AI could misinterpret a conversation.

If reviewers cannot confidently determine the teen’s intent, Meta says it will err on the side of caution and send the notification anyway. As a result, some parents may receive alerts even when there is no immediate danger.

According to Meta, it would rather notify a parent in uncertain situations than risk missing a case where a teen may need immediate support.

Each notification will also include guidance and resources to help parents have a supportive conversation with their teen.

Meta AI could eventually contact emergency services

Meta is also developing a separate system that could contact emergency services when a conversation with Meta AI suggests someone is at imminent risk of taking their own life. Unlike the parental alert feature, this future system is intended to apply to conversations involving both adults and teens.

Meta already contacts emergency services in certain cases involving Facebook and Instagram posts. When the company identifies content indicating a credible suicide risk, it can notify local emergency responders. According to Meta, it made more than 19,000 emergency referrals worldwide last year, enabling first responders to conduct wellness checks.

The company has not yet announced when similar emergency referrals will be expanded to Meta AI conversations.

Do you think Meta’s parental alert system strikes the right balance between teen privacy and safety? Let us know in the comments below.

Don’t miss these related reads:

• Meta’s new Muse Image AI can use your Instagram photos: Here’s how I turned it off
• Meta introduces Muse Image, its first Superintelligence Labs AI generator

Explore with AI: