Meta announced Thursday that it will now alert parents if their teen is discussing suicide or self-harm with the company’s Meta AI chatbot. Meta says it’s also working on the ability to contact emergency services if someone’s conversations suggest they may be at imminent risk of suicide.
These changes come as Meta and other tech companies face scrutiny from regulators and parents over how AI chatbots responds to users in crisisparticularly teenagers — a liability question that increasingly shapes how AI companies design and market their products.
Meta says it has built a proprietary AI system to identify conversations where a teenager makes clear reference to self-harm.
“We understand how distressing these notifications can be for a parent,” Meta wrote in a blog post. “That’s why, as we continue to improve our detection, all conversations flagged by our AI will be manually reviewed before an alert is sent. If a teen’s intent is ambiguous, we’ll err on the side of caution and alert the parent. Although that means sometimes we might alert parents when there might not be real cause for concern. In the right place.”
These notifications are now live for parents using Instagram Parental Supervision in the US, UK, Australia and Canada and will be released globally by the end of the year, Meta says.
This update builds on the alerts Meta already sends to parents when their teen repeatedly searches for suicide or self-harm terms on Instagram. It also builds on a feature that allows parents to see what topics their teen has discussed with Meta AI in the past week.
Meta also announced that the Restricted Content setting — which allows parents to place their teens in a more restrictive Instagram experience — now applies to Meta AI as well. Meta AI is already trained to avoid sexual or romantic or alcohol-related conversations with teenagers, and the Restricted Content setting extends these safeguards by making the chatbot reject a wider range of prompts.
In addition, Meta says it will contact emergency services if someone’s conversation with Meta AI, whether the user is an adult or a teenager, suggests that someone is at risk of suicide. It’s worth noting that Meta already takes this step when someone posts something on Facebook or Instagram that suggests they’re at risk, so this extends the same practice to conversations with its chatbot.
When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.
