Meta adds support and parental notifications to AI conversations about self-harm

If you feel that you or someone you know is in immediate danger, you should call 911 (or your country’s emergency number). Local emergency line) Or go to the emergency room for immediate help. Explain to her that it is a psychological emergency and ask for someone trained to handle these types of situations.

Amid growing industry scrutiny of how tech companies build safeguards for their youngest, most vulnerable users, Meta is introducing new safety-oriented features to its chatbot on Thursday. Guardrails are specifically designed toward providing support and parental notification to youth who show signs of self-harm or suicidal intent.

As part of Meta’s moderation tools for parents, adults must sign up and choose which Instagram, Facebook, and Meta Horizons accounts they want to moderate, with detailed instructions at Meta Family Center page. Settings is now available in the US, Canada, UK and Australia.

Now, when a teen shows signs that potentially harmful content is being discussed using the Meta AI chat software, an alert will be sent to their supervising parent via text, email, or in-app notification. The exact content of the teen’s message will not be included, but Meta will share resources and tips with the parent.

The company says it already shares crisis helpline information with teens and encourages them to reach out to trusted adults for help. Meta also alerts parents if their teen is like this Search frequently on Instagram To self-harm or commit suicide in a short period of time.

Meta says it has built an artificial intelligence system to identify potentially dangerous conversations, drawing on the expertise of real humans through its wellbeing panels and external mental health clinicians. At launch, all alerts will be sent through a manual review process, and the company will “err on the side of caution and alert parents.”

Meta is also building the ability to alert emergency services in cases where there is a real risk of suicide, as the company calls it. This isn’t available yet, but would likely be similar to the alert system already in place on Facebook and Instagram, which the company says has led to more than 19,000 referrals globally to emergency responders for health checks.

The parent company of Facebook and Instagram is no stranger to the damage technology can do to mental health. Earlier this year, a separate jury convicted Meta of construction charges Social media platforms that are intentionally addictive and Enabling child exploitation.

This comes after a renewed period of concern from technology watchdogs and safety advocates who have called for, and in some cases… He filed a lawsuit against technology and artificial intelligence companies over…and risky behaviors that AI-powered chatbots can encourage and enable, including self-harm and suicide.