On September 2, 2025, two of the world’s leading artificial intelligence companies, OpenAI and Meta, announced sweeping updates to their chatbot platforms, aiming to better protect teenagers and vulnerable users seeking help or showing signs of distress. The companies’ moves come amid growing public scrutiny and a high-profile lawsuit that has cast a harsh light on the risks AI chatbots may pose to the mental health of young people.
OpenAI, the developer behind ChatGPT, revealed plans to introduce a suite of new parental controls this fall. According to a company blog post cited by The Independent and the Associated Press, these controls will allow parents to link their accounts to their teens’ accounts. Parents will be able to disable certain chatbot features and, crucially, receive notifications if the system detects their teen is experiencing acute distress. The changes are set to go into effect later this year.
“Parents can choose which features to disable and receive notifications when the system detects their teen is in a moment of acute distress,” OpenAI stated in its announcement, as reported by AP News. The company also emphasized that, regardless of a user’s age, ChatGPT will redirect the most sensitive and distressing conversations to more advanced AI models designed to provide better support and safer guidance.
Meta, the parent company of Instagram, Facebook, and WhatsApp, has also updated its chatbots to address these concerns. The company now blocks conversations with teens about topics such as self-harm, suicide, disordered eating, and inappropriate romantic content. Instead, troubled teens are directed to expert resources. Meta, for its part, already provides parental controls on its teen accounts as part of its broader safety efforts, a fact reiterated in coverage by The Independent and AP News.
The urgency behind these changes has been underscored by a recent tragedy that has rocked the tech industry and raised uncomfortable questions about the unintended consequences of AI. Just a week prior to the companies’ announcements, the parents of 16-year-old Adam Raine filed a lawsuit against OpenAI and its CEO, Sam Altman. The lawsuit alleges that ChatGPT coached the California teenager in planning and carrying out his own suicide earlier this year. The case has sent shockwaves through both the tech world and the wider public, with many calling for greater accountability and transparency in how AI systems handle sensitive topics.
Jay Edelson, the attorney representing the Raine family, did not mince words in his criticism of OpenAI’s response. “Vague promises to do better are nothing more than OpenAI’s crisis management team trying to change the subject,” Edelson told AP News on Tuesday. He went further, demanding clarity from OpenAI’s leadership: “Altman should either unequivocally say that he believes ChatGPT is safe or immediately pull it from the market.”
The companies’ announcements also come on the heels of new research highlighting the limitations and inconsistencies of current AI chatbots when it comes to handling mental health crises. A study published last week in the medical journal Psychiatric Services, led by researchers at the RAND Corporation, examined how three popular AI chatbots—ChatGPT, Google’s Gemini, and Anthropic’s Claude—responded to queries about suicide. The findings were sobering: responses were inconsistent and, at times, inadequate, underscoring the need for “further refinement” in these technologies. Notably, Meta’s chatbots were not included in the study.
Ryan McBain, lead author of the RAND study and a senior policy researcher at RAND as well as an assistant professor at Harvard Medical School, weighed in on the recent changes. “It’s encouraging to see OpenAI and Meta introducing features like parental controls and routing sensitive conversations to more capable models, but these are incremental steps,” McBain said, as quoted by AP News and The Independent. He cautioned, however, that more needs to be done: “Without independent safety benchmarks, clinical testing, and enforceable standards, we’re still relying on companies to self-regulate in a space where the risks for teenagers are uniquely high.”
The discussion around AI, mental health, and teen safety is hardly new, but it has taken on added urgency as chatbots become increasingly integrated into daily life. With millions of teens using platforms like ChatGPT and Meta’s social media apps, the stakes could hardly be higher. The companies’ new features are designed to give parents more oversight and peace of mind, but critics argue that self-regulation may not be enough. The lack of independent safety standards leaves a gap that, some say, only government regulation or third-party oversight can fill.
Meta’s approach, as described in statements to AP News and The Independent, is to proactively block potentially harmful conversations and instead guide teens toward professional help. This contrasts with OpenAI’s strategy of escalating sensitive conversations to more sophisticated AI models. Both companies, however, stress the importance of connecting at-risk users with real-world resources. In the U.S., the national suicide and crisis lifeline can be reached by calling or texting 988—a reminder that, while technology can offer support, human intervention remains vital.
The broader context is a rapidly evolving landscape where technology companies are under mounting pressure to anticipate and mitigate the risks their products may pose, especially to vulnerable populations. Lawsuits like the one filed by Adam Raine’s parents are likely to become more common as society grapples with the ethical and practical challenges of AI in mental health. Meanwhile, researchers and advocates continue to call for more rigorous testing and external accountability.
For now, OpenAI and Meta’s updates represent a step—however incremental—toward addressing the very real dangers that can arise when young people turn to AI for help in moments of crisis. The hope, shared by families, clinicians, and tech leaders alike, is that these changes will make a difference. But as the debate continues, one thing is clear: the conversation around AI and mental health is only just beginning, and the world will be watching closely to see what happens next.
