In a year marked by rapid advances in artificial intelligence and mounting questions about its role in health care, the U.S. Food and Drug Administration (FDA) found itself at the center of a heated debate this November. At its Digital Health Advisory Committee meeting, the agency weighed whether to treat AI-powered mental health chatbots as medical devices, a move that could have sweeping consequences for innovation, patient safety, and access to care.
The stakes are high. As the FDA clarified at the meeting, it "intends to apply its regulatory oversight" to software functions considered "medical devices" whenever poor functionality could pose a risk to patient safety. This includes generative AI-enabled products—think chatbots that can converse, offer advice, and even interpret medical information. But where exactly should the line be drawn?
According to the FDA’s definition, a product is considered a medical device if it is intended to diagnose, cure, mitigate, treat, or prevent a specific disease—including mental health conditions. That definition is clear-cut for some digital health tools. Apps like Rejoyn and DaylightRx, for instance, explicitly market themselves as treatments for diagnosed mental health issues such as depression. For these, the FDA already demands rigorous oversight: annual registration fees of $11,423, extensive premarket paperwork, risk management protocols, and ongoing postmarket reporting. The rationale? When an app claims to treat or cure, accuracy and accountability are paramount.
But what about the growing number of AI chatbots that don’t promise to diagnose or treat? These tools—popularized by companies like Slingshot AI (creator of Ash) and Wysa—are designed to provide general coping skills, mindfulness exercises, and cognitive behavioral therapy–style reframing. They’re not making clinical claims. Instead, their stated goal is to support users’ well-being, offering a virtual ear and practical advice to those struggling with stress, loneliness, or anxiety.
“We aim to provide general wellness by making mental health support more accessible,” Slingshot AI wrote in public comments to the FDA. Wysa, another leading chatbot, is equally explicit: it listens and responds to users’ emotions and thoughts, but it does not diagnose or attempt to treat any conditions. These companies are careful to position their products as "wellness apps," not substitutes for professional therapy.
And yet, the benefits are real. Consider Therabot, a mental health chatbot that, according to its developers, reduced depressive symptoms by 51 percent and downgraded moderate anxiety to mild in many users after just a few weeks of interaction. Ash, in a 10-week study, saw 72 percent of users report decreased loneliness, 75 percent increased social support, and four out of five say they felt more hopeful and engaged in their lives. These are remarkable numbers, especially in the context of a national shortage of mental health providers that leaves millions of Americans without adequate care.
But as the FDA considers tightening regulations, critics warn that classifying these chatbots as medical devices would be a costly mistake. Registering with the FDA is no small feat. Beyond the hefty annual fee, companies face layers of government red tape, from premarket performance documentation to ongoing reliability assessments. For startups and smaller companies, these costs could be prohibitive—potentially stifling innovation and making wellness tools less accessible to the very people who need them most.
“Imposing costly regulations on a technology that provides significant benefits will harm Americans who are seeking help,” one industry observer noted. The concern is that, by treating general wellness chatbots like clinical devices, the FDA could inadvertently limit options for people looking for low-cost, always-available support.
There’s another wrinkle: safety. Some advocates worry that, without regulatory scrutiny, AI chatbots could become unsafe spaces, dispensing advice that’s inappropriate or even dangerous. But companies are already taking steps to address these risks. OpenAI, for example, incorporated input from mental health professionals into ChatGPT’s design to help the chatbot recognize distress, de-escalate sensitive conversations, and encourage users to seek in-person care when needed. Anthropic, the company behind Claude, has partnered with ThroughLine, a global crisis app staffed by mental health professionals, to shape how its AI handles difficult topics.
Unlike general-purpose chatbots, specialized mental health tools like Ash, Earkick, Elomia, and Wysa rely on expert input and scientific evidence to continually improve their interactions. These safeguards are designed to ensure that users receive supportive, responsible guidance—without crossing the line into clinical care.
Still, the boundary between wellness and medicine isn’t always clear. The recent experiences of patients like Mollie Kerr and Elliot Royce, reported by The New York Times, illustrate both the promise and the peril of AI in health care. Kerr, a 26-year-old New Yorker living in London, was alarmed by hormone imbalances in her summer bloodwork. Too anxious to wait for her doctor, she pasted the results into ChatGPT. The chatbot suggested her symptoms "most likely" pointed to a pituitary tumor or a rare related condition. While her doctor agreed to order an MRI, no tumor was found—ChatGPT’s guess, though plausible, turned out to be wrong.
For Royce, age 63, the story played out differently. After uploading five years of medical records—including documentation of a heart condition and a past heart attack—he consulted ChatGPT about new discomfort during exercise. The chatbot advised a more invasive diagnostic procedure than his doctor initially recommended. Royce pushed for the test, which revealed an 85 percent blockage in an artery. A stent was promptly inserted, potentially averting a serious medical crisis.
These cases highlight the double-edged sword of AI-powered advice. On one hand, chatbots can prompt patients to advocate for themselves, sometimes catching problems that might otherwise go undiagnosed. On the other, their suggestions are not always accurate, and relying on them for diagnosis or treatment can lead to unnecessary worry, expense, or even harm.
That’s why most AI mental health chatbots draw a bright line: they do not diagnose, treat, or cure. Their value lies in providing support, not clinical answers. They’re more akin to educational tools developed by licensed professionals than to medical advisers. Their advice, while sometimes therapeutic, does not establish a clinical relationship or offer personalized diagnosis.
As the FDA weighs its next steps, the agency faces a delicate balancing act. On one side, there’s the imperative to protect patients from harm and ensure that digital health tools are safe and effective. On the other, there’s the risk of overregulation—of making it harder for people to access affordable, accessible support in a time of growing need.
For now, the consensus among many experts and industry leaders is clear: AI mental health chatbots, when honestly marketed and responsibly designed, should not be regulated as medical devices. Doing so could stymie progress, raise costs, and ultimately harm those seeking help. The FDA’s decision in the coming months will shape not only the future of digital mental health, but the broader relationship between AI and medicine in America.
