AI Can Be a Game-Changer for Mental Health Support – But It’s an Ethical Minefield

Using artificial intelligence (AI) to support mental health patients is an ethical minefield.

When used correctly AI, along with machine learning (ML), can help to identify new treatments and accelerate patient treatment. But if it’s used incorrectly, it can result in misdiagnosis and stop vulnerable individuals from getting the support they need.?

At the same time, mental health practitioners are in short supply. With the World Health Organization (WHO) estimating that almost a billion people were living with a mental disorder as of 2019, there is a significant shortage of counselors, psychiatrists, and psychologists to support patients.?

In this climate, we’ve started to see software vendors creating apps and chatbots, such as Woebot and Wysa, that use AI to support users with mild symptoms of conditions like depression and anxiety. With these chatbots, users can talk about their emotions and receive basic support and guidance from an automated agent.?

While studies show that many users find these apps useful, they aren’t without their own risks. For instance, earlier this year, a Belgian man committed suicide after the AI chatbot Chai allegedly encouraged the user to kill himself after six weeks of back and forth.

In this case, the AI chatbot generating harmful responses may have influenced a vulnerable individual to take his life.?

The Central Ethical Argument Around AI in Mental Health

When considering that the stakes of using AI in healthcare are as dramatic as life or death, it’s on mental health practitioners, clinical researchers, and software developers to define a level of acceptable risk around using the technology.?

For example, if a software vendor creates a chatbot for users to discuss their symptoms with, they need to have well-defined guardrails to lower the risk of the solution hallucinating facts. Basic guardrails could include a disclaimer and live support services from qualified professionals to act as an extra layer of security.?

At a high level, any entity that attempts to deploy AI to support user’s needs to identify whether deploying artificial intelligence is putting vulnerable users at risk or accelerating their ability to get access to treatment or support.?

As one mental health researcher argues, “artificial intelligence has immense potential to re-define our diagnosis and help in better understanding of mental illnesses. A person’s mental illnesses…artificial intelligence technologies may have the ability to develop better pre-diagnostic screening tools and work out risk models to determine an individual’s predisposition for, or possibility of developing mental illness.”

That being said, solutions that use AI to diagnose mental illness need to be built on the highest quality training data to ensure optimal accuracy. Any inaccuracies in the dataset could lead toward a misdiagnosis or improper treatment for patients who are in need of assistance.??

Using AI in a mental health context is an area where the ends justify the means. If AI can be used to enhance a patient’s access to support and streamline drug discovery, then it’s a net positive. If it results in misdiagnosis, misinforms, or prevents vulnerable people from getting access to clinical support, then it’s a no-go.?

Balancing Privacy and Support?

Perhaps one of the most significant considerations in the AI ethics debate, is how the data that powers these solutions is collected, stored and utilized by AI systems. This includes everything from an individual’s personal data to sensitive emotional and behavioral information.

At a minimum, clinical researchers and software vendors processing patient data need to ensure that they have the informed consent of the individuals, or the data needs to be de-identified or anonymized so that personal identifiable information (PII), electronic protected health information (EPHI), and medical records, aren’t exposed to unauthorized third parties.?

The requirements for doing this can be extremely complex, particularly when regulations like HIPAA put strict data protection requirements on electronic health data, and even anonymization can be deciphered if it isn’t protected adequately.?

For this reason, many providers are extremely selective in the data that they use to power AI applications, so they can avoid compliance liabilities. While this helps to protect user privacy, it does so by reducing the amount of overall data available for processing.?

Ultimately there is a balancing act to be struck between protecting the anonymity of patient data and obtaining informed consent, and still gathering enough data to offer high-quality insights to inform treatment and diagnosis.?

The Bottom Line

If AI consistently delivers positive outcomes for patients, then it will justify itself as a tool for mental health practitioners to turn to.?

We’re already seeing the success of AI at diagnosing and discovering treatments to address schizophrenia and bipolar, and if this trend continues, then there will be much less anxiety over experimenting with these technologies in the industry.?

In contrast, if more chatbots hit the news cycle for failing to support mental health patients, then it could set back AI in the sector significantly. With the ethics of AI use in healthcare still being defined, it’s on researchers, practitioners, and software vendors to set the standard for ethical AI development.