I hate everything about this on so many different levels.
That people are so lonely that they invest emotionally in an AI is flooring.
People talk about AI being "good for therapy" and similar, but perhaps having a real human being on the other side of an interaction is important in case things go South.
E.g., if someone is depressed, tells an AI what's wrong with their life and that they're suicidal, what if suicide seems like a valid option? There's plenty of prompts to "jailbreak" LLMs and take away "safety" measures.
Some things simply shouldn't be left to an LLM to not "hallucinate" over or go off-script with. Yes, being a therapist is a challenging occupation, but there are plenty who love their career, find it rewarding, and do some excellent work on top of all that.
That people are so lonely that they invest emotionally in an AI is flooring.
People talk about AI being "good for therapy" and similar, but perhaps having a real human being on the other side of an interaction is important in case things go South.
E.g., if someone is depressed, tells an AI what's wrong with their life and that they're suicidal, what if suicide seems like a valid option? There's plenty of prompts to "jailbreak" LLMs and take away "safety" measures.
Some things simply shouldn't be left to an LLM to not "hallucinate" over or go off-script with. Yes, being a therapist is a challenging occupation, but there are plenty who love their career, find it rewarding, and do some excellent work on top of all that.