This is erring close to letting an LLM give medical advice to people without them fully understanding what LLMs are. People seeking help with their mental health might not be in the clearest headspace to understand the nuances of using a chatbot vs their own research or seeking a professional.
Edit0: Problem coming from the possibility of the LLM giving bad advice leading to a negative outcome for the person seeking service.
Edit0: Problem coming from the possibility of the LLM giving bad advice leading to a negative outcome for the person seeking service.