Most important piece of information is in the linked Frontiers article:
However, the overall capability of the chatbot to fully meet user needs received a lower average score (3.1/5.0), highlighting the need for further improvements.
Also there is still the problem of hallucinations, as we see in the „Evaluation“ paragraph:
Live traffic evaluations are essential for monitoring system behavior, identifying potential issues like hallucinations in production, and understanding performance on diverse live queries.
This are quite devastating results. This is a system for scientific research on medicines and mediocrity and hallucinations will kill people.
Would be interesting to know how much money was flushed down the toilet with these experts.
Would be interesting to know how much money was flushed down the toilet with these experts.