Comment by beejiu

Perhaps a more common question: "How many calories do men need to lose weight?"

Google AI responded: "To lose weight, men typically need to reduce their daily calorie intake by 1,500 - 1,800 calories"

Which is obviously dangerous advice.

IMO Google AI overviews should not show up for anything (a) medical or (b) numerical. LLMs just aren't safe enough yet.