Comment by simonw

Why did that experience lead you to that conclusion?

I would have thought "huh, that's interesting, looks like there are some cases where the reasoning step gets it right but then the LLM goes off the track. LLMs are so weird."