Comment by scarmig

LLMs have basic reasoning and a whole lot of memorization. Through that basic reasoning and pruned search, combined with piles of compute, you can prove lots of things. But the memorization of human failure prunes that possibility, and you need to expend effort convincing the LLM not to prematurely prune based on previous human failure.