Comment by lucubratory

I don't know how far "pure" next-token prediction can go with planning, although I wouldn't count them out until their performance starts noticeably plateauing. But the tree of thought architecture is a very similar concept to what you're discussing, you should definitely give it a read: https://arxiv.org/abs/2305.10601

It's not everything involved in traditional planning, but it may be a framework to use more traditional planning algorithms on LLM output.