Their example prompt is so bad that I'm split between them being wildly incompetent at understanding how transformers or autoregressiveness works (and general LLM dynamics) or they are deliberately obfuscating the task and prompt representation in bad faith.
If one really wants to know if LLMs "are capable of planning", one should keep an open mind for how planning behavior can manifest in textual form, then actually try to find any manifestations. Imposing one's view of how planning should look like is bad science. All they've proved is that their task format sucks for current generation LLMs.
If one really wants to know if LLMs "are capable of planning", one should keep an open mind for how planning behavior can manifest in textual form, then actually try to find any manifestations. Imposing one's view of how planning should look like is bad science. All they've proved is that their task format sucks for current generation LLMs.