I believe there's a confusion in this thread between what an LLM _does_ and what an LLM _is_. What is does is output a next token. What it is, is a universal function approximator ([1], i.e. a neural net).
With back probation in the neural net, there could be a full state machine being approximated inside the weight. And a state machine is the exact step-wise reasoning the author claims it needs.
I believe there's a confusion in this thread between what an LLM _does_ and what an LLM _is_. What is does is output a next token. What it is, is a universal function approximator ([1], i.e. a neural net).
With back probation in the neural net, there could be a full state machine being approximated inside the weight. And a state machine is the exact step-wise reasoning the author claims it needs.
[1]: https://en.wikipedia.org/wiki/Universal_approximation_theore...