Comment by jhanschoo

To add on, importantly, when Gebru called them stochastic parrots, CoT technique was not published yet, and LLMs had finite computational depth per emitted token, and that architectural limitation was a large contributor to hallucinations. But if you give a sufficiently general LLM file read/write (or arbitrary memory storage) and CoT before having to output a token, it is Turing-complete. Architecturally in principle you can have models that are accurate on studied domains, the question being how to grow a model that achieves this by reading literature and cheap RL on experiments, rather than curated design.