> But at the the day they're regression/forecast machines.
This level of understanding of LLMs may have been true a few years ago, but we're past far that. LLMs have solved a Millennium Prize problem, just in case.
Having emergent properties implies that an LLM has capabilities beyond the underlying mechanics (stochastic and the like), therefore to recognize emergent properties and at the same time to state that they are just regression machines is self-contradictory.
This is for example an interesting research, that investigates the existince of an internal state of truthfulness: https://aclanthology.org/2023.findings-emnlp.68.pdf. There's a lot of research on the higher level properties of LLMs.
This level of understanding of LLMs may have been true a few years ago, but we're past far that. LLMs have solved a Millennium Prize problem, just in case.
Having emergent properties implies that an LLM has capabilities beyond the underlying mechanics (stochastic and the like), therefore to recognize emergent properties and at the same time to state that they are just regression machines is self-contradictory.
This is for example an interesting research, that investigates the existince of an internal state of truthfulness: https://aclanthology.org/2023.findings-emnlp.68.pdf. There's a lot of research on the higher level properties of LLMs.