>you must be using some completely idiosyncratic definition of "emergent" here.
I doubt my usage is idiosyncratic. Rather, the term is so ill defined that people often use it inappropriately. Emergent means something along the lines of not easily predictable behavior from the behavior of the component parts. My claim is nothing in LLMs is emergent in the sense of being inexplicable from an ideal consideration of simple/linear aggregations of its neurons. This isn't something I can prove, but it certainly isn't obviously false from just considering recent mathematical results. Higher order/non-linear interactions are characteristic of this kind of inexplicable behavior from complex interactions we characterize as emergent. But there are good reasons to think this kind of non-linearity in any semantic regime is next to impossible in principle.
Semantic geometric representations are characterized by linearity in response dynamics such that a linear change in one variable results in a linear semantic change in related variables. That's how vector embeddings capture semantic relationships. Call it the linearity hypothesis. Non-linear dynamics in the embedding space just destroys semantic relationships in most cases. There is no way to keep an entire network of relationships sufficiently correlated if non-linear response dynamics is a part of the representational toolbox. Non-linear dynamics in general kills correlations.
Testing this against recent LLM proofs, what would need to be shown is that the relevant neural activations do not decompose the semantic content of the proofs (or the reasoning trace) in a straightforward manner, but rather the reasoning trace is unpredictable from the semantic content of the activated neurons that precede the tokens of the reasoning trace. But this idea strains credulity; it is almost inconceivable that this would be the case. We should be highly incredulous of the idea that complex non-linear interactions can be wholly responsible for the long-running coherent output in LLMs. The default hypothesis should be that there is no emergent behavior (as I defined it) in neural networks.
I doubt my usage is idiosyncratic. Rather, the term is so ill defined that people often use it inappropriately. Emergent means something along the lines of not easily predictable behavior from the behavior of the component parts. My claim is nothing in LLMs is emergent in the sense of being inexplicable from an ideal consideration of simple/linear aggregations of its neurons. This isn't something I can prove, but it certainly isn't obviously false from just considering recent mathematical results. Higher order/non-linear interactions are characteristic of this kind of inexplicable behavior from complex interactions we characterize as emergent. But there are good reasons to think this kind of non-linearity in any semantic regime is next to impossible in principle.
Semantic geometric representations are characterized by linearity in response dynamics such that a linear change in one variable results in a linear semantic change in related variables. That's how vector embeddings capture semantic relationships. Call it the linearity hypothesis. Non-linear dynamics in the embedding space just destroys semantic relationships in most cases. There is no way to keep an entire network of relationships sufficiently correlated if non-linear response dynamics is a part of the representational toolbox. Non-linear dynamics in general kills correlations.
Testing this against recent LLM proofs, what would need to be shown is that the relevant neural activations do not decompose the semantic content of the proofs (or the reasoning trace) in a straightforward manner, but rather the reasoning trace is unpredictable from the semantic content of the activated neurons that precede the tokens of the reasoning trace. But this idea strains credulity; it is almost inconceivable that this would be the case. We should be highly incredulous of the idea that complex non-linear interactions can be wholly responsible for the long-running coherent output in LLMs. The default hypothesis should be that there is no emergent behavior (as I defined it) in neural networks.