>Not really, because of emergent behaviours and also because there are still millions of neurons being summed (with variable weighting depending on inputs) at each layer.
Emergent behaviors in terms of seemingly random combinations of features producing unpredictable but coherent/intentional output basically don't exist. In other words, there are no "hidden thoughts" in an LLM encoded in higher order combinations of features. Inasmuch as non-semantic features of the input influence output, it does so in non-semantic ways that are degenerate in terms of the quality of output. The only thing emergent in LLMs is degenerate state leading to a breakdown in tracking relevant features of the input and a reduction in quality of output. But the relevance of this kind of degeneracy to LLM output is at the margins. Tracking activated state predicts subsequent output to a high degree of accuracy.
>but it isn't straightforward at all, and is generally very messy and you get all sorts of unpleasant sides effects (e.g. abliterated models not performing as well).
I expect this has as much to do with our lack of precision in our tools rather than with the concept in principle.
Emergent behaviors in terms of seemingly random combinations of features producing unpredictable but coherent/intentional output basically don't exist. In other words, there are no "hidden thoughts" in an LLM encoded in higher order combinations of features. Inasmuch as non-semantic features of the input influence output, it does so in non-semantic ways that are degenerate in terms of the quality of output. The only thing emergent in LLMs is degenerate state leading to a breakdown in tracking relevant features of the input and a reduction in quality of output. But the relevance of this kind of degeneracy to LLM output is at the margins. Tracking activated state predicts subsequent output to a high degree of accuracy.
>but it isn't straightforward at all, and is generally very messy and you get all sorts of unpleasant sides effects (e.g. abliterated models not performing as well).
I expect this has as much to do with our lack of precision in our tools rather than with the concept in principle.