The LLM absorbs the manifold (high-dimensional shape) of the data. The manifold contains the underlying concepts of grammatical structure, abstract thought, and inductive reasoning, which the LLM (within reasonable capacity and structural capability) captures because it is a more efficient representation of the underlying data.
Transformers are simple and contain the correct building blocks to efficiently capture these representations during training.
This allows for out-of domain generalization.
This is most, to my knowledge, of what happens in most deep learning processes and is likely 90% of the main content that one needs to know to understand neural networks at their core. I think it's pretty basic but it often gets buried in unconscionable mathematical symbols and fancy-speak to be of use to anyone reading.
The LLM absorbs the manifold (high-dimensional shape) of the data. The manifold contains the underlying concepts of grammatical structure, abstract thought, and inductive reasoning, which the LLM (within reasonable capacity and structural capability) captures because it is a more efficient representation of the underlying data.
Transformers are simple and contain the correct building blocks to efficiently capture these representations during training.
This allows for out-of domain generalization.
This is most, to my knowledge, of what happens in most deep learning processes and is likely 90% of the main content that one needs to know to understand neural networks at their core. I think it's pretty basic but it often gets buried in unconscionable mathematical symbols and fancy-speak to be of use to anyone reading.