And knowing that structure is about as meaningful as knowing "a PC consists of a keyboard, on which you type, a screen, at which you look, and a processor, which does things with binary logic".
None of that helps you understand how exactly LLMs do what they do. Because it describes an interface, not a mechanism.
The inner mechanisms of an LLM are more learned than designed. We know what an LLM does on a low level, but going from that to understanding how they work is like trying to understand how a web browser works by looking at netlists of a CPU. Low level understanding does not grant you high level understanding for free.
But ignoring all of that lets you cling to a very comforting "we understand LLMs because we made them". Ha ha. As if.
> And we also know that human beings do not hold 'internal representations' like any AI system needs to.
Bold fucking claim. Got a source on that?
Because neurobiology has been trying to crack neural representations - the very internal representations brains use - for as long as it existed, and with some success. Both reading and injecting internal representations into the brain is possible now, in narrow cases. The specifics vary region to region, but sparse population coding is a true staple. Today's SOTA for wrangling this mess is ML decoders, and not by a coincidence.
None of that helps you understand how exactly LLMs do what they do. Because it describes an interface, not a mechanism.
The inner mechanisms of an LLM are more learned than designed. We know what an LLM does on a low level, but going from that to understanding how they work is like trying to understand how a web browser works by looking at netlists of a CPU. Low level understanding does not grant you high level understanding for free.
But ignoring all of that lets you cling to a very comforting "we understand LLMs because we made them". Ha ha. As if.
> And we also know that human beings do not hold 'internal representations' like any AI system needs to.
Bold fucking claim. Got a source on that?
Because neurobiology has been trying to crack neural representations - the very internal representations brains use - for as long as it existed, and with some success. Both reading and injecting internal representations into the brain is possible now, in narrow cases. The specifics vary region to region, but sparse population coding is a true staple. Today's SOTA for wrangling this mess is ML decoders, and not by a coincidence.