I guess that's why is an active research area, and an interdisciplinary one, what is intelligence? Is what have you practiced a ton of times? Is what you have purposely and efficiently practice? What is the implication of "purposely" doing something and how much memory is involved into it? If memory is involved how much of it is relevant? What is problem solving or wisdom? creativity? How much diversity in your memory do you need for creativity?
So what should you cull and how much?
There's already techniques in CNNs to trim unused or less active neuron paths to reduce a model's size, but how do you (and how much) do it in a general LLM? A product that will be used without supervision from a child to an 80yo elder?
So what should you cull and how much? There's already techniques in CNNs to trim unused or less active neuron paths to reduce a model's size, but how do you (and how much) do it in a general LLM? A product that will be used without supervision from a child to an 80yo elder?