Comment Comment by charcircuit Scale is all you need.That's why you see other attention free large language models work. But in a world with constraints on time and compute the architecture does matter because scaling does have a cost (which is dropping) parent charcircuit 6 days ago
That's why you see other attention free large language models work. But in a world with constraints on time and compute the architecture does matter because scaling does have a cost (which is dropping)