Comment by spIrr

> limited by the number of activated parameters

not sure I got it?

Separately, the frontier labs are kinda pushing us into that behaviour by releasing models with ever-larger context windows.