"I'd like a car that goes 300mph and gets 100mpg while doing it. I'm aware of a car that gets 100mpg but it is extremely slow."
You are describing fundamental tradeoffs. Getting more performance relative to model size and training token amount is what all of the labs are solving.
You are describing fundamental tradeoffs. Getting more performance relative to model size and training token amount is what all of the labs are solving.