I mean how useful are GPT-3 weights today? That's the kind of useful they'll be relative to the frontier.
I feel like the so-called "Deepseek moment" broke too many people's brains and they've lost the ability to reason about why frontier advantages matter. I think we have to learn some new core concepts and rate of improvement over time is going to be a big one, and not understanding that causes people to exaggerate the significance of downstream distillation that's done in response to it. It's cheaper and open and that matters, but it's not the kind of mattering that replaces the relevance or advantage of the frontier, and I feel like I'm not seeing people demonstrate that they know how to really compare the relative merit of these two things so much as zoom in on one to talk past the other.
I mean how useful are GPT-3 weights today? That's the kind of useful they'll be relative to the frontier.
I feel like the so-called "Deepseek moment" broke too many people's brains and they've lost the ability to reason about why frontier advantages matter. I think we have to learn some new core concepts and rate of improvement over time is going to be a big one, and not understanding that causes people to exaggerate the significance of downstream distillation that's done in response to it. It's cheaper and open and that matters, but it's not the kind of mattering that replaces the relevance or advantage of the frontier, and I feel like I'm not seeing people demonstrate that they know how to really compare the relative merit of these two things so much as zoom in on one to talk past the other.