Synthetic data, distilled data, etc., does not necessarily mean data coming from outside of an organization. When we see performance increases with models like Haiku, it is because they are able to leverage the knowledge of larger architectures.
I fully expect that we will see increased performance from smaller architectures (to a point) simply because they are refined using larger ones.
> The problem is if we can pack a Fable in Qwen 4 27B who will pay subscription prices?
This is available to everyone developing these systems. Prices per relative unit of intelligence will continue to decrease. And market economics will continue to drive the cost down.
I fully expect that we will see increased performance from smaller architectures (to a point) simply because they are refined using larger ones.
> The problem is if we can pack a Fable in Qwen 4 27B who will pay subscription prices?
This is available to everyone developing these systems. Prices per relative unit of intelligence will continue to decrease. And market economics will continue to drive the cost down.