My point is simply this: Chinese labs going to great lengths to distill Claude is evidence that distillation is useful but not that that is what makes them so good. That one doesn't even need them to admit it.
Nathan Lambert has made this distinction explicitly: he thinks Chinese labs likely innovate heavily on distillation, while saying he wouldn't call it a crucial factor in their post-training capabilities, in part because the RL work still has to be done by the lab itself -- just like what I said above.
Nathan Lambert has made this distinction explicitly: he thinks Chinese labs likely innovate heavily on distillation, while saying he wouldn't call it a crucial factor in their post-training capabilities, in part because the RL work still has to be done by the lab itself -- just like what I said above.
https://www.interconnects.ai/p/how-much-does-distillation-re...