Comment by erichocean

Sure, agreed. But they do their own pre-training, at great expense, on outdated model backbones.

Wouldn't it be better to do something more like Cursor, and RL on an existing pretrained model if you're not innovating anyway?