Comment by airstrafer

Yeah, I'd expect model performance to be super spiky on SWE work, at least they admit it with the name of the model. It's distilled from an already-distilled model.

Maybe still worth it if their "64% cheaper" figure holds.