1. the training data contained samples demonstrating how to do this, or it was post trained into the model intentionally;
2. the training data contained enough predicate knowledge that they could reason through it themselves, or;
3. they invented a novel approach using their vast intelligence (/s).
These are in order of likelihood. Given the high probability that it was (1), can't we just ... not do that? Does the frontier model need to be as good at recipes as offensive cybersecurity?
Either:
1. the training data contained samples demonstrating how to do this, or it was post trained into the model intentionally; 2. the training data contained enough predicate knowledge that they could reason through it themselves, or; 3. they invented a novel approach using their vast intelligence (/s).
These are in order of likelihood. Given the high probability that it was (1), can't we just ... not do that? Does the frontier model need to be as good at recipes as offensive cybersecurity?