There was a good discussion about that here recently. I think people are misinterpreting the bitter lesson a bit.
It tells you that you don't need to craft special strategies for pinning, holding the centre and so on to make a program that's good at chess; you just need general-purpose search, general-purpose optimization and tons of training data.
It does not say that a fully general LLM or similar will necessarily be better at chess than a neural net trained against the chess rules and nothing else. There's no reason to think that any of the non-chess knowledge encoded in the LLM will be of any use in playing chess. To the contrary, the bitter lesson says that absolutely nothing besides the chess rules is needed. That's exactly why human-crafted strategies turn out not to be helpful.
An LLM on its own will never ever beat Stockfish, pound for pound. (Of course it doesn't need to, it can just use Stockfish.)
So I'd say problem-specific models will always beat general models, as long as the problem domain and scoring are completely defined, like chess and go are. It's CPUs versus ASICs. As soon as the problem is open-ended, a general model probably wins. Driving is in that category, although maybe on the borderline -- driving in very specific environments is close to fully defined. OCR is another good example, that's mostly well-defined but not completely so.
It tells you that you don't need to craft special strategies for pinning, holding the centre and so on to make a program that's good at chess; you just need general-purpose search, general-purpose optimization and tons of training data.
It does not say that a fully general LLM or similar will necessarily be better at chess than a neural net trained against the chess rules and nothing else. There's no reason to think that any of the non-chess knowledge encoded in the LLM will be of any use in playing chess. To the contrary, the bitter lesson says that absolutely nothing besides the chess rules is needed. That's exactly why human-crafted strategies turn out not to be helpful.
An LLM on its own will never ever beat Stockfish, pound for pound. (Of course it doesn't need to, it can just use Stockfish.)
So I'd say problem-specific models will always beat general models, as long as the problem domain and scoring are completely defined, like chess and go are. It's CPUs versus ASICs. As soon as the problem is open-ended, a general model probably wins. Driving is in that category, although maybe on the borderline -- driving in very specific environments is close to fully defined. OCR is another good example, that's mostly well-defined but not completely so.