Comment by nuz

Well the counter argument is that with an equal amount of compute, specialized models will win over generalized ones. The reason is that generalized ones have to spread out their weights to be able to do a ton of irrelevant things to the niche subject that a finetuned model can accomplish with fewer and more focused weights.