Comment by phreeza

I think a lot of LLM progress in the last 2 years or so is really from densifying the training data to niches like this and not so much from general problem solving capability. I guess the hope is that the deluge of super specific training data will pay off through better overall competence. cf https://www.dwarkesh.com/p/the-sample-efficiency-black-hole