The reason is that Dall-E 2 type models are small and can run on a wide class of commodity hardware. This makes them very accessible which means a large number of people can contribute.
Large language models gain key capabilities as they increase in size: more reliable fact retrieval, multistep reasoning and synthesis, complex instruction following. The best publicly accessible is GPT-3 and at that scale you're looking at hundreds of gigabytes.
Models able to run on most people's machines fall flat when you try to do anything too complex with them. You can read any LLM paper and see how the models increase in performance with size.
The capabilities of available small models have increased by a lot recently as we've learned how to train LLMs but a larger model is always going to be a lot better, at least when it comes to transformers.
Large language models gain key capabilities as they increase in size: more reliable fact retrieval, multistep reasoning and synthesis, complex instruction following. The best publicly accessible is GPT-3 and at that scale you're looking at hundreds of gigabytes.
Models able to run on most people's machines fall flat when you try to do anything too complex with them. You can read any LLM paper and see how the models increase in performance with size.
The capabilities of available small models have increased by a lot recently as we've learned how to train LLMs but a larger model is always going to be a lot better, at least when it comes to transformers.