These models are, in a sense, distillations of language models. They wouldn't be as good without a pre-trained language model, so it's unfair to compare them.
Image models are essentially a mapping of text embedding space -> image. So the clearer and cleaner the text embedding space is the better model you get as you need less weights trying to interpret a text prompt, recent image models even use multimodal versions of LLMs (Flux uses Qwen3 VL) so all the visual separation was done beforehand making the model even more efficient.
edit: Maybe a better analogy is that the image models are only half the model in a encoder-decoder sense, where the LLM is a pretrained frozen encoder and the image model is the decoder.
Image models are essentially a mapping of text embedding space -> image. So the clearer and cleaner the text embedding space is the better model you get as you need less weights trying to interpret a text prompt, recent image models even use multimodal versions of LLMs (Flux uses Qwen3 VL) so all the visual separation was done beforehand making the model even more efficient.
edit: Maybe a better analogy is that the image models are only half the model in a encoder-decoder sense, where the LLM is a pretrained frozen encoder and the image model is the decoder.