Comment by kolinko

Worth noting that it’s about CLIP which is way simpler than transformer-based LLMs. CLIP is basically a mapping from one embedding to another.