Comment by szmerdi

Good initiative but it's not a fair comparison IMO. The Gemini Ultra is the one that should've been compared to GPT-4. And, yes, I know it's not available behind an API yet. But does it justify comparing a < 20 billion params model to a model possibly 50x-100x larger without even mentioning this evident gap in their analysis? In the light of scaling law, I believe no.