fastball
Joined 14 years ago
Joined 14 years ago
OpenAI is back to using Nvidia chips (with small batch sizes) for the ultrafast mode of their latest models.
That takes the speed down from from 750tok/s on Cerebras to 300tok/s, and presumably is much more expensive than the Cerebras solution, but Cerebras has supply constraints. I believe they are planning aggressive data center expansion in this quarter but much harder to scale up than Nvidia chips.
That being said, I'm sure a frontier LLM could make some improvements to your CNN.
On the other hand, the fact that Astra is "winning" this bench is not great signal for me, since imo Astra produces basically unreadable code. Astra plays code golf. My own revealed preference is that I'd rather have the excessive comments and tests of Claude rather than the inscrutability of Astra.
Accelerating the ability for any given person to have access to some amount of these capabilities is terrifying, is it not?
Engineering at Known (https://known.com)
Co-founder and tinkerer at Supernotes (https://supernotes.app)