Comment Comment by typon There is ton of room for improvement "down there".* Software inference optimizations* Heavy quantization* Chips with hardcoded transformer architecture* Much cheaper HBM* Much sparser models - 1T total with ~1-10B active params e.g.* Not to mention - 2 years of today's frontier models writing RTL and kernels at superhuman levels. parent typon 3 months ago
* Software inference optimizations
* Heavy quantization
* Chips with hardcoded transformer architecture
* Much cheaper HBM
* Much sparser models - 1T total with ~1-10B active params e.g.
* Not to mention - 2 years of today's frontier models writing RTL and kernels at superhuman levels.