brrrrrm

I would guess “us vs them” mentalities are not good for productive societies

I think you're confusing the expected behavior of the product offerings. Every user gets their own VM for free. would you be similarly convinced an attack has happened if AWS gave you a remote shell to the instance you rented?

only sorts numbers? wouldn't radix be much better?

this is basically the only thing pre-training teams work on in labs. compute efficiency is the metric, the assumption that scaling = intelligence is considered a given.

they say they're looking at base models, so I think it's fairly compared as written.

this is a nice and concise writeup. what's striking to me is that these techniques really have not changed in /years/. sure, precision has become slightly lower, spec decoding acceptance has gotten slightly better and the complexity of parallelism is trickier with mixture of experts. but no new concepts in a very long time!

the absolute most impactful improvements for inference comes at architecture design time. I firmly believe everyone who cares about impacting model efficiency should look there

are these all uniform quantization? or mixed and matched by layer (can't tell from the naming scheme)

this is cool but like, are we just vibe coding NAND burners at this point? these decode times don't really tell the whole story, because prefill becomes the bottleneck.

half an hour to process 10k tokens on an M5 seems... not great

there's wifi ah, which runs on 900mhz band and has the same 30dbm limitation

I've used it with some raspberry pis to create hi-fidelity walkie talkies it's quite pleasant.

Models are increasingly showing their ability to extrapolate into the human unexplored (math proofs being the most apparent). What gives you confidence the absurdity of life is uniquely difficult for models to source?

> you are basically looking at a whole system prompt just describing the new language

whats wrong with this? You may be over-indexing on the need for large quantities of examples. These days self-play through RL is far more effective and data (not compute) efficient.