> Small models are rapidly growing in capability, require less compute to train and serve
According to Jevons' paradox a reduction in resource requirements (improved resource efficiency for the same payoff) leads to an increase in demand. This stops working when demand for compute is completely exhausted, but we are very far from that. There's even some very silly predictions floating around (see the latest Dwarkesh Patel podcast) that say compute will soon be most of the economy, even dictating market interest rates. Now, that has to be wrong, but the directional outlook is closer to correct than "very small and efficient models mean there will be ~0 demand for HPC-like compute".
According to Jevons' paradox a reduction in resource requirements (improved resource efficiency for the same payoff) leads to an increase in demand. This stops working when demand for compute is completely exhausted, but we are very far from that. There's even some very silly predictions floating around (see the latest Dwarkesh Patel podcast) that say compute will soon be most of the economy, even dictating market interest rates. Now, that has to be wrong, but the directional outlook is closer to correct than "very small and efficient models mean there will be ~0 demand for HPC-like compute".