Prices for everything are all in flux at this moment. Consumer cards like RTX 5090 are bonkers. Even 3090s have like doubled in the last six months. Meanwhile, prices for renting this class of hardware on vast.ai are spiking too, with rates increasing like 50% in September alone.
It's hard to know if we're at the beginning or the end of this trend. If manufacturing continues to focus on high end datacenter gpus at $40k a pop, and we continue to have supply constraints, this could last a while.
On the other hand, if Chinese models continue to nip at the tails of the frontier, cheaper by 20x, running on new Chinese chips, the whole bottom could fall out of the US market, and things could get very weird.
It's hard to know if we're at the beginning or the end of this trend. If manufacturing continues to focus on high end datacenter gpus at $40k a pop, and we continue to have supply constraints, this could last a while.
On the other hand, if Chinese models continue to nip at the tails of the frontier, cheaper by 20x, running on new Chinese chips, the whole bottom could fall out of the US market, and things could get very weird.