Comment by dada216

2000$ dollars per month will get you 8xAMD-MI300X on Oracle.

I have deployed multiple setups with 2/4/8 x H100/200 to do data entry with LLMs at big companies. Trillions of tokens already inferenced ok those. The starting price is about 100k.