My 2 cents is in 3 years the inference products will be a commodity, extremely competitive with diminishing returns, seeing that the open weight models are getting so good and nearing par with sota.
I feel 90% of sota for 10% of cost / compute is good enough for 80% of workloads.
Question is, do we need all of this hardware? Are build outs going to get canceled. 4300 new data centers seems excessive. I personally haven't experienced any service disruptions...
Does Microsoft still have 1 million gpu's in storage? Is that what I heard earlier this year?
I feel 90% of sota for 10% of cost / compute is good enough for 80% of workloads.
Question is, do we need all of this hardware? Are build outs going to get canceled. 4300 new data centers seems excessive. I personally haven't experienced any service disruptions...
Does Microsoft still have 1 million gpu's in storage? Is that what I heard earlier this year?