1x MI300x has 192GB HBM3.
1x MI325x has 256GB HBM3e.
They cost less, you can fit more into a rack and you can buy/deploy at least the 300's today and 325's early next year. AMD and library software performance for AI is improving daily [0].
I'm still trying to wrap my head around how these companies think they are going to do well in this market without more memory.
I'm still trying to wrap my head around how these companies think they are going to do well in this market without more memory.
[0] https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html