Comment Comment by fxtentacle „273 GB/sec memory bandwidth“Really? Less RAM bw than an Epyc CPU? And 4x to 8x less than a consumer GPU?How come this doesn’t massively limit LLM inference speeds? parent fxtentacle 1 year ago
Really? Less RAM bw than an Epyc CPU? And 4x to 8x less than a consumer GPU?
How come this doesn’t massively limit LLM inference speeds?