Comment by nrub

This is exactly what Taalas has already done, and the reason they got quickly acquired by AMD. Their chip runs Llama 3.1 8B at 17k tok/s, and even if llama 3.1 is dated, I can think of many problems I could use it for, especially at those speeds. They're certain to be working on a newer set of weights by now.