Comment by YetAnotherNick

How did author just assumed that CPU are competetive for inference. Maybe yes if you just want to run 7 billion parameters model with batch size of 1, but with batching(including continuos batching of vllm), GPU have 2 order higher throughput. And even assuming moore's law is well alive, it will take decade to reach current GPU throughput. There is no way companies will shift to CPU for inference.