I recently ran some machine learning benchmarks on CPU versus GPU. The gap for a multicore CPU was much smaller than I expected. I, for one, am excited by a 64-core CPU in a way I wouldn't have been a year ago.
Between Hugging Face, Stable Diffusion, and Whisper, I'm using ML workloads a lot more. Being able to do so:
* with a standard instruction set
* with open-source software
* with my full system RAM
* without having to worry about what is in VRAM versus main RAM
is a big step up. I see about a 10x speed difference between an older 16-core CPU and a hot-off-the-press high-end Ampere card costing 3x as much as the CPU. If 64 core could bring that within 2x, or even 4x, I'd dump the GPU entirely.
Between Hugging Face, Stable Diffusion, and Whisper, I'm using ML workloads a lot more. Being able to do so:
* with a standard instruction set
* with open-source software
* with my full system RAM
* without having to worry about what is in VRAM versus main RAM
is a big step up. I see about a 10x speed difference between an older 16-core CPU and a hot-off-the-press high-end Ampere card costing 3x as much as the CPU. If 64 core could bring that within 2x, or even 4x, I'd dump the GPU entirely.