For the same reason Google's TPU was an order of magnitude more efficient than Nvidia's previous "pure" GPUs, or why video codec accelerators are also faster and more efficient than GPUs for video decoding. The GPU is still pretty "general purpose" compared to a chip that only does image processing and not much else.
I hope Google will eventually reveal if the IPU shares any DNA with the TPU at all.
I hope Google will eventually reveal if the IPU shares any DNA with the TPU at all.