Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work.
I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!
I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!