Comment by midko

Indeed, and that's why there are a couple of startups working on new chips and why Google has the TPU. Here's a nice technical talk from Graphcore's CTO about that https://youtu.be/Gh-Tff7DdzU