Comment by cma

Google TPUs use optically routable fibers like circuit switching to be able to change the whole topology of the setup depending on training load/matrix dimensions.