Comment by yosefk

A lot of people including "me" (TFA author, of course "me" = larger team) were doing hardware matrix multiplication for many years before 2012 [let's say before deep learning.] I count "moving data around" as "dispatching costs" [dispatching = figuring out what to do, on what data and where to put results as opposed to actually doing it.]