A good mental model is to compare the number of floats being processed -vs- the number of primitive computations. Matrix multiplication has n^3 computation with n^2 data. Multiplication of large matrices is therefore special in the potential for "data re-use" (each float is used against all the columns or rows of the other matrix) -- so systems are designed to have a much higher flops throughput than memory bandwidth. A dot product is at the other extreme, where each float is used only once (loosely).
Roofline plots [1] are framework to visualize system design from this perspective.
Roofline plots [1] are framework to visualize system design from this perspective.
[1] https://en.wikipedia.org/wiki/Roofline_model