Comment by JBits

The original 'Attention is all you need' paper also used random variables to explain the sqrt(d_k) factor. Even if the keys aren't random, it seems reasonable to treat them as random variables when calculating the 'interference'.