Comment by purplesyringa

The right-shift is the problem. You need to shift right by `j * 8`, which itself requires a shift to compute (`j << 3`), so you have two shifts on the critical path, resulting in a latency of 2 cycles. It's better than a load, but it's still noticeable.