Comment by antixk

I guess OP meant that each tensor core in the latest NVIDIA GPUs fundamentaly performs 4x4 matrix multiplication [0].

[0] https://nla-group.org/2020/07/21/numerical-behaviour-of-tens...