A single CPU thread should be treated as basically a warp executing 4 simd vectors in parallel.
The naïve implementation of __syncthreads() would be an atomic mechanism shared across all threads that contribute to what is GPU workgroup.
Looks like this entire paper is just about how to move/remove these barriers.
Looks like this entire paper is just about how to move/remove these barriers.