Comment by sosodev

One agent typically blocks the others on a local device because the GPU is already completely utilized either in terms of memory or compute. You can have true parallelism at home, but you need an absurd amount of resources. It's not a simple threading problem.