There is always a minimum cost of moving data from one place to another. If you're computing on the GPU, the data must arrive there. The problem is that PCIE bandwidth is often a bottleneck, and so if you can upload compressed data then you essentially get a free multiplier of bandwidth based on the compression ratio. If the decompression time is faster than having sent the full uncompressed dataset, then you win.
But yeah, direct IO to the GPU would be great but that's not feasible right now.
But yeah, direct IO to the GPU would be great but that's not feasible right now.