Comment

Comment by SillyUsername

You can also pipe it through zstd on the fly, for data that compresses well you'll often see 1.5 to 3× the raw throughput, so a gigabit link can effectively move 165–330MB/s.

Receiver: socat -u TCP6-LISTEN:1234,reuseaddr STDOUT | zstd -d -c | tar -xpf - -C /destination

Sender: tar -C /source -cf - directory | zstd -T0 -6 -c | socat -u STDIN 'TCP6:[fd42:dead:beef::2]:1234'

-T0 uses all cores. Bump the level above -6 for more compression, drop it for more speed, but if your CPU can't keep up, high levels will actually slow it down. Already compressed data won't see much benefit.

parent SillyUsername 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton