Comment

Comment by scheme271

The dedup functionality in something like zfs or btrfs isn't all that great. It tends to be extremely memory hungry and to slow things down significantly. E.g. ZFS needs around 1-5GB of ram per TB of storage and writes need to be compared to a hash table to dedup properly.

Using hints or knowledge at the app level is a much better experience if the app can tell the FS that two files are identical. The FS doesn't have to worry about hashing blocks within the file, correcting alignments, etc.

parent scheme271 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton