Comment by ZedZark

I did this with two scripts - one that produces and cached sha1 sums of files, and another that consumes the output of the first (or any of the *sum progs) and produces stats about duplicate files, with options to delete or hard-link them.