Comment by divyekapoor

Why do you need a cryptographic hash function for detecting duplicates? xxHash, rapidHash or anything that’s fast will do the job. There’s no need to be secure against inverses.