Comment

Comment by infogulch

I implemented the 1brc in go a few weeks back [1] and this repo was inspiration to try to find a custom perfect hash function that put each station into its own bucket with no collisions. Then I noticed the rule that you can't customize the hash function to the data before the program starts and dropped the idea.

I made a test harness to check random constants (start values, multiplication constants, shift/rotate amounts, etc) and print out the best constants found so far based on the number of colliding buckets and how many collisions. I think I got it down to 1 bucket with just two colliding values with a ~40% fill rate. Interestingly, I found that the best performing constants included similar values for the number of positions to shift regardless of the other constants, so I ended up hard coding them.

[1]: https://github.com/infogulch/1brc-go

parent infogulch 2 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton