Recent submissions

Comment · 2 years ago
PTHash and other minimum perfect hash functions return an arbitrary value if the query key did not exist when building the MPHF, so they can be a lot smaller. B-field can identify query keys that don't exist in the set (with high probability?).

What I'm wondering is why the Kraken2 probabilistic hash table doesn't work. It uses 32 bits per element in an open addressing hash table. For 1 billion k-mers and 19 bits for the value, 32 - 19 = 13 bits of the key hash can be stored alongside the value, helping disambiguate hash collisions. If the load factor is 1.25x, then that's 4 * 10^9 * 1.25 = 5GB total, better than ~7GB. Also, this is only one cache miss (+ linear probing that can be SIMD accelerated) per lookup.

Comment · 5 years ago
Comment · 5 years ago
Comment · 5 years ago
Comment · 6 years ago
For binary trees, indexing can be done by saving the subtree size of each node and doing a sort of binary search. Not sure if this is fast for B-trees that have more than 2 children nodes, though.
Comment · 6 years ago
I'm a high school student that is quite new to Rust, and I am liking it so far. I learned a lot from this project and I hope that it can be a good learning resource for others.

In addition to this, I am also working on a SIMD edit distance library in Rust.

Comment · 6 years ago
Since I am storing the entire DP matrix as diagonal vectors that are flattened, I don't think there will be many cache misses. Each diagonal only depends on its previous two diagonals, and each diagonal is stored contiguously in memory.

The problem with handling diagonals is that indexing cells and comparing characters on the diagonal becomes complex. Dealing with this without many branches (less branch mispredictions) is the hard part.

Comment · 6 years ago
I took a look at the code, and read the paper. It seems that they directly calculate the entire 2D DP array, but use SIMD to allow each cell to contain multiple values, one for each query string. My approach uses anti-diagonals instead, but it is fast for one vs one comparisons, instead of handling multiple query strings.

Regardless, my goal was to learn some SIMD and Rust (first time for both), so I did not read many background papers.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton