nelhage
Joined 18 years ago
Karma
1,489
Submissions
68
Member since
Feb 17
Recent submissions
Comment · 1 year ago
Comment · 1 year ago
> Btw, what about some clang18.tc comparison
(post author here) Oh, this is something I could have called out explicitly: The tail-calling interpreter relies on a feature (the `preserve_none` calling convention) that only landed in clang-19. That means you can only test it on that version. That coincidence (that 19 added both this feature, and the regression) is part of why this was so easy to miss at first, and why I had to "triangulate" with so many different benchmarks to be confident I understood what was going on.
Comment · 4 years ago
(author here)
I actually did a v0 writeup in Go, but I wanted a language that had a bit more powerful type system and more support for a fluent/functional style in some of the expressions. I was optimizing for what I know and felt was highly expressive; I've since gotten a bunch of feedback from people who are interested in the content but not comfortable reading Rust, so perhaps it wasn't the best choice.
Comment · 6 years ago
We used precisely this optimization in [sorbet](https://sorbet.org), a brand-new type checker for Ruby, which also contains a high-performance LSP server.
We wrote the entire thing (and tested, using ASAN and fuzzers and other techniques) to avoid leaking memory, and then strategically inserted [the equivalent of a rust `mem::forget`](https://github.com/sorbet/sorbet/blob/0aae56e73c7680ec6053b3...) into the end of the `main` driver during standalone mode, to avoid calling those destructors when we're about to exit anyways.
This optimization is definitely still relevant for new systems today.
Comment · 11 years ago
Better yet, just run this on a t2.micro on a throwaway EC2 account. Doesn't matter if they own the box, they get literally nothing they couldn't get for free from Amazon anyways.
Comment · 11 years ago
zero-width matches (and empty lines) were a huge source of stupid edge-case bugs in livegrep[1], and being rigorous about maintaining this mental model definitely helped a lot.
[1] livegrep.com
Comment · 11 years ago
I get into this a bit later on, but I think the exact same model applies to pointers: You're much better off in most cases thinking of pointers as pointing at the zero-width points between elements, than at elements themselves.
Comment · 11 years ago
Live data, but they're stored redacted, and/or with sensitive data (e.g. credit card numbers) replaced with opaque tokens that reference an encrypted store that's carefully access-controlled.
Comment · 11 years ago
This is a great question, and it's definitely a problem we have.
We don't have a single answer we use for every system we work on, but we employ a few common patterns, ranging from just keeping hard-coded strings containing the expected output, up to and including implementing our own fake versions of external infrastructure. We have, for example, our own faked ISO-8583 [1] authorization service, which some of our tests run against to get a degree of end-to-end testing.
Back-testing is also incredibly valuable: We have repositories of every conversation or transaction we've ever exchanged with the banking networks, and when making changes to parsers or interpreters, we can compare their output against the old version on all of that historical data.
Comment · 11 years ago
Our testing running infrastructure spins up a pool of database instances on each worker machine, one for each worker process. The test spinup and teardown code handles schema management, hooking into our DB access layer to create and clean up database tables only if they're used by a given test.
> The problem is 95% about laying out the instruction dispatching code for the branch predictor to work optimally.
A fun fact I learned while writing this post is that that's no longer true! Modern branch predictors can pretty much accurately predict through a single indirect jump, if the run is long enough and the interpreted code itself has stable behavior!
Here's a paper that studied this (for both real hardware and a certain simulated branch predictor): https://inria.hal.science/hal-01100647/document
My experiments on this project anecdotally agree; they didn't make it into the post but I also explored a few of the interpreters through hardware CPU counters and `perf stat`, and branch misprediction never showed up as a dominant factor.