mcmatterson
Joined 16 years ago
Joined 16 years ago
Now that I have you on the hook, is there any benefit to this over virtio for a single KVM passthrough situation? I previously ran a proxmox based gaming PC setup (docs here: https://github.com/mtrudel/rabble/tree/4d9329f3dd0fb09123a8f...), and was lucky enough that the GPU passthrough part of that build 'just worked'. I'd started down a path of trying to share the GPU between VMs based on a naive 'one VM owns it at a time' setup, but never really got it off the ground.
Security findings are one place where we as maintainers simply do not have the choice to not play ball, whether we like it or not. It seems likely that the only way that we meet the moment is to adopt these tools ourselves -- once again -- whether we like it or not. Reconciling this with the ground truth that 'OSS doesn't owe anyone a goddamn thing' is proving to be really hard for me.
We've ceded HTTP specification development to the big guys, and in so doing have made it more or less impossible to implement without resources on their scale. Have you looked at RFC 9000 et al? They're monstrously big, far larger than most independent shops could ever hope to economically pull off. The only way to comprehensively implement something of that scale is to have Google level resources to throw entire teams of engineers and years of focus at it.
I've long said that any protocol worthy of being foundational should be reasonably implementable as a fourth-year term project. It doesn't have to be production ready or ergonomic or even generally useful, but if a group of fourth year CS students can't pull an end-to-end implementation together in a semester, the protocol is just too complex. It's not perfect, but it's as good of a yardstick as I've found.
HTTP/1 passes this test easily; you can make a working version of it in about ninety seconds right in your terminal. HTTP/2 looks intimidating at first glance, but it's so much better specified than HTTP/1 that it's almost easier to get to a reasonable implementation. HTTP/3 on the other hand is...... well, weeks (if not months) of work just to get a QUIC foundation working reasonably well enough that you could hope to start iterating on connections from a 'real' peer, and THEN you have to start on RFC 9114. Not to mention that the way it's structured you end up doing most of that work in the dark, hoping that you line everything up just so so that your first Hello World actually works. It's a way of working that is completely at odds with the hacker ethos that the best foundational protocols have in spades, and ends up looking and acting like what it is: a tool by the big guys, for the big guys. The rest of the internet need not apply.
https://github.com/mtrudel/bandit/blob/main/lib/bandit/http2...
TBH, from an implementors perspective this is a super obvious thing to cover off. It had long been on my radar and was something that I'd always figured other implementations had defended against as well.
A couple of things:
0. The physical quality of this build is out of the world good. PCB, plastics, switches, it's all amazing
1. The software as provided has a pretty old school build process (part of the charm?). I tightened up a bunch of it and dockerized it at https://github.com/mtrudel/pibox/tree/main/pidp11
2. I wish the build would have used something like an MCP23017 for IO instead of claiming so many RPi GPIOs. There's only a few (2-3 IIRC) GPIOs unused by the front panel, and the matrix LED/switch scan setup burns a ton of CPU
I will say though, that I've also noticed the contrast before with MIT grads, who tend to have a very strong LISP bent to their styles. It's true that each school has their own unique flavour, and much like accents it may just be that you don't notice your own.
What a throwback!
Interrupt code is at https://gist.github.com/mtrudel/c29fa60e5b2f3b6fdc46a9e3c65d.... I've been meaning for years to clean this stuff up and resurrect it. Maybe this is the kick in the ass I need to finally do so!
Anyone who's done the UWaterloo trains course will recognize these patterns immediately, and (IIRC) interrupt dispatching is was done in a similar manner there as well.
Finally, the supervision patterns here strike me as being very similar to those within the BEAM, and remind me of the infamous quote from Robert Virding (http://erlang.org/pipermail/erlang-questions/2008-January/03...). Obviously a necessary reimplementation here, but humorous nonetheless.
Great project, great talk.
More broadly, there's just less functionality in Bandit (by design; it's a very focused library that doesn't need to do a bunch of the stuff that Cowboy does). All that stuff ends up adding up when you're doing it 100k a second.
This correlates with Bandit's relative numbers being better in HTTP/1 (up to 5x faster than Cowboy) than they are in HTTP/2 (a paltry 2.3x faster than Cowboy).
In terms of the 32 concurrent connections peak, my hunch here is that it's due to the server the test is running on being a 32 core VM (see https://github.com/mtrudel/network_benchmark for details of the benchmarking process). Truthfully I'm no expert with benchmarking & this may not hold a lot of truth; at the end of the day the test is pretty apples to apples so I think it has comparative merit, but the absolute numbers are a bit murkier to me.
Specialist in net-negative commits.
t: @mattrudel h: http://mat.geeky.net e: [email protected] p: TO/Waterloo