aeadio
Joined 4 years ago
Joined 4 years ago
For a long time a new garbage collector has been an open issue for LuaJIT, which would fix this particular issue, and make the language even faster. Last I checked this was actively being worked on.
There are very valid complaints about Lua. It lacks useful quality of life features. Some of its design decisions grate on users coming from other languages. Its standard library is barebones verging on anemic, and its ecosystem of libraries does not make up for it.
But after using Lua in many places for well over a decade, I’ve gotta say, this is the first time I’ve heard someone claim it’s slow. Even without JIT, just interpreter to interpreter, it’s consistently 5-10x faster than similar languages like Python or Ruby. Maybe you’re comparing it to AOT systems languages like C and C++? That’s obviously not fair. But if you put LuaJIT head to head with AOT systems languages you’ll find it’s within an order of magnitude, just like all the other high quality JITs.
Which, yeah, I don’t think that’s much of a revelation to anyone discussing Intel’s future. There’s no argument put forward that it’s not still the best path for Intel.
In fact, it's been partially done for FreeBSD, https://github.com/dspinellis/unix-history-repo
We could in principle do something similar for Darwin (if we had enough of the historical code), which is the core of MacOS, which is based on NeXT, which was based on BSD with a new kernel. That makes MacOS every bit as much a member of the Unix/BSD family as FreeBSD is.
Mac OS X was essentially a continuation of NeXTSTEP, which is BSD with a new novel kernel. In fact, if you look into the research history of the Mach kernel at the core of XNU, it was intended as a novel kernel _for_ BSD. NeXT went and hired one of the key people behind Mach (Avie Tevanian), and he became one of the core systems guy that designed NeXTSTEP as a full OS around Mach.
Early in the proliferation of the Unix family, member systems went in one of two directions -- they based their OS on upstream AT&T Unix, or they based it on Berkley's BSD, and added their own features on top. NeXT was one of the latter. Famously, the original SunOS also was.
While Sun would eventually work closely with AT&T to unify their codebase with upstream, NeXT made no such change. NeXTSTEP stayed BSD-based.
The other extant BSDs like FreeBSD and NetBSD were also based directly on the original BSD code, through 386BSD.
If I have my history correct, Apple would later bring in code improvements from both NetBSD and FreeBSD, including some kernel code, and newer parts of the FreeBSD userland, to replace their older NeXT userland which was based on now-outdated 4.3BSD code. I think this is where the confusion comes in. People assume MacOS is a only "technically" a Unix by way of having borrowed some code from NetBSD and FreeBSD. They don't realize that it's fully and truly a BSD and Unix by way of having been built from NeXT and tracing its lineage directly through the original Berkeley Software Distribution. That code they borrowed was replacing older code, also BSD-derived.
This is an unusual take considering Drew DeVault actually does have experience developing new kernels [1] in experimental languages [2].
Drew's own post [3] (which the linked article references) doesn't downplay the effort involved in developing a kernel. But you're definitely overplaying it. 30M SLOC in the Linux kernel is largely stuff like device drivers, autogenerated headers, etc. While the Linux kernel has a substantial featureset, those features comprise a fraction of that LOC count.
Meanwhile, what Drew's suggesting is a kernel that aims for ABI compatibility. That's significantly less work than a full drop-in replacement, since it doesn't imply every feature is supported.
Not to mention, some effort could probably be put into developing mechanisms to assist in porting Linux device drivers and features over to such a replacement kernel using a C interface boundary that lets them run the original unsafe code as a stopgap.
[1] https://sr.ht/~sircmpwn/helios/
[3] https://drewdevault.com/2024/08/30/2024-08-30-Rust-in-Linux-...
Many EDR and other security products are beginning to use it, although IIRC Crowdstrike does not yet.
And you can reap the benefits of microkernel ideas in isolated domains. Apple is choosing to benefit from microkernel style userspace APIs for third party code in particular, because that's a very likely vector for security and stability issues. You simply can't know what degree of testing and safety practices are being employed by third parties, and their crash analytics almost certainly bear out that most system crashes happen there.
Microkernels are not an automatic antithesis of performance. As usual, you have to make informed tradeoffs. Do you keep components X in ring 0, or not? And if you move it out, how do you design the interface between the two to try and mitigate the performance impact of that choice. In the case of Apple with MacOS, additionally, how do you develop the SoC in tandem to make the context switch cheaper?
(Really, Apple's apparent competitive advantage from vertical integration between hardware and software is a farce. There was never anything stopping Microsoft from approaching Intel and AMD and working jointly on making the same kind of improvements at the hardware level for the benefit of the software. They just didn't care enough to do so.)
That the industry gave up on microkernels is not a testament to them being untenable from a performance perspective. It's a testament to the laziness and lack of willpower to advance the state of the art because insecure and unreliable systems were good enough for their customers.
The way our technology looks today is not some utopian extension of all the best ideas we've come up with so far. Choices like monolithic kernels coming to dominate is not evidence that they're a more fit solution. It's evidence that defeatism is the path of least resistance.
When I see Apple do stuff like move to deprecate kexts, I take it as evidence that they give more of a shit about the quality and robustness of their software than the next guys. Meanwhile, we still have Windows forcing EDR components to operate in ring0 and bring down entire industries when a bug happens because they couldn't be bothered to implement a safe API for EDRs in which a memory safety issue -- which we are all acutely aware happen all the time -- won't crash the kernel.
Windows tacitly enabling the Crowdstrike debacle by underdesigning known vectors of failure is not a blameless situation. Crowdstrike has taken more flack than they deserve here, because we've known about issues like this at the platform level, and Microsoft's competitors have even taken the time to fix what Microsoft couldn't be bothered to.
Ideas like microkernels are what you get when people analyze the technical deficiencies in the state of the art, and then go and try to come up with solutions. Of course there's challenges. But that pure microkernels on their own are at a performance disadvantage compared to monolithic kernels is not because that line of thinking is a dead end, but because nobody bothered to put in the R&D to solve it.
https://drewdevault.com/2020/09/25/A-story-of-two-libcs.html
That said, I've been using the tool for a while now and it's been really rock solid. And once you have it installed and working, you don't really have to touch it again, until some hypothetical time when a new backward-incompatible zpool feature gets added that you want to use, and you need a newer ZFSBootMenu build to support it.
Because it's just an upstream Linux kernel with the OpenZFS kmod, and a small dracut module to import the pool and display a TUI menu, it's mechanically very simple, and relying on core ZFS support in the Linux kernel module and userspace that's already pretty battle tested.
After seeing people in IRC try to diagnose recent GRUB issues with very vanilla setups (like ext4 on LVM), I'm becoming more and more convinced that the general approach used by ZFSBootMenu is the way to go for modern EFI booting. Why maintain a completely separate implementation of all the filesystems, volume managers, disk encryption technologies, when a high quality reference implementation already exists in the kernel? The kernel knows how to boot itself, unlock and mount pretty much any combination of filesystem and volume manager, and then kexec the kernel/initrd inside.
The upsides to ZFSBootMenu, OTOH,
* Supports all ZFS features from the most recent OpenZFS versions, since it uses the OpenZFS kmod
* Select boot environment (and change the default boot environment) right from the boot loader menu
* Select specific kernels within each boot environment (and change the default kernel)
* Edit kernel command line temporarily
* Roll back boot environments to a previous snapshot
* Rewind to a pool checkpoint
* Create, destroy, promote and orphan boot environments
* Diff boot environments to some previous snapshot to see all file changes
* View pool health / status
* Jump into a chroot of a boot environment
* Get a recovery shell with a full suite of tools available including zfs and zpool, in addition to many helper scripts for managing your pool/datasets and getting things back into a working state before either relaunching the boot menu, or just directly booting into the selected dataset/kernel/initrd pair.
* Even supports user mode SecureBoot signing -- you just need to pass the embedded dracut config the right parameters to produce a unified image, and sign it with your key of choice. No need to mess around with shim and separate kernel signing.