mlugg
Joined 3 years ago
Joined 3 years ago
Right now, yep, corrupting the binary that the compiler reads would crash the compiler. In future, we want to detect the corruption and force a clean build. Note however that the Zig compiler is writing the binary to its internal cache directory (typically `.zig-cache/`), and the build system then copies the final artifact to your output ("prefix") directory (`zig-out/` by default), so it doesn't matter if the user messes with the final binary in `zig-out/bin/my_program`, because that's just a copy. Lastly, this is not implemented yet, but we will definitely make sure that Ctrl+C leave the cache in a clean state. I'm pretty sure our incremental compilation system has a nice property that you can just cancel an update partway through and continue it later without too much effort. See also Zig's IO interface [0] for information about cancelation. The compiler should already support graceful-ish cancelation internally (I won't claim to have verified this, because we never actually do cancel compilation right now, but I'm not aware of any glaring issues!). I don't think it'd be a crazy amount of work to improve that so that it also leaves incremental compilation in a valid state.
The only thing I discussed that we could potentially avoid using your suggested strategy is incremental linking, but AFAICT that would only be easily avoidable if we made a separate shared object for each individual function. I guess we could group them arbitrarily and then re-run codegen for everything in one shared library when the other parts change, but that frankly doesn't sound much less awkward than incremental linking!
But putting all that aside: the only thing this approach would really achieve is offloading the linking work from the static linker to the dynamic linker (aka runtime linker). So if it did make compilation faster, I'd expect all of the saved time to just become runtime execution time---which is kind of worse, because the average number of times you run a compiled program is probably >1!
Of course, there's also the obvious point that dynamic linking only works in cases where you can run dynamic executables. For instance, that approach wouldn't work when doing operating system development, while our approach should work completely fine for that use case.
A bit after that quote I have a note about `inline` functions in Zig, where I mention that they perform semantic inlining, which means dependencies triggered by the function actually get associated with the call site. Well, `comptime` function calls work just the same way---in fact, to the compiler, `comptime` calls are almost exactly identical to `inline` calls. So when we encounter a comptime function call, we start analyzing the ZIR for that function's body, but we don't switch our analysis unit, so comptime stuff doesn't really complicate the dependency graph at all (aside from the fact that it means you can depend on any number of source code hashes, instead of everything depending on exactly one).
With all that being said, there actually is a (completely unrelated) way in Zig to can depend on the body of a runtime function (hence why the quote includes "at least in the simplified view I'm presenting here"). It's to do with "inferred error sets" (IESes for short). If a function's return type is written `!T`, that means it can return an error, but we're asking the compiler to figure out exactly which errors are possible. So if at some point we need to know that set of errors (e.g. because the user has done some reflection to try and access the list of errors), that's where we get a dependency on a runtime function body, because we need to analyze the function body to learn about all the places it might return an error.
(To be clear, squeek502 is a part of the Zig core team [0], so he knows what he's talking about :D)
TL;DR: only debug builds for now, could extend to release builds once we have our own optimisation passes one day, but some optimisations will still be inapplicable.
Aside from that: I'm not familiar with how standard base64 deals with endianness, so I'm not sure if it would match that, but this `@bitCast` would certainly give you a base64 encoding. But it would probably emit pretty terrible code to do that---our lowering of `@bitCast` isn't really optimized for moving around huge amounts of data in one operation! (But maybe LLVM would surprise me.)
What do you mean by "interim"? As I explicitly stated in the comment you quoted, it has never, and likely will never, been planned for the Zig compiler to become incapable of using LLVM. The LLVM backend still sees plenty of active development by the core team [0]---that's perfectly compatible with improving the experience of users (including ourselves) by avoiding unnecessary uses of LLVM [1].
> ...is for Zig to generate LLVM binary files that can be passed to a separate LLVM instance as part of the build process. Is that "a first-class supported backend target for compilation"? I suppose it's a matter of semantics, but that certainly won't be the current LLVM backend that does LLVM API calls.
I think you are incorrectly assuming that we currently make heavy use of the LLVM API. As indicated by #13265 being closed, that is not true. The Zig compiler already generates bitcode by itself, without touching the LLVM API. The only thing we actually use the LLVM API for is feeding that bitcode to LLVM, which can easily be done by invoking a CLI instead. Users quite literally would not be able to tell if, for instance, we changed the compiler to pass the bitcode to Zig's embedded build of Clang over CLI.
[0]: https://ziglang.org/devlog/2026/#2026-04-08
[1]: https://ziglang.org/download/0.15.1/release-notes.html#x86-B...
Hm, are you referring to the CI failure on the x86_64-linux-release CI job on the current tip of master on the main Zig repository? That one's a flaky test, not indicative of a serious problem. (We usually just disable flaky tests, but this particular one I'm leaving on for the moment to try and gather a little more information about it.)
The tarball builds for the 0.16.0 release are still moving along okay: https://codeberg.org/ziglang/www.ziglang.org/actions/runs/20...
OP here---I think you got confused somewhere, because this isn't true! The hashes we take are of source code, they're just stored inside of the first IR. Source code is just pretty convenient to hash. If the change does turn out to be something trivial, we'll only invalidate the first-order dependencies of the source hash, which is never usually a big deal. In a particularly bad case, maybe we re-analyze, re-codegen, and re-link many different instances of a generic function, but even that wouldn't usually take long at all.
Also, it's actually important that we do notice these kinds of "trivial" changes in some parts of the pipeline---see Andrew's comment [0] on the Lobsters discussion for this post.
[0]: https://lobste.rs/s/rmzzdb/inside_zig_s_incremental_compilat...