boris

"The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt."

Bertrand Russell

It is lz4. While all that you say about disk being slow makes sense, measurements show that in this specific situation disabling compression improves performance by a little. My hand-wavy explanation is that freeing CPU to doing compilation rather than [de]compression is a better use of it, with all the uncompressed disk writes probably ending up in the system's file cache.

> Couldn’t you amortize this to 0 by just caching the result and only changing it if the binary timestamp changed?

Yes, that would be nice, but the tricky question is can any of this information change without the compiler binary mtime changing? First off, GCC's gcc/g++ binaries are drivers and are not what does the actual compilation, it's private cc1/cc1plus binaries that do the job. Can one of these change but not the driver? I think it's plausible (some package manager optimization where the file is not touched if it hasn't changed). So at a minimum we would need to discover where those are located (probably by invoking gcc/g++) and checking them as well. Could there be something else? Who knows. We value speed very much but we value correctness even more.

I think a more fruitful direction to explore is to improve GCC itself to dump all this information in a single invocation and in a machine-readable format (JSON). I think if we go from 70ms to 14ms (and perhaps even lower because this special GCC mode could conceivably do things faster than how we do it now), it would be good enough.

[build2 author here] Thanks for the feedback! Some additional details:

> The first one is a criticism of CMake, not Ninja (?), so I don't think it can be why.

Fair enough. The point I was making is that if you want to compete with Ninja, you cannot leave any potential performance gains on the table.

> The second reason given is doing some work like header dependencies in multiple threads. This is the most plausible reason to me but it still feels unlikely.

We are talking about ~2% performance difference here. Parallelizing even a small amount of work across 24 threads rather that doing it serially saving a percent or two feels plausible to me.

> There is some hand waving about file access patterns but I am skeptical; if the end-to-end build time is 3 seconds then the project is small enough to all fit in kernel caches.

It fits into the system's file cache unless there is memory pressure, like one would expect from having 24 C++ compiler jobs running in parallel. We actually measured this in isolation (with more detailed results in the linked article) and it has a measurable effect.

> They also mention doing other things like invoking the compiler to get version information. This seems like it would dwarf any performance gain from number 2.

I measured this, it costs 70ms or ~2% of the overall time.

At least you know it was written by a fallible human. But if you point out my mistake, I will be happy to fix it.

I don't know, someone who solves their borrow checker errors by adding more dynamic allocations (which seems to be common/recommended), will be taught how to program in Java, not C++.

> Commands in a CLI … I think what makes for good product design is having a very small number of primitives.

Small but not too small. Case in point: shell scripts (POSIX shell, bash) where the scripting part was decided to be modelled as commands thus not introducing another bunch of concepts. We all know what the result is (hot, slow mess).

GCC (libstdc++) as all other major C++ runtimes (libc++, MSVC) implements the small object optimization for std::function where a small enough callable is stored directly in std::function's state instead of on the heap. Across these implementations, you can reply on being able to capture two pointers without a dynamic allocation.

The strangest thing about Flow is that its compiler is implemented in C#. So if you decide to use it in your C++ codebase, you now have a C#/.Net dependency, at least at build time.

I believe the point is if something is UB, like NULL pointer dereference, then the compiler can assume it can't happen and eliminate some other code paths based on that. And that, in turn, could be exploitable.

What is the filesystem story in OpenBSD? Anything CoW/snapshot'able on the horizon?

What would be the SQLite's equivalent to indexing starting from 1, not 0? Off the top of my head I can't think of anything that would go so much against the grain.

> We have been optimized very hard by evolution to be good at running, so there shouldn’t be any “easy” technologies that would make us dramatically faster or more efficient.

I wonder if these new shoes have the same affect on natural (i.e., non-paved) surfaces? Plus, they all look quite high off the ground (probably all those plates and foam need space) and that doesn't help with stability when running over rocks, etc.

There is a parallel with database transactions: it's great if you can do everything in a single database/transaction (atomic monorepo commit). But that only scales so far (on both dimensions: single database and single transaction). You can try distributed transactions (multiple coordinated commits) but that also has limits. The next step is eventual consistency, which would be equivalent to releasing a new version of the component while preserving the old one and with dependents eventually migrating to it at their own pace.