pron

> That doesn't make sense. If the cost of RAM is high and being used by models then yes, you should try to use less RAM.

No, that's not the conclusion. If using models costs you, say, $10 an hour, and a GB of RAM costs you, say, $0.0001 an hour, then clearly you shouldn't spend your time on reducing the insignificant cost if you want your work to deliver the most value to users.

But again, "less RAM" is more or less meaningful whether you're reducing the RAM usage from, say, 5% to 1% or from 80% to 40%. The latter is more valuable even though it's a lower relative reduction. That's because what the users care about is in what conditions they reach 100% of either CPU or RAM, and the second 2x reduction, therefore, offers more value than the first 5x one. This is similar to how a 10% discount on a car offers far more value than a 99% discount on a pack of chewing gum. This is still the case if you're buying 100 packs, but maybe not 10,000, and the same goes for software: it may also matter how many copies of the app the user is expected to run concurrently. The point is that you can't assign a value to a 10x reduction if that's all you know.

> I'm entertaining this nonsense because I want to clarify, that I use a 20 year old web gui and it consistently eats 8 gigs of ram in Webkit because it uses CSS to update the forms.

And I'm entertaining this nonsense because you're clearly not a very experienced engineer and you may learn something. In the ninetees, when nearly all GUI apps were written in C or C++, we had plenty of examples where a C++ GUI app consumed 50% RAM. That doesn't mean that C++ is worthless, only that some software has serious resource bugs, and it sounds like that web page you're referring to is a case of that. Most web-based GUI apps today do not consume a higher portion of RAM than their C++ counterparts did 30 years ago. Today's browser-based MS Word consumes less (relative) RAM than the C++ version did in 1998, and it certainly has a far smaller negative impact on concurrently using other apps on the machine.

And, of course, there are buggy/leaky programs now just as there were then. It's silly to say that the existence of such programs means that the approach is bad, because that would apply to any sufficiently-popular language or toolkit.

Anyway, our job is to offer the most value to our users, and reducing RAM consumption by 3% or even 5% (of available RAM) is usually not where you'd find much value.

> I don't know why you needed a paragraph to say that 1000% memory inflation is a trade-off to optimize dev costs

Because that's not what I said. My point was more that a "1000% memory inflation" is not a meaningful cost metric to begin with (to use just one toy example, perhaps you'd agree that going from 10% RAM to 100% has a very different impact than going from 1% to 10%, even though both are a "1000% inflation").

Memory (at least on a personal computer) is not rented by the hour on a per-MB basis; you buy it in bulk. What matters isn't how much memory you could have used but what you'd use the "saved" memory for. Also, when you buy a resource like RAM or CPU as a package, nominal values matter less than under what conditions one of these would reach 100% and your experience may be degraded.

For example, if you have 16GB of RAM and you only ever run one program at a time, it doesn't matter to you at all if it uses 3GB or 3MB: you've paid for 16 and the total savings of the 3MB app are zero and in this hypothetical case, there's no tradeoff at all. Now, the average user on the average hardware doesn't use only one GUI app at a time, but they're also not using 200. The only relevant question pertaining to waste is what is the value of 3% memory to your users (integrated over all of them)? In many situations (again, not all), the answer is not much. (And BTW, some developers seem to care about 3% memory more than they do about 3% CPU, even though CPU is much more expensive).

When it comes to agent harnesses, the analysis is clearer. On the one hand, people using coding agents are more likely than others to put even a little extra free RAM to some good use, but on the other hand, the cost of using the models themselves is so high that it makes the cost of RAM for the GUI negligible.

Saying that a number going up by 1000% is bad is not good engineering. Good engineering requires thinking about the actual value and cost of that number. It varies over time, over app type, over different types of users, and over different kinds of deployments (i.e. how RAM is packaged and paid for).

I'm a 25+ year C++ programmer, and what you're saying is nonsense. First, managing costs is an important part of engineering. More importantly, though, the use of browser engines for GUI apps isn't about JS but about the browser engine. These engines are among the best optimised and most sophisticated rich text layout engines we have. Even their high RAM usage is overstated. 1GB of RAM is 6% of the physical RAM on average hardware, and because these GUI apps are usually low-CPU, their RAM is mostly inert and can be swapped out without much impact, so really 1GB of RAM for such an app amounts to no more than 3% of the average personal machine's memory resources. Even if you can reduce that to zero, gaining 3% of memory in exchange for higher costs and compromising on presentation quality is often (not always!) a reasonable tradeoff from a pure engineering perspective.

Never mind the policy, why do tech CEOs have to sound like frat boys?

Yes (or something along those lines), it's just that this isn't the first use case addressed by this capability, which is being delivered gradually (as we normally do in the JDK), and not even all the basic building blocks are out yet.

For the past several years, Java features are released piecemeal, with each piece released as soon as it can be useful to some significant portion of users or users who can offer useful feedback (and when it's not expected to restrict future pieces). The AOT cache is already used in production despite not being as user-friendly as it could be, because the companies using it find it valuable.

The first use case is the very common one (for Java) where developers control deployment (internal deployment). The last basic building block, caching machine code, will be released next year in JDK 28, and then the feature is likely to evolve to better support other use cases, such as external deployment.

Generational moving allocators don't move objects much, and so the total amount of work they need to do is substantially less than malloc/free allocators -- that's the whole idea of these algorithms and the reason why they're used (that's not under dispute; it's just the mathematics of the algorithms [1]). Still, there might be some situations where allocators in C++ could be faster, but if that were the case for my purposes, I would have stuck with C++. We needed very good performance for highly concurrent workloads, and C++ just doesn't come close in many of those (maybe at 10x the budget).

[1]: More precisely, the work they need to do doesn't scale linearly with the allocation rate as is the case with non-moving approaches. Of course, arenas are sort of degenerate moving collectors, so they also display the same effect, but they can't always be used.

The way this works is that you allocate in the young generation with a pointer bump, and to free memory you compact (move) the surviving objects, which make up a small minority, a few times and then promote (move) them to the old generation (where you either manage memory with the same technique or a different one). And so the total amount of moving is not high (objects either never move or rarely move), and what you end up with is essentially two heaps, one where the allocation rate may be high but you can completely negate all the extra work by increasing the size of that heap, and one where the allocation rate is low. This ability to, optionally, not double the amount of work when the allocation rate doubles is what gives moving collectors their efficiency (and that you can bump-allocate in virtually all cases).

Of course, it's possible to come up with programs where objects do move a lot (or there's some other common work), e.g. programs that don't exhibit the behaviour of the generational hypothesis, but the point is that in a very large class of programs, this technique is hard to compete with.

Of the general-purpose methods it does; that's is exactly what these collectors were invented for - not to carry all that overhead of malloc/free. I guess you could argue over the frequency of situations where it yields a significant boost. It's also true that malloc/free have also improved, especially in the very large runtimes (like TCMalloc and jemalloc), but they can't do all the optimisations moving collectors can.

1. When I pick a low-level language, that's precisely the kind of control I'm after (otherwise, why go for a low-level language?).

2. Heap allocation is a very common cause of serious performance problems in low-level programs because malloc/free overhead is so high. Some languages have a moving GC to reduce the high overhead of heap allocation, but low-level languages typically can't use that optimisation, so you have to resort to arenas (which offer a similar optimisation to that of moving collectors). This isn't always faster than malloc/free, but it usually is, and if you want malloc/free you can pick that.

Anyway, I wouldn't say that the default in Zig is to use an arena, rather it's to give you the control that's the whole purpose of low-level languages.

Ah, I see how that works. Thank you. So I guess the main tradeoff is giving up concurrent data structures.

Interesting, but can you write something like an LRU cache?

> When Java ends up adopting those features, suddenly it's "we accommodate a non-minority number of developers" and "we've been thinking about this since before Java was released".

It's not those features, though! Obviously, a hoarder has everything, so anything you buy is something he has. We are adopting a small portion of the features that feature-rich languages have, which is exactly the difference between a more feature-rich language and a more minimal one.

> ...aaand there it is again. "The lesser languages".

Again? I haven't said "the lesser language" once, although you've now said it a few times as if quoting me. You're arguing with things I haven't said and then misquote me to completely change the meaning of what I actually wrote.

No one, on the Java team or outside it, considers Lisp, Smalltalk, and ML - the languages that inspired Java - "lesser" languages. They are, however, all less successful languages as a matter of fact. It is because they are not lesser but they are less successful that we selectively adopt a portion of their ideas but do not adopt their strategy.

It's not about snobbery, but basic business strategy. Suggesting that it's good business for a product to adopt the strategy of some competitor product doesn't make much sense when that other product is doing worse than your own. I mean, maybe it could be interesting if there were some other key element to the plan, but not on its own.

Almost every single feature in Java was inspired by (or sometimes copied from) a feature in a less successful language (although not always less successful at the time the feature was adopted), but Gosling was always very clear about copying good ideas into a product with a very different strategy (he thought it was the strategy that hindered those other products, not their ideas).

> When Java ends up adopting those features, suddenly it's "we accommodate a non-minority number of developers" and "we've been thinking about this since before Java was released".

No, we've also been thinking about features that may well probably never make it. I'm not even sure what you're trying to say, but you should know that anyone who works in language design has been familiar with type inference, generics, typed lambdas, pattern matching, and algebraic data types at least since the late eighties because ML has been the biggest single influence on language designers and other PL fans in the past couple of decades (ML was released in the early seventies, but it didn't become so influential right away).

> I'm glad it seems you're in the minority seeing how Java has picked up its development pace.

Java has picked up its development pace compared to the time we were involuntarily stagnant due to declining investment from our corporate overlords. As always, most of the development is in the runtime, as that has always been our strategy, and we all think it's a good one. I'm very happy that we can do more now than we could fifteen years ago, but the Java language won't become a (relatively) feature-rich language, even though it will continue to add features.

ZGC is effectively pause-free. It uses a STW pause for thread synchronisation of the epoch as an optimisation (we tried synchronising without the pause, but OS scheduling just slowed us down; maybe we'll use new OS techniques for synchronisation in the future). ZGC does zero GC work in these pauses. There is also no strong reason to remove the synchronisation pause because it is no longer than OS jitter (i.e. C programs can experience pauses of similar duration on non-realtime kernels).

BTW, SGCL is a non-moving mark-and-sweep collector and non-moving collectors are no more similar to moving collectors than malloc/free is similar to moving collectors. Garbage collection contains different classes of algorithms with different goals that are so different from one another that it's almost pointless to put them in the same category. Java uses moving collectors because it's the class of algorithms designed to reduce the throughput overhead of all non-moving approaches (from manual malloc/free, through automatic RAII and refcounting GC, through tracing mark-and-sweep collectors like SGCL or Go's GC). In important ways, all non-moving approaches are more similar to each other (regardless of the use of a GC) than any of them are to moving approaches. (Java did have a concurrent mark-and-sweep collector at one time, but we removed it some years ago)