jrk

For averaging multiple different benchmarks, the geometric mean makes sense. For removing measurement noise from a single benchmark, min or median is usually the right statistic.

Commercial air travel is extremely competitive and has very low margins.

The Fed transitioned from QE to tightening almost 5 years ago.

It's not that you need to turn on some extra library backends and rebuild, it's that the abstractions themselves are fundamentally at odds with hitting peak performance on many things so you have to rewrite your code.

Individual image processing operations are often very low arithmetic intensity. If you don't combine them into much larger subroutines—which are necessarily less generic and orthogonal—you spend all your time waiting on memory between every little op.

To make "next to nothing" concrete: MIT EECS PhD students are currently paid about $4700/mo. This is substantially less than they'd make in industry, but it's around the US median personal income across all working-age adults, and well above the average 24 year-old. They frequently make a substantial extra at summer internships, putting them well above US median in the years when they do.

Also: it is school, not just a job. They are developing deep expertise and specialized skills. As a result, among other things, their earning potential tends to be significantly higher coming out of the PhD than out of undergrad.

The 1TW is the rated peak power output. It's essentially the same in space. The thing that changes is the average fraction of this sustained over time (due to day/night/seasons/atmosphere, or the lack of all of the above).

It's still the same 1TW theoretical peak in space, it's just that you can actually use close to that full capacity all the time, whereas on earth you'd need to over-provision substantially and add storage, so 1TW of panels can only drive perhaps a few hundred GW of average load.

The PyTorch version is using the GPU (with Metal Performance Shaders); this C version is currently using (in the docs I saw) a single CPU core, with AMX (via Apple Accelerate BLAS) but not yet with OpenMP for parallelism. It’s not slow because LLM code is bad, but because it’s not running on the same hardware. That said, it’s also not as fast as it is because of the LLM—all the critical code is in kernel libraries it calls (the same as for PyTorch).

Why are GPUs necessarily higher latency than TPUs? Both require roughly the same arithmetic intensity and use the same memory technology at roughly the same bandwidth.

Not sure when you last tried, but Gemini, Claude, and ChatGPT have all supported pretty effective PDF input for quite a while.

It is not only not that much more complex, it is often less complex.

Higher-level services like PaaS (Heroku and above) genuinely do abstract a number of details. But EC2 is just renting pseudo-bare computers—they save no complexity, and they add more by being diskless and requiring networked storage (EBS). The main thing they give you is the ability to spin up arbitrarily many more identical instances at a moment’s notice (usually, at least theoretically, though the amount of the time that you actually hit unavailability or shadow quotas is surprisingly high).

You know that AWS will come back up. You definitely don’t know whether your own instances will come back or if you’ll need to redeploy it all.

Yes but you can also do the same thing with autoregressive models just by making them smaller. This tradeoff always exists, the question is whether the Pareto curve for diffusion models ever crosses or dominates the best autoregressive option at the same throughput (or quality).

I think they weren’t asking “why can’t Gemini 3, the model, just do good transcription,” they were asking “why can’t Gemini, the API/app, recognize the task as something best solved not by a single generic model call, but by breaking it down into an initial subtask for a specialized ASR model followed by LLM cleanup, automatically, rather than me having to manually break down the task to achieve that result.”

I think the point was not that gem-grade synthetic diamonds are ugly, but that, as industry masters gem-grade production, presumably below-gem-grade production (“ugly synthetic diamonds”) would become cheap enough to deploy in more engineering settings where diamond’s other unique properties were the key concern.