joseph_grobbles

This is a really terribly written article. Sorry for the negativity, but it was tough to try to dig through.

For those confused by the title, Intel released a Xeon that includes 64GB of high speed RAM on the chip itself, configurable as either primary or pooled memory, or a memory subsystem caching layer.

It's a variation of Parkinson's Law -- we just keep expanding what we are doing to fit the hardware available to us, then claiming that nothing has changed.

CI is a fairly new thing. The idea of constantly doing all that compute work again and again was unfathomable not that long ago for most teams. We layer on and load in loads of ancillary processes, checks, lints, and so on, because we have the headroom. And then we reminisce about the days when we did a bi-monthly build on a "build box", forgetting how minimalist it actually was.

Google's first TPU was developed a year after Tensorflow. And for that matter, Tensorflow works fine with CUDA, was originally entirely built for CUDA, and it's super weird the way it's being referenced in here.

Tensorflow lost out to Pytorch because the former is grossly complex for the same tasks, with a mountain of dependencies, as is the norm for Google projects. Using it was such a ridiculous pain compared to Pytorch.

And anyone can use a mythical TPU right now on the Google Cloud. It isn't magical, and is kind of junky compared to an H100, for instance. I mean...Google's recent AI supercomputer offerings are built around nvidia hardware.

CUDA keeps winning because everyone else has done a horrendous job competing. AMD, for instance, had the rather horrible ROCm, and then they decided that they would gate their APIs to only their "business" offerings while nvidia was happy letting it work on almost anything.