fentonc

I was an architect on the Anton 2 and 3 machines - the systolic arrays that computed pairwise interactions were a significant component of the chips, but there were also an enormous number of fairly normal looking general-purpose (32-bit / 4-way SIMD) processor cores that we just programmed with C++.

I think a quad-CPU X-MP is probably the first computer that could have run (not train!) a reasonably impressive LLM if you could magically transport one back in time. It supported a 4GB (512 MWord) SRAM-based "Solid State Drive" with a supported transfer bandwidth of 2 GB/s, and about 800 MFLOPS CPU performance on something like a big matmul. You could probably run a 7B parameter model with 4-bit quantization on it with careful programming, and get a token every couple seconds.

The Cray PVP line was also doing double precision floating point, and could overlap vector memory operations with math operations. My guess is that you would need a microcontroller operating at several hundred MHz to beat a Cray-1 in practice. The later Cray-1/S and /M variants also supported a 10gbps link to an SSD of several hundred megabytes, which is hard to beat in a microcontroller.

Fun article! I was one of the architects on Anton 2 and Anton 3 at DESRES.

I took a different approach by just making an FPGA-based multi-core Z80 setup. One core is dedicated to running 'supervisor' CP/NET server, and all of the applications run on CP/NET clients and can run normal CP/M software. I built a 16-core version of this, and each CPU gets its own dedicated 'terminal' window, with all of the windowing handled by the display hardware (and ultimately controlled by the supervisor CPU). It's a fun 'what-if' architecture that works way better than one might expect in practice. It would have made an amazing mid-to-late 1980s machine.

That's awesome sounding - I love how creative people got with mechanical designs before everything went digital.

This was one of my rare projects that was as fun to use as it was to build!

I spend all day writing C++ or Python, and like playing around with Turbo Pascal on a circa-1984 Kaypro 2 as a hobby machine - the projects are certainly smaller, but my edit-compile-run loop on my Kaypro is usually faster in practice (even running on a 4 MHz Z80 with 64KB of RAM compared to an 8-core 3+ GHz workstation with 64GB of RAM) than my 'modern' work. It's genuinely crazy to me how usable a 40 year old machine is for development work.

I am now adding "Silicon Carver" to my resume.