trsohmers

I’m so tired of these silly love song references.

It actually stands for "lizard brain"... it is (or at least was) an Infineon Aurix control and monitoring microcontroller, they may have changed to a newer one.

Only with the oscillation overthruster flag enabled.

I was put on it in 2015 after an acquaintance of mine that was previously on the list recommended me… I only heard from Forbes a few days before the list came out, they asked me for a photo and asked if I approved the 2 sentence blurb they prepared, and that was it. For years afterwards they would try to get me to come to their events, but I never had any interest, and I assume that was how they made money… but I never paid anything to be on the list or had real interest in being on it, and I don’t think it led to anything other than my technically illiterate parents thinking that it was impressive.

Based on their S1 filing and public statements, the average cost per WSE system for their (~90% of their total revenue) largest customer is ~$1.36M, and I’ve heard “retail” pricing of $2.5M per system. They are also 15U and due to power and additional support equipment take up an entire rack.

The other thing people don’t seem to be getting in this thread that just to hold the weights for 405B at FP16 requires 19 of their systems since it is SRAM only… rounding up to 20 to account for program code + KV cache for the user context would mean 20 systems/racks, so well over $20M. The full rack (including support equipment) also consumes 23kW, so we are talking nearly half a megawatt and ~$30M for them to be getting this performance on Llama 405B

Do you think that the 16k GPUs get used once and then are thrown away? Llama 405B was trained over 56 days on the 16k GPUs; if I round that up to 60 days and assume the current mainstream hourly rate of $2/H100/hour from the Neoclouds (which are obviously making margin), that comes out to a total cost of ~$47M. Obviously Meta is training a lot of models using their GPU equipment, and would expect it to be in service for at least 3 years, and their cost is obviously less than what the public pricing on clouds is.

+1 this commenter. I just visited the UK for the first time at the beginning of this month and had a fantastic ~3 hours at Bletchley Park, but felt I had to cram TNMOC and the amazing Colossus live demonstration (where I asked a million questions) and everything else in the museum in the 90 minutes I was there. If I assume other HN readers are like me, I would dedicate at least 2.5-3 hours for TNMOC to actually get a chance to actually see and play around with their extensive collection of vintage machines.

They meant that there is no support for Codestral Mamba for llama.cpp yet.

Founder of REX Computing here; I highly recommend checking out my interview on the Microarch Club podcast linked elsewhere on the thread; will also answer questions on this thread if anyone has them.