NickHoff

An H100 has a TDP of 700 watts (for the SXM5 version). With a die size of 814 mm^2 that's 0.86 W/mm^2. If the cerebras chip has the same power density, that means a cerebras TDP of 37.8 kW.

That's a lot. Let's say you cover the whole die area of the chip with water 1 cm deep. How long would it take to boil the water starting from room temperature (20 degrees C)?

amount of water = (die area of 46225 mm^2) * (1 cm deep) * (density of water) = 462 grams

energy needed = (specific heat of water) * (80 kelvin difference) * (462 grams) = 154 kJ

time = 154 kJ / 39.8 kW = 3.9 seconds

This thing will boil (!) a centimeter of water in 4 seconds. A typical consumer water cooler radiator would reduce the temperature of the coolant water by only 10-15 C relative to ambient, and wouldn't like it (I presume) if you pass in boiling water. To use water cooling you'd need some extreme flow rate and a big rack of radiators, right? I don't really know. I'm not even sure if that would work. How do you cool a chip at this power density?

Comment · 2 years ago

I'm an American living in Germany. When I first arrived, the way Germans write the digit 1 surprised me. They write it with the upper hook thing very long, almost like a capital lambda (Λ), which sometimes makes 1 and A visually ambiguous. This isn't really a problem, just something funny about moving to a new country.

Comment · 2 years ago

> Superb Owl

That's a great typo.

Comment · 3 years ago

ASCII arrow: own work.

LOL

Comment · 3 years ago

I clicked their link to Knative so I could read about it. On the Knative webpage, the cookie banner pops up. I don't want to accept so I click "learn more". That expands the cookie banner to include a button that says "I understand how to opt out, hide this notice" as well as a link to a lengthy explanation of cookies. Well I don't want to click the button so I click the link to the cookie explanation. At the bottom of that page are more links to browser documentation that might eventually explain how to opt out, but I can't click those links because everything on the page is disabled - because the same cookie popup is on this page too. It blocks interaction, including clicking on their links about how to opt out, until you opt in. This stuff is getting worse.

Comment · 3 years ago

There's a recent Tom Scott video about an office inside an elevator. It has a desk and chairs, the whole thing. It's on the corner of the building too. It seems, from his video, that nobody is really sure why it was built and what it was used for.

https://www.youtube.com/watch?v=1yfX84RMQ3M

Comment · 3 years ago

My question here is about underlying fab capacity. This chip is made on TSMC 4N, along with the H100 and 40xx series consumer GPUs. I assume Nvidia has purchased their entire production capacity. I also assume that Nvidia is using that capacity to produce the products with the highest margins, which probably means the H100 and this new GH200. So when they release this new chip, does it mean effectively fewer H100s and 4090s? Or is that not how fabrication capacity works?

I'm asking because whenever I look at ML training in the cloud, I never see any availability - either for this architecture or the A100s. AWS and GCP have quotas set to 0, lambda labs is usually sold out, paperspace has no capacity, etc. What we need isn't faster or bigger GPUs, it's _more_ GPUs.

Comment · 3 years ago

This program [1], for example. It just accepts some input on stdin and returns the same input, but mirrored along the diagonal. So the first row of the input becomes the first column of the output, second row becomes second column, etc. But the program is functionally invariant when given itself as input. In other words, you can flip the source code of the program along its diagonal and the result is a program which has the same functionality - it flips stdin.

Or this one [2], which parodies java in C. It's functional C code that looks like Java, including the

  class LameJavaApp
  {
    /** The infamous Long-Winded Signature From Hell. */
    public static void main(String[] args)
    ...

Or this one [3] that calculates pi by estimating its own surface area.

Or this one [4]. It's a lovers quarrel, written simultaneously in C and English. It's incredible, seriously, read it.

[0]: https://www.ioccc.org/

[1]: https://github.com/ioccc-src/winner/blob/master/1994/schnitz...

[2]: https://github.com/ioccc-src/winner/blob/master/2005/chia/ch...

[3]: https://github.com/ioccc-src/winner/blob/master/1988/westley...

[4]: https://github.com/ioccc-src/winner/blob/master/1990/westley...

Comment · 3 years ago

There's also this one. https://www.youtube.com/watch?v=F3FkAUbetWU

He shoots the Prince Rupert's drop with a bullet. The bullet wins. In high speed, you can see the bullet splat against the glass and break into shards, long before the glass breaks. Really neat.

Comment · 3 years ago

I've read about this. Studying the psychology of group dynamics and conflict in a confined environment was a part of the experiment. Sure enough, the crew of 8 quickly split into two factions. It still seems like a success to me though, because even though some of them were barely on speaking terms despite being friends before, they continued to work together.

> I don't like some of them, but we were a hell of a team. That was the nature of the factionalism... but despite that, we ran the damn thing and we cooperated totally.

Comment · 3 years ago

So, what do people think of Vivaldi? I'm a long-time Firefox user but I've been scanning for a new browser for a while now. Even if it weren't for stuff like this, I'll have to change anyway when Firefox dips below ~3% and websites stop supporting Gecko.