csunoser

This is the darnest thing - all the testimonies are just jepgs? Did they run out of time to work on the css?

np :-)

for > Calculate the number of bits to encode that value using log2(938933556), which is ~29.8

This is roughly same as saying: "If you rewrite 938933556 as a binary number / usize, it will need 30 bits".

Sanity check: 1101111111|0110111111|0100110100 (| delimits every 10 bigits).

> Since the file is 128 bits long, one would expect this place to be around the 2*128th bit.

This statement is a bit more subtle. As a first ord approximation, we can see pi sort of as a RNG.

If we write pi (ignore the decimal point), as a binary number, we get: 11011001111111011110010101011110001010101111101101110001001100001...

You can... kind of squint and pretend this is a random sequence of 1s and 0s.

Now, if you had a file that is 128 bits (so lots of intermingling 0s and 1s), and each next digit of pi is effectively a coin flip. Pretend 1s are heads, and 0s are tails. You basically have to get the exact 128 consecutive coin flips of the same result as your file to get your file back.

Imagine now, PI not as a number, but a sequence of experiments of flipping the coin 128 times.

  - (11011..01000)(10000...00100)....
  - ^attempt 1     ^attempt 2

You have to try, on expectation, quite a few times to win this game! Now, you could easily get lucky for sure. But on average, your chance of winning per attempt is roughly 0.5^128! So, how many times do you have to try to win this game? Something like 2^128 times - and you have to consider that each attempt uses 128 bits as well. So more like 2^135. But you don't have to start fresh in each attempt, you can see it as like this:

  - 11011................00100...
  - (       128 flips     )
  -  (  another 128        )
  -   (                     )
  -     ... so on and so on

That's where the 2^128 number came from.

Oh hey, this is the jj workshop person!

My intention is to highlight the fact that human conversations are cleverly disguised examples of natural selections

Not sure why you are downvoted, this is essentially correct (assuming Volcano Engine tech refers to Deepseek as provider).

Yes*. At least from my limited usage of deepseek-flash for a few billion tokens on openrouter, the cache-hit rate is >95%. And I simply used the claude code harness pointed at the openrouter anthropic compatible endpoint with no fluff.

I feel dirty using emdash as the discriminator between human effort and non-effort. But it sure has quite a few.

Mayhaps! But I think as far as google, anthropic[1] and apple[2] goes, they do use the tpus for training. Ofc v4 and v5 (older generations of tpus) were more specialized for search related embedding workloads and i could see people not using them for training.

[1]: We train and run Claude on a range of AI hardware—AWS Trainium, Google TPUs - April 6th, Anthropic on Google and Broadcom partnership [2]: "[Apple foundation model]... builds on top of JAX and XLA, and allows us to train the models with high efficiency and scalability on various training hardware and cloud platforms, including TPUs and both cloud and on-premise GPUs" - Apple in 2024

I am not sure what context Jensen said that. But midjourney uses tpu. Apple uses tpu. They are no other frontier labs that use it, but Google + Anthropic is 2 out of 3 frontier lab so.....

You could reasonably say that "A majority of frontier labs uses TPU to train and serve their model."

I am more sympathetic to the OG.

There are many good criticisms against data center. And yet, the water issue always comes up first. Must we spew false/untruthhood just so our political message is catchy? I suppose yes - in times of war/politics, the laws/truths are silent. But it doesn't have to be so here.

Huh. I initially thought this is just another finetuning end point. But apparently they are partnering up with customers on the pretraining side as well. But RL as well? Jeez RL env are really hard to get right. Best wishes I guess.

I have used both (albeit 2 years ago, and things change really fast). At the time, Candle didn't have 2d conv backprop with strides properly implemented. And getting Burn running libtch backend was just a lot simpler.

I did use candle for wasm based inference for teaching purposes - that was reasonably painless and pretty nice.