h_mirin

I wonder if bots get any pleasure from karma farming.

I can be fairly confident that when Google says they don’t see your data, they don’t.

I am also fairly confident that nobody sees the moderation decisions either.

The 2027 incident report will explain how the agents exchanged Morse code through the shared L3 cache.

Previously in my ~/Downloads:

linear_algebra_done_right.pdf, 0 pages read, July 2023

linear_algebra_done_right (1).pdf, 0 pages read, Oct 2023

linear_algebra_done_right (2).pdf, 0 pages read, Sept 2024

Downloading (3) now.

This is really an argument about test-time scaling, even though the post never uses the term.

These days "test-time scaling" mostly means letting the model talk to itself for longer, but the first genuinely surprising results came from plain sampling. Google's AlphaCode generated millions of candidate programs and filtered them down to a handful of submissions, which beat the average human programmer in 2022, before ChatGPT even showed up.

Sampling is what AI is good at. Making examples and doing LeetCode are similar in that verification is clear and cheap. Compared to that, "proof" is still a vague concept, except where Lean works. See the fuss over the ABC conjecture. So humans are still needed.

The interesting question to me is what happens after enough learning from "sampling." Isn't AlphaGo's move 37 an AI's nose? If that happens in mathematics, we may end up with results that are correct, machine checkable, and not explainable in any way we find satisfying.

In Japan, these 22GB 2080 Tis have been showing up on flea-market apps for months now, and I suspect many of them may be coming from China. I’ve been wondering whether buyers realize they’re modified cards, not factory models.

Every time I see this kind of story, two things bother me.

First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining your fine-tune.

Second, the $500 training bill is the cheapest line item in this story. The expensive parts are creating the data and maintaining the model afterwards. How many use cases can actually produce 177k scored episodes? Here they had to generate them synthetically from Amazon Berkeley Objects. To me, that dataset is the strongest evidence in the article of how hard fine-tuning is to apply: if the data existed naturally, nobody would need to manufacture it.