cshimmin

Interesting, I wonder if the rugby thing is a common bias. I did find myself in the weights, as the top result. But apparently there are also Australian rugby versions of me!

That is simply not true. The naive “glorified auto-complete / stochastic parrot” argument may have some merit when applied to generic pre-trained models, which only learn from unsupervised next-token prediction. But the post training through reinforcement learning the frontier models undergo is very sophisticated and they genuinely learn to do novel things that are purely the work of the model being trained (and the work of the GPUs they burn along the way of course).

It's on the page, if you click the little info icon in the upper-right. Here's the text but there's some nice graphics there too:

  Snake Game, training entirely in the browser. Built on tinygrad: the rollout / targets / train graphs are TinyJits authored in Python, then compiled once to WGSL and replayed here under WebGPU.

  Observation: flat 10×10 board (100) + 4-dim prev-action one-hot = 104 dims. fc_pi.weight is zero-init so the opening policy is uniform over the legal actions; fc_v uses tinygrad's default Kaiming init.

  Per rollout: T=24 × N=384 parallel snakes (9,216 transitions), then K=3 epochs × 4 mini-batches of PPO updates. GAE γ=0.99, λ=0.95; AdamW wd=0.01; ratio clip ε=0.1; grad-norm 0.5; Huber value β=1, val_coef=1; entropy bonus 0.008333333333333333.

  Action mask + value clip + KL early stop. The 4-dim prev_a obs tail lets fc_pi zero the U-turn logit (the env silently overrides same-axis reversals anyway). Value loss is max(huber(v_new−td), huber(v_clip−td)) at ε=0.2. Approx-KL is sampled after each epoch and breaks the loop at 1.5·kl_target.

Very helpful! Naïve question (I haven’t had a chance to read TFA at all and diffusion/flow models are not my area of expertise). Doesn’t learning the integral/solution of the diffusion process in a single pass just take us back to like OG generative CNN that we had before diffusion models took over? Surely the answer is “no” but would love to hear your framing as to why.

Not gonna lie, the first thing I noticed was that the first author was in an anesthesiology department. Your guidelines for sniff-testing are not unreasonable, and can definitely be helpful to people who are unfamiliar with the research area. But I quite intentionally did not appeal to any of those. As a (somewhat) subject matter expert, it's important to _ignore_ things like ad hominem judgement, and instead address the paper on its self-contained merits. And more importantly, to share my assessment of those with the lay public.

When I was a research physicist I spent a lot of time looking at the effects of ionizing radiation in pictures, although mostly in the context of digital images. The mechanisms are a bit different for photo emulsions, but to me the reason I'd discount radiation is because they're specifically filtering for features that exhibit the expected point spread function (which is a geometric property of the telescope's optical assembly itself). I guess you could test by exposing emulsion plates to ionizing radiation and seeing how often you get PSF-like images by chance. Also, their search is for +/- 1 day of nuclear testing, which seems weird. Certainly radiation from fallout wouldn't make sense on the day before testing. It would have been useful to see +1 day and -1 day separately. Or 0-2 days. The way it's chosen makes me suspect they couldn't find a signal in those windows, and therefore it's probably just statistical noise that they've massaged out of the data.

But to me the biggest flag is that these images are from 50 minute exposures. The objects don't appear as streaks, so they are either very, very short flashes (much shorter than 50 min), or they are very far away. The authors interpret this to mean the objects should be in geosynchronous orbit, which doesn't make sense; objects in geosync would still appear to move relative to the star background over the course of 50 min. Yet this is the entire basis for their "shadow deficit" window calculation. You could constrain the duration vs distance by looking at the effect it would have on smearing the PSF, which would be interesting.

Overall it seems pretty unscientific. If you go looking through enough statistically noisy data for signals in enough places, you'll eventually find it.

It kinda sounds like a post-doc, in that it provides an on-ramp to working in the industry/institution. But without having to waste your time getting a PhD.

The 6 is part of 3.6, the model version. 35B parameters, A3B means it's a mixture of experts model with only 3B parameters active in any forward pass.

Incidentally, I recently learned the origin of the term. Cyber - short for cybernetic - is from the greek κυβερνήτης (kybernetes), meaning helmsman. The original use of cybernetics is in the context of automated control systems, so steering a rudder was a good analogy. It is also the origin for the name k8s.

Yeah perhaps a better term for Loser is Abstainer. Because the Sociopaths also can certainly lose at the game of maximum capitalist profit. Loser/Abstainer just chooses not to play the game.

It's basically just a way for the LLM to lazy-load curated information, tools, and scripts into context. The benefit of making it a "standard" is that future generations of LLMs will be trained on this pattern specifically, and will get quite good at it.

Was this after you and two zany friends made a scheme to divert a fraction of a penny from each of your employer's transactions into a bank account that you control? And then you gave it all back but the building burned down and Milton made off with the cash?

So… your argument is that it’s not counterculture unless it’s mainstream culture? And that one should only credit derivative works once they become mainstream, rather than the original inspiring works because they were too obscure?

I don’t think anyone is trying to “gotcha” you. You’ve just got a bad take.

If I understood correctly, the global lock is so that notify events are emitted in order. Would it make sense to have a variant that doesn't make this ordering guarantee if you don't care about it, so that you can "notify" within transactions without locking the whole thing?