gyrovagueGeist

You can imagine the general idea by imagining the state space of a bunch of billiard balls: their position and velocity in a 6D space.

This is a metaphor and not the state or experiment that superdeterminism seeks to answer. I am also speaking well outside the areas of physics I was trained in. So huge grain of salt.

In classical determinism, the trajectories of all billard balls can be derived precisely from their initial conditions. Due to sensitivity to initial conditions and our ability to know them precisely, we can see chaotic mixing behavior where the trajectory of even two billiard balls that start at nearly the same place become more and more uncorrelated as time goes on as they bounce around and into each other. One expects that many accessible observables and functions loose detectable correlations as this billiard ball universe goes on and the billiard balls go on to roughly fill the full 6D state space up to conservation of momentum.

Now imagine the weirder scenario, that there exists some function and statistical comparison you can make between state variables from sets of billiard balls. Although these state variables themselves seem uncorrelated after enough time has passed, there are these properties that always hold with perfect correlation even after infinite chaotic time bouncing around. The points in state space that the balls cover is restricted to some manifold where these statistical properties are true when checked. The system cannot evolve in a way where those correlations of bulk quantities would disappear.

In this toy metaphor it’s not so horrifically odd. Superdetermism gets more flak because: (i) the state space and property in question is so much weirder and more complicated than the kinematic example, the correlations are between objects that measure and the objects being measured! (ii) if these properties are always true under any check we can make there is no way to falsify. Then it doesn’t matter if the behavior is from some unimaginable underlying universal state space geometry or local quantum effects. This same argument could also be applied more generally and inappropriately to reduce any experiment to “that’s just the way the structure of the universe is”.

Electricity (on continental US) is pretty cheap assuming you already have the hardware:

Running at a full load of 1000W for every second of the year, for a model that produces 100 tps at 16 cents per kWh, is $1200 USD.

The same amount of tokens would cost at least $3,150 USD on current Claude Haiku 3.5 pricing.

Not wrong, but I think it's more accurate to say:

Mamba is an architecture for the middle layers of the network (the trunk) which assumes decoding takes place through an autoregressive sequence (popping out tokens in order). This is the SSM they talk about.

Diffusion is an alternative to the autoregressive approach where decoding takes place through iterative refinement on a batch of tokens (instead of one at a time processing and locking each one in only looking forward). This can require different architectures for the trunk, the output heads, and modifications to the objective to make the whole thing trainable. Could mamba like ideas be useful in diffusion networks...maybe but it's a different problem setup.

This is just standard Fourier theory of being able to apply dense global convolutions with pointwise operations in frequency space? There’s no mystery here. It’s no different than a more general learnable parameterization of “Efficient Channel Attention (ECA)”

They also didn't have nuclear weapons to use in global conflicts over resources.

I've always found curriculum learning incredibly hard to tune and calibrate reliably (even more so than many other RL approaches!).

Reward scales and horizon lengths may vary across tasks with different difficulty, effectively exploring policy space (keeping multimodal strategy distributions for exploration before overfitting on small problems), and catastrophic forgetting when mixing curriculum levels or when introducing them too late.

Does any reader/or the author have good heuristics for these? Or is it still so problem dependent that hyper parameter search for finding something that works in spite of these challenges is still the go to?

Thief of Time by Terry Prachett has a great minor bit about characters who are naming themselves after colors running out of human made labels, as they have to get increasingly esoteric with the names. It's fun to see that visualized.

It's from the depth of the computation, not the work

In the middle term, I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised.

Because they generate so much code, that often passes initial tests, looks reasonable, and fails in nonhuman ways, in a pretty opinionated style tbh.

I have less context (and need to spend much more effort and supervision time to get up to speed to learn) to fix, refactor, and integrate the solutions, than if I was only trusting short few line windows at a time.

Useful and interesting but likely still dangerous in production without connecting to formal verification tools.

I know o3 is far from state of the art these days but it's great at finding relevant literature and suggesting inequalities to consider but in actual proofs it can produce convincing looking statements that are false if you follow the details, or even just the algebra, carefully. Subtle errors like these might become harder to detect as the models get better.

For people who are interested Kokkos (a C++ library for writing portable kernels) also has a naming scheme for hierarchical parallelism. They use ThreadTeam, Thread (for individual threads within a group), and ThreadVector (for per thread SIMD).

Just commenting to share, personally I have no naming preference but the hierarchal abstractions in general are incredibly useful.

I can't reproduce this, possibly it's another environment error or the problem has been fixed in the versions I am using. (uv 0.7.11, zsh 5.9, python 3.13.5, MacOS 15.3.1)

It does? What problems have you had?

Yep! Optimize (solve in infinite dimensions) and then discretize onto a finite basis has typically led to much better and stable methods than a discretize and then optimize approach.

Time-scale calculus is a pretty niche theoretical field that looks at blending the analysis of difference and differential equations, but I'm not aware of any algorithmic advances based on it.

Does anyone know why they added minibatch advantage normalization (or when it can be useful)?

The paper they cite "What matters in on-policy RL" claims it does not lead to much difference on their suite of test problems, and (mean-of-minibatch)-normalization doesn't seem theoretically motivated for convergence to the optimal policy?