astro1234

I agree with this and think this is the only argument I’m aware of for real decisions to use of Fortran, which I suppose means its future is doomed with human turnover + ease of porting in these AI days…

The answer was “my adviser insisted because that’s the language he used”. I had to fight to use Fortran 95 and not 77. So I think like you say value was building on an existing community’s codebase in a language they were familiar with and comfortable with.

Having written an open source Fortran library (for x ray scattering and absorption in a particular regime so this is fairly “niche” shall we say), genuinely curious: what if any advantages or strengths does Fortran have these days over other language choices? Are there circumstances where it’s arguably the “right” choice?

It gives me 404 error

I used Astra to help me build an immersive VR experience where you can watch the universe evolve from ~50M years old until the present day. Uses the TNG100-2 simulation. Works in browser too!

Well I think in the absence of convincing pieces of evidence to the contrary you might be right. You’re making an empirical statement but we have already answered it today:

- we get novel, emergent properties and capabilities of these models that were not trained

- they have very clear generalization to out of domain problems

The point is people conflate the end product: a model that can clearly do very novel, useful and interesting things, with the vehicle for getting there which is a series of optimization steps involving next token prediction loss.

You mention limitations; we all clearly know the practical limitations of these models today, but if you look at scaling laws and empirical performance trends (epoch capability index for example) as well as the trajectory over the last couple of years (very stable), the claim that there is some sort of fundamental limitation is now surprisingly the claim that has the burden of proof.

You can claim it may be e.g. finite context. That is fundamentally bad for certain classes of tasks. This was the hypothesis of a lot of lab leadership of urgently trying to anticipate how to get around this bottleneck (still of course lots of work on this) but the surprising thing is it does not appear to be at this point a blocker.

I think you make a fair point, but also remember: new paradigms don't necessarily require confusing / contradictory observations. You could have the simple idea of "what if gravity is an inertial force?" at any period in time and work out the mathematics of this. It would make theoretical predictions which could then be falsified, but then again, who would take it seriously enough to test it if it was maybe say 1850 and not 1899.

A better example is maybe Maxwell's laws. Maxwell wasn't inventing a theory to try and explain confusing results, he was unifying a chaotic, empirical laws from existing experiments. That may be a cleaner example. That knowledge compression into satisfying theoretical framework is likely what is attractive.

You can potentially ask the same thing about like you say -- general relativity / quantum gravity but also likely plenty of other areas that may be like this today. Again going outside my particular area of expertise: standard model physics is in a large important sense empirical; lots of values and numbers that are simply unmotivated by theory or where we don't have a good way to make a principled theoretical choice. That could be a place where these models are able to help.

But right now: I doubt it. This is what everyone is working furiously on right now. How do you close a "science" verification loop? In principle this should be easy right: you have ideation (exploration, sampling with ~high temperature maybe as an analogue) and you have verification (which of these ideas are good) which amounts to rejection sampling in idea space. You have to have a sampler that is good at picking _good_ ideas for efficiency sake and you need a relatively fast and reliable verification step of "is this idea good and worth continuing to explore". But I may oversimplify

Do you mean after? People do this!! But I think it’s a bit different. It won’t be apples to apples because the data volume I think is just so much different. Maybe there are good experiments for something like this.

I think they mean improve our understanding of physics with new theoretical results or paradigms. Like if it’s 1899, would Astra develop General and Special relativity on its own?

My job has gone from coding, plotting, writing to solving the riddle of what Opus 5 is saying and figuring out what is bullshit, what is valuable, what is a total divergence from what I asked it to do. Then eventually at token like 300k starting to shout at it in all caps with obscenities.

Things were slower and harder before but the baseline of frustration / rage was never this high, even as the models have gotten objectively better in many or most respects.

I don't think it's a stall, two ways I would believe there is a stall:

- Does the epoch capability index progress show signs of plateauing? I consider this a good aggregate measure of diverse benchmarks into a single capability index. If we see things slowing down here thats a pretty direct and convincing piece of evidence for a stall. - Do we see any signs that scaling laws are beginning to fail? That would be by far the most alarming to me, since I would interpret that to mean that the entire premise of this unprecedented capital allocation tsunami is broken.

Neither of these are true (for now). Progress is marching the same as it has for 4+ years now. It's still the same time to get a generation leap (I think like ~16-18 mo? Epoch has it) like GPT4->5. My theory is that people interpret plateauing because the releases are far more frequent now than they were in the past.

I’ve noticed this too but it hasn’t been obvious to me that this style of search is not a learned behavior. Tool calling is very much part of the post training phase, I would expect that these style searches just naturally emerge during training. This is just my prior though.

The Opus-human patois is unbearable

Im curious if you find this to be a parody in a bad way or simply a “the state of the art in math research right now is telling a machine to believe in itself”. I am in the latter camp…