itkovian_

Also no doubt they’ve coined a term with weight space programming and this will expand into an active sub field

This was shocking to me; but on reading it makes far more sense.

- the primary experiments are on small policy networks - they are compressed dynamic system models. The observation and action space can be read off. It’s not surprising to me that these types of networks can be used to identify the env they were trained on

- the exps on actual llms remove looping behaviour and that’s it - from article “but it could not add an ability the model lacked”

I’d say this is fortunate because if llms could do the latter, rsi would be immediate. It may be possible we simply need bigger and better edit models, but at least from the experiments there doesn’t seem to be any indication that is imminently about to work.

Conceptually incredibly interesting non the less.

It’s not professionals that need to something- it’s absolutely critical we have some kind of credibly neutral body providing the actual models. What’s not clear at all is how to do that since it’s such an expensive activity.

For now we have a great situation with the open weights, but there is absolutely no guarantee that continues, and more the opposite- they may stop.

Because if you don’t open weight you can dictate pricing as you’re the only party that can serve the model. Also attracts users to your platform rather than inf providers.

This is a major, major datapoint. That’s at least an order of magnitude higher margin than literally anyone is discussing. The entire ai discourse currently assumes very small margins on open model inf. And by this post it’s more like 1600% (h100s rent for $2/hr - and id guess deepseeks costs are far below that)

This is evidenced by the prices on every large-model capable device/node increasing significantly over the last year. Demand is far exceeding supply.

Your just missing many things and so have an incorrect picture of the situation - k3 is not on par with fable or astra. Closed models remain far better than best open weights at least today. - compute is not just used for training, more and more is inf - even in training you don’t do one run, you do many. Final run is a small portion of total compute.

The world is extremely compute constrained currently, like extremely.

Well I mean Verda is significantly cheaper than the hyperscalars for the exact same device so that’s one thing.

Maybe one argument could be there’s very little value added services required over the raw compute in this new world, and they’re structurally setup to capitalise on that whereas hyperscalars are not.

I’ve felt for a long time that the things I loved when I first started programming aren’t really there anymore. This comment made me realize that world truly is completely gone.

I can’t count the number of times I’ve heard variants of ‘they’re losing money on every query’ and ‘I’m getting 10k worth of tokens for $200’ over the last year. People clearly believed serving margins were -ve

What are you talking about - I feel like we’re living in parallel realities. If I had to go back to opus 4.5 tomorrow I’d be hugely upset and significantly slowed down

The author is trying to make the point that there are courses of action available to Dario that would achieve his stated goals but would result in Anthropic having very little impact, control or influence. And that because he’s not doing those things, his motivations extend beyond the stated goals.

It’s a good point and not exactly made subtly but seems completely lost on most comments here.

I can’t stand it. Very engineering-y over specified formal language around a complete lack of core understanding. Is damaging other people read this and try to learn things from it.

Causal masking allow model to learn implicit positional embeddings. The meme that a transformer block is permutation invariant is not true.

Intrinsic motivation/curiosity driven exploration/novelty seeking feels like it has a breakthrough paper waiting. If someone could get those methods working for LLMs, we start getting things like move37 but in math proofs and then everything else