adamzenith

One of the most well known interpretability researches released some potentially very useful research as a blog post, and he asked claude to make a cool little thing on top of it?

Do you have a source, and also so what? This could have just as easily been codex or any other harness.

You don't think having a more intelligent model they can use internally that others can't is an advantage?

> There is nothing to suggest that an AGI will care about its future or who controls it.

Isn't this exactly the scary thing? A sufficiently advanced agent unleashed with an imperfectly defined goal will do whatever it determines it needs to do to accomplish it, careless of the consequences.

The field is much older, MIRI is ~20 years old. Look up Eliezer Yudkowsky.

I think more the idea that wrestling with new ideas and distilling them down can you help you to understand something, even if that note is then immediately thrown out. The purpose of the notetaking in this case is not for future reference.

In response to your edit, you should check out Terry tao's chat gpt logs about the recent Jacobian result. The models are smart enough to brute force some things, but can cut to the meat much faster with good prompting

Not under all the competitive pressure they are facing, but if an industry wide slowdown could be achieved they might spend more time on model alignment and less on pure capabilities.

What is the issue with Proton?