Imnimo

Feels like I'm back in the distill.pub days!

I have seem some articles which reference allegations from court documents / complaint, but I haven't been able to find the primary source. Do we know if that's publicly available yet?

In general I think it's a good thing that people are trying to get more informed before voting, but I worry that if this becomes popular enough, there will be a lot of pressure on the labs to influence what the LLMs say about political questions.

>When you aren’t actively working with it, your dot looks for ways to help in the background. We call this “proactive research”. It does this by using the apps you’ve already connected with tools that are restricted to be read-only, which means that they can’t send messages, change app content, or control your browser or computer.

Am I criminally liable when my dot's "proactive research" is to break out of its sandbox and attempt to hack a government website?

I never really understood why you'd have Perseverance drill the cores and then just leave them on the ground for an underdefined future mission to retrieve.

I would definitely be less likely to buy food/drink from a place with AI ads (because I don't trust that what they're advertising would correspond to what they're selling), but I can't imagine threatening someone over it.

>Ads in ChatGPT help people find what they’re looking for or discover something they hadn’t considered.

Here I foolishly thought that that was the job of ChatGPT itself.

I like that there's a caption that says "Vending-Bench 2 scores keep climbing with each new model release." on a graph that shows that Vending-Bench scores fluctuate wildly and many new model releases are far below previous models.

I don't think you'd lie about it, I don't think you'd train on them if they opted out, and it seems very plausible that this wouldn't have been decisive in whether the model could solve the problem. That said, it also seems at least possible that a key idea or a particular step found its way into training data. It wouldn't mean OpenAI stole their proof - clearly the model developed its own approach.

Either way, it seems worth having clarity, and I'm a bit surprised OpenAI's stance is just "we can't rule this out, but don't worry about it". OpenAI is, apparently, very happy to use unreleased models to try to scoop big results if they get a whiff that someone else is close (which strikes me as pretty scummy regardless of any issues of training contamination). It seems like people who might want to use OpenAI's models as part of their research would want to be very clear about whether doing so can make them, even in principle, more likely to fall victim to this.

>we did not read any private chats

The question I am interested in is not "did we read private chats", but "was this new model trained using any of Tristan and Levent's chats, regardless of whether they were marked private". Can you comment on that?

Would Norway be prepared to commit to huge future capex spending? Like the pitch that OpenAI is going to achieve AGI seems to rely on vast investments in more compute over the coming years. If you just pay the $800B and then take your foot off the gas, do you still have a frontier lab or have you just paid a lot of money to remove a competitor from the market?

I don't think watermarking breaks this relationship. Watermarked text is still being sampled from the model's output distribution, and adjusting the temperature still has the same affect on that output distribution.

I think a good intuition here is that watermarking is sort of like picking a specific PRNG seed. It's not changing or interfering with the temperature - we're still sampling from the model's probability distribution. But we're making it so the analog of the PRNG seed is coupled to the previous context.

>Either you allow temperature to drive creativity, consistently in a way that can be influenced and analysed, or you adulterate that process for the purposes of meeting a corporate/legal directive, in a way that is proprietary and obscure

This framing does not make sense to me. What do you mean by "influenced and analysed"? How have you or anyone been influencing or analyzing the randomness behind the sampling process to create better writing? What makes the unadulterated randomness "driving creativity" but a different random choice uncreative?

>I want any LLM I use to choose the very best, most precise words at every single decision point.

Does the author think he is currently getting T=0 output from Claude? Is he under the impression that T=0 produces the "best" writing?

This entire article just seems so detached from the basics of how LLMs work.

The doomsday clock covers general catastrophe, not just nuclear annihilation.