frabcus

The coding was cute but not particularly empowering autocomplete which you had to carefully review until about 1 year ago. It's considerably more capable now, to the extent it is a fundamentally different use case. You can really just ask the AI to fix a ticket and it will and won't mess everything up.

If only!

Alas natural selection (other people didn't try to start companies to make superintelligent AI) has picked leaders with the opposite strategy - do it first, and try to use that power to control everyone else in, at best, an attempt to stop disasters caused by others.

These are all ad hominem arguments. And while I can see how it's possible to find them compelling, it avoids discussing the reality of the arguments about AI risks, which are varied and convincing in and of themselves given observable capability growth.

Interestingly, pretty well everyone I've mentioned my own emotions about AI risk to is worried about it too. I don't think they'd be able to rationally express scenarios of the earth being tiled with data centres, but there is an ambient intuitive sense that this change is having impact. They don't really nitpick which impact I'm talking about either (there are lots of them).

I was a bit too young to really remember what adults felt and said about the cold war. By the 80s it felt slightly less scary I think than it would have in the 60s? There already being a nuclear hotline and SALT talks and other things learnt from the Cuban missile crisis. Fewer countries had nukes too.

There would be one solution, if Apple actually proposed having one and formed a standards body, and made one solution with Google and ticketing companies and the rest of the industry. That they don't shows they don't care about customers.

So this is a whole extra workflow that makes Apple only wallets?

How about instead agreeing a standard joint with Google, so people only need to make passes once, not separately for each smartphone platform?

Would be good if the tech companies could actually make things easier, not try to lock businesses into particular platforms, and make everyone do extra special work just for their platform.

The difference is that potentially every (white collar) job could change radically in the next four years. That's what makes it hard for society to deal with, in a way any previous example I can think of doesn't.

Right, but what happens when everyone uses that at scale? Without agreements and standards it isn't pretty.

It's particularly important we all scan and destroy our own unique books!

That's the goal of AI, yes.

They can't right now - but with enough compute they sure can grant the wish of e.g. hacking a billion dollar company, or getting root on the eval cluster of a frontier model lab.

What's your definition of "wish"? Tell me the above 10 years ago, and it would be at "wish level".

I'm pretty (from talking to people) that they're not subsidised in the simple sense. They are:

* low/no margin, unlike the API which is very high margin

* gym-membership subsidised - most subscribers don't max them out, mainly us coders are being "subsidised" from users just using it as a research chatbot

Even the models we have access to can do basic coding tasks really well, in a way they couldn't a year ago.

I don't really care what you call "intelligence" - what matters is capability, and the consequences of that capability.

They're not selling the superintelligence yet - the models in the Hacking Face incident and that solved the Millennium prize are not released yet. And that is by no means the final capability level the current process looks like it will get to.

The freeze is to stop superintelligent models. Not to stop other companies catching up with current models we all have access to. It would help them catch up.

Yep - fair on legal details of "rogue agent". I think OpenAI should be directly responsible, and the agent was rogue. We should pass new laws if necessary.