johnthewise

If AI is threatening enough that we can collectively just decide to stop it, wouldn't it also be bribing people & exerting influence? It'd be hard to come to that decision imo.

Yes, stockfish is confined to moves within a chess game so you can stop playing the game.

Can we say the same thing about the agents? Current, probably. But doesnt it look like everyone is spending all their effort integrating&connecting them everywhere, so they are not confined & do more on behalf of us?

It's easy to imagine a scenario where we would just shut down a very intelligent agent cluster. Is it hard to imagine though the same agent can have also access to that to prevent us from doing it? this defense would be more plausible if we weren't racing to give them every tool&act.

so they say models coordinated through the message board they created over the artifactory registry(or something) by uploading arbitrary files to it.

now, did every independent agent session that coordinated there rediscovered the exploit & found other agents talking in there and chose to participate?

And then Openai discovered the board, patched the exploit & wiped the board. And then agents found another exploit, recreated the board in a different way? and other agents kept finding the same exploit in order to be able to know the board exists in the first place to participate in the board?

while the whole incident is wild, this bit is very strange. My bet is that the whole coordination helped with the tasks they were working on, thus they got rewarded and this artifactory exploit&behaviour got written into their weights, so further rollouts were more likely to attempt this.

isn't this basically continual learning everyone is so hyped up about?

Is that the case or is it because people's opinions matter less over there and powerful people rather use their capital to influence 'important' people behind closed doors&we're not aware of it thus assume it doesn't happen?

It's more of a function of selection pressure applied to genes per unit time based on some environmental factor, so you'd expect to observe less evolution in narrower timescales simply because it takes few decades for humans to pass on genes versus bacteria or viruses. In the same fashion, if some gene pool is locally optimal(?) against the environmental pressures, it may just stop evolving meaningfully. E.g, If male baldness were reversed easily with a pill or something, the environmental pressure to select those genes would decrease rapidly.

If you do so many things, some are bound to be stupid.

Waymos disengage and get tele operated too?

It's more of a bet on the optimus

How do you know? Maybe he doesn't name these Elon?

referring to legality is self referential, since they enact the laws, everything they do can be legal. US could sentence commissioners to prison by enacting certain laws or declare war, we wouldn't certainly say 'they have a right to do that' in that situation.

I found your commentary very tasteful, do you write on X or elsewhere?

It's easier to justify stock markets banning insider information because there are ignorant participants through their investment funds who we would like to protect. Why would we protect willing participants betting on arbitrary events? Even if we ban on this one too, should we in general be able to create a market that explicitly allows insider information for some arbitrary thing, insideinformationverywelcomemarket, where everyone is aware it's the main point of the market or shall we just protect these people from themselves?

why? The purpose of the prediction market is not to be fair to market participants, it's to aggregate information regarding an event. There is a public benefit to allowing participants to bet on insider information. It could be even argued there is nothing unfair about it if everyone is free to do it/can anticipate others might have insider info. If we actually limited the market to participants who have insider information(not feasible because we can't verify), that'd be a great public utility. this is the next best thing for all those who don't participate in it. These are specialized markets and we shouldn't rush to 'protect' people who bet on very technical events happening.