roca

If hacking one devops engineer's laptop is enough to move $6B of assets out of Binance, Binance is staggeringly incompetent and their customers need to move their funds out right now.

North Korean nuclear weapons engineers thank crypto fans for their ongoing contributions.

The people I personally know who signed the "pacing the frontier" petition (including me), or who called for it in other ways, are sincere in their concerns that things are moving too fast for safety.

With stolen cloud accounts and credit cards you can rent GPU and TPU time from the cloud providers. Stealing enough to train a new model might be hard but stealing enough to do inference with a frontier model would not be hard.

It's not that simple; the goods of AlphaFold are mostly not yet realized, and some negative effects of AI are being realized.

More importantly, as I tried to make clear in my post, the main reason I quit is not specific fears of AI doom (I'm uncertain about that), but my belief that AI is progressing too fast.

Google (unlike Meta say) makes a lot of money from a lot of things that have nothing to do with the addictiveness optimization.

I don't believe at all in the idea that cheaper LLMs will somehow "democratize LLMs". Rich people will always be able to buy far more compute than poor people, and we know how to turn more compute into more capability (bigger models, more reasoning and more agents).

The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.

You have a fair point. I am financially secure so it's not a great sacrifice.

It was psychologically difficult to give up the opportunity to make more money at a fun job, though.

Thanks for the thoughtful comment. I go on this sort of trip pretty regularly so I don't expect this one to reorient me dramatically. They are good for my mental health though, for sure.

I think there are pretty clear mechanisms by which AI (a few model generations from now) could bring down civilization as we know it. One of them is a runaway cybersecurity scenario where hacking one set of systems quickly enables the AI to hack more systems (via supply chains, stolen credentials etc) and also host more instances of itself, and eventually everything Internet-connected that's not invulnerable just falls apart, which probably includes most electrical grids and a lot of other infrastructure. Yeah, the wilderness and countryside would still be there, but I for one can't live there without returning to the supermarket regularly for supplies. A lot of people would suddenly have to live (or not) without their meds. It would be bad.

I didn't, and still don't, think Google was unambiguously good or evil. They did dirty stuff with Chrome, they do addiction optimization, but they also provide great free services, fight endless wars against spammers, and provide reliable secure infrastructure to a lot of other people, and lots of other good things. I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.

It's pretty hard to live in the world and be completely disconnected from anything unethical. Jesus doesn't seem to demand that (e.g. he didn't tell centurions to leave the Roman army).

> Are you aware tech bros have been saying this same thing about every AI advancement since the 1950s?

Whatever AI people have been saying for the last 70 years, AI capabilities have improved stupendously in the last 10 years and there is no sign of the rate of improvement slowing down. I would like to deny it, but I can't.

FWIW I was in the GDM GenAI unit. Not exactly working on AI directly, or I would have quit earlier, but in the middle of it.

Yeah, it's an age-old question. My high-level answer is, God's plan will be done regardless of what I do, but my actions determine (at least from my point of view) whether I'm aligned with the plan or not. And I want to be aligned with the plan.

(OP here) FWIW I despise the addictiveness, er engagement optimization that Google and others do. I have said so inside and outside the company many, many times. I would never have agreed to work on that stuff myself. One of the hobby projects I would like to work on is exploring how to use personal AI to beat those algorithms.

I respect the argument for not working for a company that does addictiveness work anywhere, even if you don't work on it yourself. But that's not where I drew the line.

Also FWIW I disagree the danger of AI is abstract. A lot is happening right now, in education for example.