gurkendoktor

Humans are great at making up purpose where there is absolutely none, and indeed this is a helpful mechanism for dealing with post-scarcity.

The philosophical problem that I see with the "AI overlord age" (although not directly related to AI) is that we'll then have the technology to change the inherent human desires you speak of, and at that point growing tomatoes just seems like a very inefficient way of satisfying a reward function that we can change to something simpler.

Maybe we wouldn't do it precisely because it'd dissolve the very notion of purpose? But it does feel to me like destroying (beating?) the game we're playing when there is no other game out there.

(Anyway, this is obviously a much better problem to face than weaponized use of a superintelligence!)

Your comment summarizes what I worry might be a more widespread opinion than I expected. If you think that human extinction is a fair price to pay for creating a supercomputer, then our value systems are so incompatible that I really don't know what to say.

I guess I wouldn't have been so angry about any of this before I had children, but now I'm very much in favor of prolonged human existence.

One avenue of hope for successful AI alignment that I've read somewhere is that we don't need most laypeople to understand the risks of it going wrong, because for once the most powerful people on this planet have incentives that are aligned with ours. (Not like global warming, where you can buy your way out of the mess.)

I really hope someone with very deep pockets will find a way to steer the ship more towards AI safety. It's frustrating to see someone like Elon Musk, who was publicly worried about this very specific issue a few years ago, waste his time and money on buying Twitter.

Edit: I'm aware that there are funds available for AI alignment research, and I'm seriously thinking of switching into this field, mental health be damned. But it would help a lot more if someone could change Eric Schmidt's mind, for example.

To be fair, ants have not created humanity. I don't think it's inconceivable for a friendly AI to exist that "enjoys" protecting us in the way a friendly god might. And given that we have AI (well, language models...) that can explain jokes before we have AI that can drive cars, AI might be better at understanding our motives than the stereotypical paperclip maximizer.

However, all of this is moot if the team developing the AI does not even try to align it.

For one thing, we could try to come up with safety measures that prevent the most basic paperclip maximizer disaster from happening.

At this point I almost wish it was still the military that makes these advances in AI, not private companies. Anyone working on a military project has to have some sense that they're working on something dangerous.

Not you specifically, but I honestly don't understand how positive many in this community (or really anyone at all) can be about these news. Tim Urban's article explicitly touches on the risk of human extinction, not to mention all the smaller-scale risks from weaponized AI. Have we made any progress on preventing this? Or is HN mostly happy with deprecating humanity because our replacement has more teraflops?

Even the best-case scenario that some are describing, of uploading ourselves into some kind of post-singularity supercomputer in the hopes of being conscious there, doesn't seem very far from plain extinction.

Self-correcting because frustrated users will simply start their own Google? Even if that happens, their second generation of employees will start a revolt if their company doesn't follow the latest DIE best practices.

I honestly think that only the Russian/Chinese model of a nationalized IT ecosystem has a chance to resist these trends.

I disagree. The appeal to solidarity w.r.t. masks would never have worked because a tiny minority of scalpers can ruin it all. But the white lie didn't work either, so it seems like the worse option regardless.

Appeals to solidarity w.r.t. staying at home, wearing masks, getting vaccinated to flatten the curve all worked quite well in my bubble. Projects like Zero Covid would have required insanely high levels of compliance. If you say "anything involving solidarity with others does not work anymore", how high is your bar? What used to work but now doesn't?

(I'd guess that solidarity is lower than it used to be, but if anything I am surprised how much goodwill still exists considering growing inequality, atomization, erosion of public trust and all that.)

Agree with everything you said, but want to add one more point:

I have made my peace with all of these bureaucratic failures. Maybe a certain amount of contradiction and chaos is part of living in a democracy.

The scary part is that at the same time there has been an authoritarian push to crack down on "disinformation", you know, like the lab leak theory, or the conspiracy theory that vaccination will become mandatory. That has sent my trust from low into negative territory.

Notably, the Mac Mini does not have a stupid external power adapter, but just uses a standard cable that costs like $1. I did carry mine around for a while, and it was glorious! I'm not sure why Apple's competitors insist on including a brick that's almost as large as the PC itself.

I went from a 16" MacBook Pro with an i9 and 32 GB of RAM to the MacBook Air with 16 GB of RAM, and my build times at work were almost exactly the same. But the single-core speed makes everything _feel_ so much faster than the stupid slow i9 cores. (At less than half the price, and without a fan!)

In my opinion, the Air is the perfect addition to a heavy Linux/Windows workstation. You get the full spectrum from iOS development to gaming, albeit at the cost of having to babysit two or three operating systems. You can use the Air for meetings or conferences, because everything that's not a game has a macOS version (think Microsoft Teams etc.)