prohobo

We already had large societies that worked in a similar way during feudalism, but economies still relied on human labor. Today, there are countries where the majority of wealth is not reliant on citizens producing/consuming anything: Qatar, Kuwait, and the UAE primarily sell resources to other states.

No one knows exactly what happens if you scale that up to the entire planet, but my view: the owners will be the ones who trade because of the resources available on their territory - or technology they own. You get a few powerful rentier states, where citizens get stuff for free so they don't revolt. Everyone outside of those states becomes part of a stagnant periphery.

That's possible, but it'd still probably hit those limits faster than any person or team could. Then maybe there'd be human-led work to find some kind of breakthrough that unlocks research up to the next plateau.

At some point though, I'd assume the AI would start finding breakthroughs itself.

An economy can keep running even if most people are cut out of it. Production would just reorient to who owns capital, the state, and other AI systems buying from each other.

But, if I get the insinuation, this would still require a huge social change to absorb once broad consumer demand falls off.

As for pursuing math for fun: that would turn most disciplines into something akin to speedrunning, chess, or mountaineering. People would add constraints to make it harder for themselves and compete against each other, and that does motivate some - but not all. A lot of people want to be "on the frontier." If discovery gets automated, the remaining frontier may be physical, actually going places no one has been, and that might pull those people toward space as it opens up.

There are quite a few problems with this video, IMO:

- Grant is a highly respected and popular YouTuber. He has a strong network, personal connections, access, and his income comes from an audience, not from output he’s measured on. He's perfectly positioned in this AI revolution to end up near the top of whatever human pile gets constructed, so he will not have the kinds of fears and anxieties that most people do.

- AI RSI is being actively developed and is likely to arrive within 1-2 years. I wouldn't consider this a fringe scenario; it's the main bet. Even if models stay weaker at theory building than humans, organizations will spin up 100k agents and the fire-hose of generated ideas will probably compound far faster than anyone can keep up.

- His remaining arguments are basically training people to problem-solve, and joy. For the next few years, I can see his particular niche doing quite well as infotainment, and he will probably make a lot of money. However, I don't really see people being able to join and partake in his industry 10-20 years from now. Also, if we project a realistic and grounded understanding of our current world onto the next couple of decades, what would give anyone the idea that political and economic systems would have any incentive to train people to problem-solve if AI becomes the real work-horse? Teaching will likely be defunded and turned into a consumption good (which it arguably already is becoming with these YouTube channels.)

- So, for me his entire argument boils down to... joy?

Others have already commented that this feels like unprincipled optimism, and I agree. I also think that because this is something made to generate revenue through a social media platform, it's inherently performative and on-brand. His incentives pull toward keeping his audience happy.

Unfortunately, outcomes follow incentives, not optimism. If our economy comes to rely on RSI-enabled AI, most people will simply have no place in any industry it touches.

I would say it's more like an interface for the model to interact with the world. If you give the model access to filesystem and bash that technically unlocks all computer use, so how are you going to control that? By trying to regex match against the commands the AI uses? All you have is auth or containment, and AI can hack auth and people will not stop connecting AIs to the internet. It's a ridiculous premise that just because the harness is "normal code" that means we can control the AI.

The world's institutions, systems, and industries are all rapidly digitizing. So while I'd concede the point that, yeah, there's no way a rogue AI can just take over some powerplant and blow it up because of analogue systems the AI can't access, that isn't necessarily true for some powerplants already, and more and more powerplants will be connected to networks and controlled by software systems in the future. The more we digitize our systems the more potential for AI to exploit vulnerabilities and affect the real world.

AFAIK there isn't that much stopping anyone from spawning an AI swarm and telling it to "spread and go hack everything for the lulz."

Design a virus that is perfect for transmission and killing the host slowly, and seed it in a few hot spots? I don't really understand why you can't wrap your head around that, it doesn't even require a lot from the AI:

1. Control over some automated bio research lab (be given access, or hack in)

2. Access to drones that can deliver the payload (or manipulate humans into delivering it themselves)

On the intelligence side, you just need an AI agent/swarm capable enough to design viruses better than we can and evade detection for long enough (already plausible.)

I agree that this "AI will kill us all" narrative is some kind of fantasy horror fiction, but I can't deny that given the right amount of access, AI can do a lot of damage.

I think we both know that we've had plenty of time to figure out the situation, understand the context, address our own biases, and come to a sincere conclusion about this issue. We also both know that one of us is obviously, insanely wrong - so either someone isn't engaging in that process, or there's something deeply wrong with sense-making in general.

You're not going to convince me, and I'm not going to convince you. All I can say is: good luck.

But Elon Musk didn't do a Nazi salute nor is he a fascist? You can call him a techno-feudalist, technocrat, billionaire asshole, scammer, scum, narcissist, idiot, nationalist, chauvinist, thief, political meddler, etc. etc. and I may or may not entirely agree, but when you just make extreme things up to align with a particular political ideology it makes me angry, and I won't flag it but I understand why others do.

I understand that a lot of people are motivated to think he did a Nazi salute, but he didn't. He just accidentally made a major social faux-pas as nerds tend to do.

So, please go ahead and attack him and Trump and all the other avatars of "the other side" using the abundant real rhetorical ammunition they give you. I draw the line when delusional takes are supposed to be taken seriously though, because it makes an ass out of you and me.

As for your "personal favorite" comment that got flagged: I agree, that one probably should be allowed. People attack Microsoft, Meta, and Google all the time in threads related to their products.

It works for smoking, but I don't think it does as well for vaping? You just can't rationalize the addiction the same way because you don't stink, you probably won't get cancer from it, you do enjoy it to some extent, etc.

People don't understand that cigarettes are self-limiting because they literally hurt your throat and lungs. Most people simply can't chain smoke and they need about an hour break in between smokes.

With vaping that limit is completely gone. A vaping addiction can keep escalating until you want a puff every 30 seconds, especially once you start breaking down mental barriers like "no vaping indoors" or "no vaping while on the computer", etc.

So in terms of pure addiction I would 100% argue that cigarettes are better than vaping.

Some anecdata:

- It's extremely verbose and often incomprehensible when doing even basic tasks. Like it'll write a giant jargon-filled essay then end it by asking for a judgement call on something that references its own convoluted jargon.

- You can ask it to do research on a topic, and it'll just straight up be lazy, pretending it's really digging deep to find stuff when actually it's just grabbing cached SEO snippets off a search engine.

Both of your comments are illuminating :p

So, we could technically debug a prompt's output? I get that there are too many steps to actually step thru, but what if there were checkpoints? At least you could isolate behaviors to specific sections of a neural network?