goolulusaurs

The reality is that o1 is a step away from general intelligence and back towards narrow ai. It is great for solving the kinds of math, coding and logic puzzles it has been designed for, but for many kinds of tasks, including chat and creative writing, it is actually worse than 4o. It is good at the specific kinds of reasoning tasks that it was built for, much like alpha-go is great at playing go, but that does not actually mean it is more generally intelligent.

In my younger years, particularly during my schooling, I held a deep resentment towards the educational system. It felt overtly clear to me, as a student, that schools failed to effectively foster learning and growth. However, my perspective has evolved over time. I've come to understand that the issues I observed are not unique to the school system but rather characteristic of large institutions as a whole.

The pervasive failure of these institutions to meet their stated objectives isn't an isolated phenomenon. It's symptomatic of a larger, systemic problem – the widespread presence of perverse and misaligned incentives at all levels within large organizations.

Unless we find a way to counteract this, attempts at reform will merely catalyze further expansion and complexity. The uncomfortable truth is, once an organization surpasses a certain size, it seems to take on a 'life of its own', gradually sacrificing its original mission to prioritize self-preservation and expansion. Who has ever seen an organization like this voluntarily reform itself? I certainly haven't.

It depends on what you mean by "this discussion", but I don't think that follows.

If for example, we were in scenario 2 and it was still the case that a large number of people thought AI doomsday was a serious risk, then that would be a much stronger argument for taking the idea of AI doomsday seriously. If on the other hand we are in scenario 1, where there is a long history of people falling prey to apocalypticism, then that means any new doomsday claims are also more likely to be a result of apocalypticism.

I agree that is is likely that humans will go extinct eventually, but I am talking specifically about AI doomsday in this discussion.

>It was clear that nukes were a risk before they were used; that is why there was a race to create them.

Yes, because there were other kinds of bombs before then that could already kill many people, just at a smaller scale. There was a lot of evidence that bombs could kill people, so the idea that a more powerful bomb could kill even more people was pretty well justified.

>if AGI is to become a thing, what does the moment look like where we can see it is coming and still have time to respond?

I think this implicitly assumes that if AGI comes into existence we will have to have some kind of response in order to prevent it killing everyone, which is exactly the point I am saying in my original argument isn't justified.

Personally I believe that GPT-4, and even GPT-3, are non-superintelligent AGI already, and as far as I know they haven't killed anyone at all.

> So far, 100% of those which are as intelligent as humans have dominated the planet.

This is a much more subjective claim than whether or not the world has ended. By count and biomass there are far more insects and bacteria than there are humans. It's a false equivalence, and you are trying to make my argument look wrong by comparing it to an incorrect argument that is superficially similar.

Consider two different scenarios:

1) Throughout history many people have predicted the world would soon end, and the world did not in fact end.

2) Throughout history no one predicted the world would soon end, and the world did not in fact end.

The fact that the real world is aligned with scenario 1 is more an indication that there exists a pervasive human cognitive bias to think that the world is going to end, which occasionally manifests itself in the right circumstances (apocalypticism).

Well, for example I believe that nukes represent an existential risk, because they have already been used to kill thousands of people in a short period of time. What you are saying doesn't really counter my point at all though, it is another vague theoretical argument.

Sure, but don't they have a vast network of contact tracing? Presumably if a bunch of cases showed up that weren't connected to any other cases because of the lockdowns then that would be a really good indication of where the animal reservoir was located. But they haven't found the animal reservoir yet either, even with all of the lockdowns.

Well its been years, why haven't they found the reservoir yet then? They would obviously want to since it would prove that it wasn't a lab leak, yet as far as I know they haven't claimed to have found it yet.

IMO, it seems obvious from the behavior of China's government that they know it is a lab leak. If it wasn't a lab leak, then presumably there is an animal reservoir of the virus somewhere in China, but as far as I know they haven't claimed to have found it. But if there is an animal reservoir of the virus in China, then how could the Chinese government ever expect a lockdown to work? A lockdown on travel would only really prevent the virus spreading from people bringing it into the country but obviously wild animals would still be spreading it. Yet the Chinese government claimed that their lockdowns did work. How is that at all compatible with the virus being from wild animals and not being a lab leak? It doesn't make any sense.

There is a lot of negativity towards the idea of AI in this thread, and I feel like someone has to say it: it is quite likely that in the near future computers will be better than almost all humans at almost all cognitive tasks.

If you have a task or are trying to accomplish something, and the way you do it is by moving a mouse around or typing on a keyboard then it is very likely that an AI will be able to do that task. Doing so is a more or less straightforward extension of existing techniques in AI. All that is necessary is to record you performing the task and then an AI will be able to imitate your behavior. GPT3 can already do this for text, and doing it instead with trajectories of screen, mouse and keyboard is not fundamentally different.

So yes, it is true that there is a lot of hype right now, but I suspect it is a small fraction of what we will see in the near future. I also expect there will be an enormous backlash at some point.