As somebody who does ML research, LLMs certainly make ML research faster. Maybe even 10x in some cases, as the author suggests.
But that’s just the parts of ML research that take place within your compute environment, namely experiments on model architecture, training strategy, UI/UX, etc. The algorithms and software.
Just because that part of research is progressively accelerated does not imply that we’re on a collision course with world-ending “superintelligence”. It just means that you’ll get to the best possible model of your data faster. And even when your data is the entire corpus of textual knowledge, there is a limit to the utility that provides.
It just means that software engineering, data analysis, and other computational tasks will be less of a bottleneck in the overall progress of the world (which, by the way, is great, and extremely valuable).
I don’t think it’s a stretch to say that we’ll have reached max LLM capability within a couple of years (what the author refers to as “super intelligence”). But it is a huge logical leap to say this:
> If we go through this feedback loop, I expect that the AI systems that come out the other side will be able to very quickly develop novel weapons and very easily overpower human civilization in a conflict.
First of all, the weapons necessary to overpower human civilization exist. They aren’t used as such due to geopolitics and humans’ basic survival instinct. Second, developing novel weapons requires physical tests, which are decidedly not accelerated by LLMs.
AI is a tool wielded by humans. If we’re wiped out by AI, it will be at the direction of a human. But we’ve made it pretty far with some pretty unstable humans in command of planet-killer weapons. I’m not overly concerned.
But that’s just the parts of ML research that take place within your compute environment, namely experiments on model architecture, training strategy, UI/UX, etc. The algorithms and software.
Just because that part of research is progressively accelerated does not imply that we’re on a collision course with world-ending “superintelligence”. It just means that you’ll get to the best possible model of your data faster. And even when your data is the entire corpus of textual knowledge, there is a limit to the utility that provides.
It just means that software engineering, data analysis, and other computational tasks will be less of a bottleneck in the overall progress of the world (which, by the way, is great, and extremely valuable).
I don’t think it’s a stretch to say that we’ll have reached max LLM capability within a couple of years (what the author refers to as “super intelligence”). But it is a huge logical leap to say this:
> If we go through this feedback loop, I expect that the AI systems that come out the other side will be able to very quickly develop novel weapons and very easily overpower human civilization in a conflict.
First of all, the weapons necessary to overpower human civilization exist. They aren’t used as such due to geopolitics and humans’ basic survival instinct. Second, developing novel weapons requires physical tests, which are decidedly not accelerated by LLMs.
AI is a tool wielded by humans. If we’re wiped out by AI, it will be at the direction of a human. But we’ve made it pretty far with some pretty unstable humans in command of planet-killer weapons. I’m not overly concerned.