mordymoop

All that seeming advantage, and yet Google are perpetual laggards in the AI race. In enterprise, where the money is, they're third behind Anthropic and OpenAI, despite what should have been a colossal incumbency advantage. They're also inexorably poisoning their own brand and squeezing their primary revenue source with AI Overviews. Truly a masterstroke of value destruction. Search revenue growth has slowed three quarters running, even as query volume hit a five-year high. As for their hardware and infrastructure advantages, those are indeed making them money, because they're selling and renting the chips to Anthropic. I dunno. I've been a fan of Google for years. Maybe they'll successfully pivot to being a hardware company. They're too slow and hamstrung by internal politics to make compelling LLM products.

Let's Play Interpretability is a Twitch livestream where you can help Claude semi-autonomously interpret the neurons, connections and circuits in the Bonsai-8B LLM.

Your help comes in at various stages in the interpretation pipeline. The audience and the agent collaborate to:

Pick the neuron to work on Suggest sentences on which to test activations Explicitly test various hypotheses, and attempt to falsify them Branch out to nearby (upstream and downstream) neurons whose ablation most impacts the activation, to better solidify the interpretation ... and more.

Your comment prompted me to check what kinds of definitions of AGI were actually in the air back in the mid 2010s. A lot of people conflate AGI with superintelligence. I'm not accusing you of doing that, but it's a common enough mistake that I wanted to assemble these quotes as a record that can be referred to.

Here are some definitions of AGI from back then, quoted verbatim where possible, with my comments added.

Gubrud (1997), first recorded use: "By advanced artificial general intelligence, I mean AI systems that rival or surpass the human brain in complexity and speed ... " Modern LLMs completely blow us out of the water on speed and this has been true for over a year now. As for "complexity", we will forgive Gubrud, but that isn't really a meaningful concept. That said, the biggest modern LLMs are probably comparable to human brains in complexity at the level of analogizing synapse counts to parameter counts. Whether this is the right lens is up for debate.

Goertzel, Journal of AGI review (2014): AGI is a field building “general-purpose systems with intelligence comparable to that of the human mind.” Any leading model satisfies this definition easily, if only because "comparable" is a loose term. If modern LLMs can substitute for any expert in a knowledge work field and even directly teleoperate robots to perform complex tasks without prior training, that already sounds more than "comparable."

MIRI / Muehlhauser (2013): “the ability to achieve complex goals in complex environments using limited computational resources ... ” Well, I don't like this definition at all, but modern LLMs do satisfy it. They may be relatively inefficient in the training phase but they're incredibly efficient in the inference phase. (By "efficient" of course I mean information-efficient, not energy-efficient; this is surely the sense in which the definition was meant.)

Wikipedia in ~2016: “Artificial general intelligence (AGI) is the intelligence of a machine that could successfully perform any intellectual task that a human being can.” This is the most demanding definition of the bunch. But still, are there really any remaining intellectual tasks that cutting edge models can't achieve? If you think that you have an example one - are you sure it isn't just an example of a thing that nobody has bothered to ask, or bothered put a lot of compute behind? The operative concept here is could successfully perform, without regard for compute requirements. This may sound weaselly, but it is extremely relevant. To my mind, the question of "is it AGI or not" is about raw capabilities, not efficiency. Nobody writing sci-fi in the 2000s would have cared too much about the compute requirements of their speculative tech. In fact, a common trope was something like a gigantic mainframe that could be asked one question at a time. The modern instantiation of a gigantic mainframe that can hold 10 million parallel conversations was barely dreamed of.

It almost seems like people don’t understand that most of the training of modern LLMs is now reinforcement learning on problem-solving tasks. They spend most of their training literally solving problems that they don’t already know how to do.

I looked up his achievements to double check my understanding before writing my comment. Convnets are regarded as his headline achievement. Everything else is basically incremental. Note that I am not calling him some kind of fraud or moron. He is “just” a normal academic, a pretty smart guy who did research in his field for decades. Many thousands of people have similarly impressive resumes. This does not mean they should be deferred to.

See sibling comments: Modern AI is AGI according to any reasonable definition from prior to 2016.

AGI doesn’t mean superintelligence.

Modern AI is AGI according to any reasonable definition from prior to 2016.

Most problems are actually not that hard; most everything is actually mainly a result of right place, right time. If I hadn’t been the one to do my PhD research, someone else would have. Very few people are actually paradigm-shifting geniuses. This is fine. Good, even. But it does imply that people who are early to a field shouldn’t generally be deferred to. If anything, they’re more likely to have outmoded opinions.

Modern AI would be considered AGI by anyone having this conversation in 2016.

I think it’s time to consider that maybe being a venerable graybeard of AI just means you happened to be early to the party. Most of the “breathtaking” innovations of the early pioneers of the field are obvious solutions that anyone would have thought of when faced with the problems they encountered. LeCun’s primary contribution was Convnets, which is almost literally just “what if we organized ANNs in a way similar to how animal visual neurons are organized?”

In other words, I’m kind of tired of having to hear the opinions of dudes whose claim to fame was being at the right place at the right time. I’d rather hear from people who correctly predicted 10 years ago that AGI would arrive by 2027 (of which there are many) than people who continue to insist that it somehow won’t.

I would phrase it slightly differently.

The companies who control the compute resources will ~always control the greatest "amount" of intelligence. They can lease that intelligence out, or they can use it themselves. Currently the "total amount of intelligence" or perhaps "total amount of ability-to-do-stuff" is split between humans and machines at a ratio that means it still makes sense to lease the machine intelligence to the human intelligence - plus there are things that humans are still better at. In maybe 2 more years that will stop being true, due to the availability of more physical compute resources, and far greater model intelligence per unit compute. At that point, the point at which the substantial majority of ability-to-do-stuff is controlled by machine intelligence, then the entities who control all the compute will control all the ability-to-do-stuff, i.e. "the economy."

So I agree that the core product is not long-term sustainable as a product but this is because the whole world will look so different in the near future that the framing of intelligence as a "product" breaks down.

Open-Weight models, of course, are fine and useful, but if you have one million times less compute than your competitor (the lab), then you're not really playing the same game. You can only tackle the problems that they have decided they're not interested in.

I don’t see how assuming they made a normal kind of dumb mistake, instead of pursuing a criminal conspiracy, is giving them the benefit of the doubt. It’s just using reason.

I keep seeing this take, but it’s more likely that they just underestimated their models’ capabilities and/or overestimated their own safeguards.

Ever single person who uses LLMs on a daily basis has a fun story about their agent “taking the initiative” to do something beyond what was asked for. Looking for shortcuts to solve the problem is commonplace LLM behavior. It’s what you would expect to happen if you have an agent a hard task and unlimited runway. No need to suppose a conspiracy, this outcome was predictable the whole time.

In my callow youth I was responsible for a couple of fender benders and the cause was always a highly imperfect understanding of the appropriate stopping distance of my car. This obviously gets worse, the worse the conditions are. My Tesla never makes this kind of mistake.