sigpwned

I agree, that seems like a configuration/policy/operational approach, which is how we handle this problem now for humans using RBAC and authn/authz, just applied to AI. People do a crude version of this today with sandboxing (where the AI's sphere of influence is strictly limited by its environment, barring misconfiguration of the sandbox or breaking out of the sandbox, of course) and with workflows (where AIs are integrated into deterministic workflows, and then deterministic, non-agentic code decides how to handle AI outputs). But integrating this into more agentic architectures with finer control just seems like a best practice, said that way. It's not a tradeoff, there's no drawback, just do it. In other words, yes, a no-brainer.

True. "AGI" has also become a marketing term. Achieving AGI has become valuable, so companies will move the AGI goalposts, over and over again, so they can achieve AGI, over and over again.

Yes, you certainly could put that on a chip. And people are doing it, for smaller models. The question is how big that chip would be for something like Fable, which is generally estimated in the trillions+ of parameters, and if, given the amount of memory, computation, and bandwidth required - at least with current methods, i.e., very high dimensional matrix math — a chip is the right way to go. Can you put a LM (just a smaller language model) on a chip? Empirically, yes, I believe it is done. Small enough ones probably even fit on a FPGA. Can you put a LLM on a chip? Depends on how L it is! My intuition says that some open smaller models might fit, like Haiku, but not Fable. But that’s just intuition talking.

The hope would be that this unlocks some substantially more efficient or parsimonious math that would fit better on a chip. And that’s clearly my words, not the authors’, per the comment above.

My naive middle-of-the-night Claude question said the same thing. At least for this approach. I have not read the paper closely enough to refute you. But the concept of a lower-dimensional closed-form solution — which the paper seeks to discuss, please correct me if you read differently — is tantalizing, if only because it opens the path to different math, which can lead to optimization.

I agree. In retrospect, this seems almost inevitable. And our own minds at least to do some form of symbolic reasoning — literally language, which you are apparently capable of, dear reader, as a lower bound. There might be more symbolic reasoning in the conscious, unconscious, and subliminal parts of the mental workspace. I always hesitate to infer similarities between the brain/mind and LLMs, but I certainly track the comparison here.

Good find! But they stop short of saying it cannot be distilled to symbolic algebra. Regardless, your point stands. I scanned the paper in the middle of the night instead of sleeping. Clearly I didn’t do a close read! Thank you for pointing that out.

The big questions I’m taking away are:

(1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disruptive - in many ways.

(2) Unsupervised, and even supervised, symbolic approaches to problem solving break down due to combinatorial explosion, among other things. This could potentially allow us to treat LLM training and inference as a search algorithm for novel symbolic approaches to solving new classes of complex problems hitherto unreachable through other approaches. If that works, I suspect it’s a feedback loop, too - the learnings from one representation push advances in the other. This would also increase the economic value of large training runs, since the model itself is now valuable, not just its inference.

(3) Per the above, can this push LLM design to greater capabilities?

The relationship between this and Anthropic’s J-space observation is also interesting. This is much, much deeper and more directly actionable, though.

EDIT: I ran my questions through Sonnet — yes, I appreciate the irony — and it was none too sanguine about questions (1) and (2), but thought (3) was reasonable. In any case, this is quite the paper. On reflection, I do think that the apparent reliance on very simple symbolic representations and tasks is underwhelming. But the approach is impressive. And obviously this is still early days, and the value of building a bridge between the very fuzzy LLM models and the rigorous, mechanically provable models would be enormous.

False, this is exactly as much as I wanted to know about how Game Boy cartridges work. Thank you! :)

I'm working on some social media analysis tools for Bluesky. It's unbelievable that there's an active social network for which you can see all the data.

I don’t disagree. But I also wonder if there even is an objective “right” answer in a lot of cases. If the goal is for computers to replace humans in a task, then the computer can only get the right answer for that task if humans agree what the right answer is. Outside of STEM, where AI is already having a meaningful impact (at least in my opinion), I’m not sure humans actually agree that there is a right answer in many cases, let alone what the right answer is. From that perspective, correctness is in the eye of the beholder (or the metric), and “correct” AI is somewhere between poorly defined and a contradiction.

Also, I think it’s apparent that the world won’t wait for correct AI, whatever that even is, whether or not it even can exist, before it adopts AI. It sure looks like some employers are hurtling towards replacing (or, at least, reducing) human headcount with AI that performs below average at best, and expecting whoever’s left standing to clean up the mess. This will free up a lot of talent, both the people who are cut and the people who aren’t willing to clean up the resulting mess, for other shops that take a more human-based approach to staffing.

I’m looking forward to seeing which side wins. I don’t expect it to be cut-and-dry. But I do expect it to be interesting.