NaiveEmpiricist

I'd be interested to see where this goes, but it seems to rely on LLMs being mostly or entirely compositional (i.e. ultimately, they can be taken apart to be understood). But if LLMs have notable emergent properties - which seems likely - then those parts of the LLM will be effectively off-limits to this sort of interpretation.

And emergence doesn't need to involve much of the LLM's weights:

> In preliminary audits of Qwen3-32B, we estimate that roughly 90% of neuron poles have one main job, often with small related offshoots; another ~7% have two related jobs; and only ~2-4% are not yet understood

If those ~2-4% are the central hubs of emergent cognition, then understanding 96%-98% of the weights may be less valuable than the author thinks.

Butlerian Jihad?