Comment by emtel

It means that NNs don't (as previously thought) smoosh unrelated concepts together into the same neurons, which would have made interpretability very difficult.

For instance, if you had a single neuron that fires for "puppies" and "murder", it would be much more difficult to tell if the model is thinking something nice or something not so nice.