Comment by stingraycharles

> It's actually how organic brains work - specialized tasks are offloaded to local cortical columns.

How are small isolated language models more similar to that than MoE in LLMs?