>Instead of a single transformer, we combine (i) a stack of heterogeneous DNNs paired with small language models as perception modules
It seems that we're reinventing the brain's organs one by one from first principles. (Though Transformer + Common Crawl unintentionally builds a whole bunch of them we don't even understand yet.)
I found some broader context and the whole thing is indeed very harness-shaped:
Well, Harness is the wrong word here... "environment/tools the LLM interacts with" definitely fits though. Or "other organoid" to use the previous metaphor.
>Instead of a single transformer, we combine (i) a stack of heterogeneous DNNs paired with small language models as perception modules
It seems that we're reinventing the brain's organs one by one from first principles. (Though Transformer + Common Crawl unintentionally builds a whole bunch of them we don't even understand yet.)
I found some broader context and the whole thing is indeed very harness-shaped:
>Using Interfaze as a Tool Inside Your Agent
https://interfaze.ai/blog/using-interfaze-as-a-tool-inside-y...
Well, Harness is the wrong word here... "environment/tools the LLM interacts with" definitely fits though. Or "other organoid" to use the previous metaphor.