Comment by mg

I wonder how a decentralized, hierarchical LLM would perform.

For example:

    LLM A is trained on all of Wikipedia
    LLM B is trained on all of Hacker News
    LLM C is trained on all of Project Gutenberg

User asks question Q on webservice W.

W sends Q to A and B.

Then W sends a question to C "Hey C, I have a user who asked Q. Here is A's reply and B's reply. Given those, how would you answer Q?"

Would the answer be as good as or better than what an LLM which is trained on Wikipedia, Hacker News and Project Gutenberg would return?

If it is of similar quality, then we could build a hierarchical tree of consumer hardware LLMs which are hosted all over the world.