>while having way less diversity of answers to that question than humans would.
Why is this ironic? The actual number of different LLMs is tiny. And a lot of those are just distilled from SOTA models.
>The weights are initialized randomly and in theory the labs should have quite some divergence in datasets by now due to different scraping,
I think the opposite, the models actually converge over time. They ingest FAR more data than any human ever could in countless lifetimes. Any one particular bit of data may only have a part in a trillion of effect. From this data you start to reach means and statistical averages across all models.
Now, I don't always believe it will be this way. As training models get cheaper and faster we will see unique LLMs start to diverge and be used more often.
Why is this ironic? The actual number of different LLMs is tiny. And a lot of those are just distilled from SOTA models.
>The weights are initialized randomly and in theory the labs should have quite some divergence in datasets by now due to different scraping,
I think the opposite, the models actually converge over time. They ingest FAR more data than any human ever could in countless lifetimes. Any one particular bit of data may only have a part in a trillion of effect. From this data you start to reach means and statistical averages across all models.
Now, I don't always believe it will be this way. As training models get cheaper and faster we will see unique LLMs start to diverge and be used more often.