ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021.
Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog OpenAI. The example in the post about getting new information about the Webb space telescope seems to imply Google has this advantage.
So my question is, how hard of a problem is that with LLMs? I get the sense that LLMs are trained on a very large data set all at once, but that it is difficult to incrementally update them with new data. Is that a true assessment? Even incrementally updating an inverse text index can be a scalability challenge, and so it seems like, given the way that LLMs are trained, that it would be even harder to do with LLMs without spending an absolute fortune on training.
ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021.
Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog OpenAI. The example in the post about getting new information about the Webb space telescope seems to imply Google has this advantage.
So my question is, how hard of a problem is that with LLMs? I get the sense that LLMs are trained on a very large data set all at once, but that it is difficult to incrementally update them with new data. Is that a true assessment? Even incrementally updating an inverse text index can be a scalability challenge, and so it seems like, given the way that LLMs are trained, that it would be even harder to do with LLMs without spending an absolute fortune on training.