Comment Comment by stevenhuang Anyone who has used local LLMs will come to this conclusion.Too much context confuses the LLM. Reduce unnecessary context and performance improves. Not to mention, too much context and you just run out of vram.Such limitations may be mitigated in future architectures, but it certainly isn't something you can ignore currently. parent stevenhuang 1 day ago
Too much context confuses the LLM. Reduce unnecessary context and performance improves. Not to mention, too much context and you just run out of vram.
Such limitations may be mitigated in future architectures, but it certainly isn't something you can ignore currently.