One way to increase the context window, I thought, would be to teach the LLM a compressed language based on abbreviations etc and to have some compressing/uncompressing script do the translating with the LLM. That would allow longer prompts too.
Not as sophisticated as this LLMLingua but good enough for basic users.
One way to increase the context window, I thought, would be to teach the LLM a compressed language based on abbreviations etc and to have some compressing/uncompressing script do the translating with the LLM. That would allow longer prompts too.
Not as sophisticated as this LLMLingua but good enough for basic users.
[0] https://en.wikipedia.org/wiki/Speedtalk