Comment Comment by nico Is there a limit for the number of tokens that can be fed to the models when run locally?OpenAI limits prompts to 4096 tokens.If there was no limit, could the LLM be fed a 100 page document in the prompt and then answer questions about it? parent nico 3 years ago
OpenAI limits prompts to 4096 tokens.
If there was no limit, could the LLM be fed a 100 page document in the prompt and then answer questions about it?