Comment by AdrenalinMd

But a lot of the question /responses could be trivial cached. No need to run expensive LLM every time for the same basic "how are you today?" prompts, it only has to be cached once.