Comment by SimFG

At present, we have started applying GPT cache to online products. According to existing data, the effect is good. In fact, it can also be found that some LLM products actually have caching capabilities, such as Github Copilot. However, the cache required by different products varies, and the caching mechanism is worth noting.