I wasn't talking about latent space, I was talking about the literal text a person types into the text box in their favourite AI tool.
If the prompt is sufficiently short and often repeated, it just makes me wonder - what is cheaper for a provider over a given timespan? Cache a large enough set of responses and randomly return one of them, or generate it each time time?
If the prompt is sufficiently short and often repeated, it just makes me wonder - what is cheaper for a provider over a given timespan? Cache a large enough set of responses and randomly return one of them, or generate it each time time?