Comment Comment by orph Hono still doesn’t support request cancellation well.So if you’re streaming tokens from an LLM and you cancel the request from the client you’ll be wasting money. parent orph 2 years ago
So if you’re streaming tokens from an LLM and you cancel the request from the client you’ll be wasting money.