cyberes
Joined 7 years ago
Karma
1
Submissions
16
Member since
Jun 24
Joined 7 years ago
It is so annoying that they don't give you a way to do this. I built an LLM inference API but am still working on how to cancel an inference job if a client cancels. I really don't want to rewrite my entire server in Twisted so I'll probably end up just building the HTTP responses like you did.