Comment Comment by thelastparadise How is the $ per inference, say 4k tokens, on a Hetzner box vs an A100?Not all tasks require low latency. parent thelastparadise 3 years ago
Not all tasks require low latency.