Comment by mtkd

I was a bit skeptical when read the initial pr on it, yesterday ran a test involving ~250M tokens, something we measure went from ~60% to >80% success (with almost no tuning) and at less than 50% cost the low-end LLM was running at, looking at it more seriously now ... the servers are US-only currently I understand and ZDR is by request