Akka focuses on enterprise agentic with a focus on creating certainty and solving scale problems. We have a customer, Swiggy, which is >3M inferences per second for a blended set of models, both ML and LLMs, with a p99 latency of roughly 70ms.
This level of throughput is achieved by including memory database within the agentic process and then the clustering system automatically shards and balances memory data across nodes with end user routing built in. Combined with non-blocking ML invocations with back pressure you get the balance for performance.
This level of throughput is achieved by including memory database within the agentic process and then the clustering system automatically shards and balances memory data across nodes with end user routing built in. Combined with non-blocking ML invocations with back pressure you get the balance for performance.