Can anyone explain why Cerberus needs to be _fast_ instead of _cheap_?
I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s.
Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.
I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s.
Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.