Comment by GreenWatermelon

yeah they mention:

> Required for a complete inference service (not included in this release):

> A KV cache system

> VE/Prefill/Decode disaggregation (most required engine functionality is already implemented)

> Load balancing across data-parallel replicas for both prefill and decode