JasonWang1027

I'm excited to share with you a new open-source project called langport, which aims to provide a lightning-fast large language model serving platform.

Inspired by lmsys/fastchat, we've built a distributed LLM serving system. Our focus on performance means we use batch inference to get higher throughput on the serving platform. Langport offers a range of core features, including streaming API interface support, batch inference for higher throughput, and OpenAI-Compatible RESTful APIs.

The placed block isn't synchronized to MongoDB immediately. Chunks data is stored in memory, while some players nearby.

It needs quite fast network between distributed servers in clusters. A distributed database, MongoDB, is used to store Minecraft terrain and entities.

The readme doesn't show these. It needs to be improved.