Letting the kernel use SSD based swap space for something this big would be a good way to destroy its cumulative write endurance over a period of just a couple months. I would be very interested in seeing SMART self reported drive cumulative write and wear out stats if this was done for more than a short test.
In my experience llama-server is better run with --no-mmap on things that will fit entirely into RAM. Though obviously you need a 2TB server for full Kimi k3 and 1M context.
In my experience llama-server is better run with --no-mmap on things that will fit entirely into RAM. Though obviously you need a 2TB server for full Kimi k3 and 1M context.