Comment by mappu

Standard llama.cpp can mmap the gguf, so it'll stay on disk if it doesn't fit on memory, and the kernel page cache will ensure the hot parts ("resident trunk") stay resident.

What's the benefit of a custom implementation at all?