russianGuy83829
Joined 5 years ago
Karma
245
Submissions
112
Member since
May 14
Joined 5 years ago
I just saw this simple patch to enable MTP (potentially 2x performance) on older GPUs (Kepler etc), so maybe it will work for you
https://github.com/ggml-org/llama.cpp/pull/25680
Also, for Qwen, the 4 bit _XL quantization seems to have a good balance of performance to size.
archived versions here: http://archive.today/8LX8s http://archive.today/00mIw
https://duckduckgo.com/?q=https%3A%2F%2Fservury.com%2Fdatace...
It is not visible in the live webpage.