Comment Comment by vmirnv Also I would highly recommend Q5_K_M for both 7B and 13B models.It has the best balance between quality and weight of the model and almost indistinguishable from original f16: https://www.reddit.com/r/LocalLLaMA/comments/142q5k5/updated... parent vmirnv 3 years ago
It has the best balance between quality and weight of the model and almost indistinguishable from original f16: https://www.reddit.com/r/LocalLLaMA/comments/142q5k5/updated...