Comment

Comment by danielhanchen

Hey we did not remove the MTP for sizes above 8GiB - but yes for small GGUFs under 8 ish GiB, we removed the MTP module (IQ2_XXS and lower), because it's 500MiB to 750MiB in size, and on small 8 GiB machines, even 500MiB is needed.

As someone in the comments said we made a separate Q4_0 MTP if that's helpful so you can use that.

But I would suggest using UD-IQ3_XXS for 10.9GB for 16GB machines or Q2_K_XL

parent danielhanchen 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton