Comment by cschneid

Data Point: I am currently having issues getting Mixtral Q4_K_M running in LMStudio on my 32gb M1 Max. I'm trying Q3 to see if it fits.

I can have it run in on 'cpu' which is very slow, but offloading to the GPU runs out of memory.