Comment by tarruda

Small models retain much less of the knowledge they were trained on, especially when quantized.

One good use case for 32gb Mac is being able to run 8b models at full precision, something that is not possible with 8-16gb macs