Comment by Meleagris

So I did test this, and it doesn't work because the quantized layers need torch._int_mm, which PyTorch's MPS backend doesn't implement. It just throws NotImplementedError.