Comment by bjornsing

Sounds a bit unexpected from an information theoretical point of view: you’ve seemingly managed to remove this knowledge from the full 32 bit representation of the model, but when you compress it down to 4 bit the knowledge reappears. Makes you wonder what information was actually lost in the compression / quantization step…