Comment by naraic0o

you mean quantized? it’s only got 16B active parameters so it can be pretty fast at decode.