learn
Quantization
What Quantization means, how it works, and which tools and models on this site relate to it.
Definition
Quantization stores model weights (and sometimes activations) in lower-precision numbers to reduce memory and speed up inference.
How it works
Common formats include 8-bit and 4-bit. The tradeoff is quality versus hardware fit. Always measure your actual tasks after quantizing.