An Introduction to Quantization

Zipping Neural Networks

Standard AI models use extreme mathematical precision. Quantization is akin to saving a high-res image as a JPEG. The image still looks the same from afar, but uses a fraction of the disk space. This is what makes local AI feasible on normal workstations.