A comparative breakdown of Activation-aware Weight Quantization vs Generalized Post-Training Quantization.
Quantizing weights to 4-bit representation can degrade reasoning capabilities on critical tasks. AWQ (Activation-aware Weight Quantization) mitigates this by identifying and protecting the top 1% features / channels that contain the most critical weights.
"By keeping core activation channels unquantized while mapping secondary parameters to 4-bit, AWQ models retain near-FP16 perplexity scores."
— Open Weights Coalition
GPTQ, on the other hand, relies on continuous second-order error optimization during inverse Hessian calculations. We analyze the runtime decompression latency of AWQ vs GPTQ to show how AWQ shines for real-time edge streaming.