Quantization

Researchers benchmark instruction-tuned LLMs using FP8, GPTQ, and SmoothQuant

FP8 quantization slashes the memory footprint of 70B-class open-weight models, maintaining accuracy within 0.