Breakthroughs

NVIDIA unveils NVFP4 4-bit pretraining methodology for efficient AI

NVIDIA has successfully pretrained a 12-billion-parameter Mamba-Transformer model on 10 trillion tokens using a new 4-bit precision method.