NVFP4 vs FP8 vs GGUF on a 16 GB Ada GPU: NVFP4 decodes 2.3x faster and ties BF16 on GSM8K

Automated excerpt

When you type «nvfp4 vs» into Google it completes with «fp8», «gguf», «q4_k_m» and «bf16». On October 16 NVIDIA released Nemotron Labs Dory 1. 0 4B in BF16, NVFP4, MXFP8 and GGUF at once, so I measured the same model in every format on an RTX 4070 Ti SUPER (Ada, 16 GB, no FP4 hardware), with the rules written down before the first run. Quality, full GSM8K (1319 questions, 8192-token cap): BF16 1249, FP8 1241, MXFP8 1250, NVFP4 1238.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.