NVFP4 decode with 16-bit activations is claimed to be up to 25% faster than a 4-bit activation baseline

humans& says its Blackwell-targeted kernel keeps weights at 4-bit precision while using 16-bit activations. It reports up to 25% faster decode than FlashInfer’s optimized NVFP4 MegaMoE, with up to 10% more generation throughput and 7% faster latency on real workloads. The company says the work is used for Ampersand’s RL training and Boba serving and has been contributed to FlashInfer, SGLang and Miles.