Report
QDA hands AI quantization's recipe book to an agent, MIT and NVIDIA researchers say
QDA’s researchers report candidate scoring in minutes, helping an agent tune recipes for lower-precision language models.
TLDR
MIT and NVIDIA researchers say QDA builds and tests recipes for quantization—running language models at lower precision. The team reports beating the best public recipe on 11 of 12 model-precision pairs, with searches using 64 GPU-hours each. The paper describes scoring candidates in minutes, then checking promising ones with long generations for repetition loops.
Combined views
7.1K
3 Sources, first seen ago

Comments