Engineers Open-Source Speculative Decoding Tutorial and Harness

Madison Kanna, an engineer at Baseten, posted that she and @lily_gpupoor built an interactive tutorial on speculative decoding for the NeurIPS Education Track. They open-sourced the tutorial, the site, and the LosslessBench harness. Kanna stated that speculative decoding is one of the most important optimization techniques in LLM inference. The post presents the release as their own work on the education track submission.