/TECHStory update pending

Roofline Model Guides Optimal Draft Lengths for Speculative Decoding

Commentary on X

Highest ranked

@haozhangml great piece! I wonder if you saw our recent roofline model for SD speedups/optimal draft lengths? https://modal.com/llm-almanac/spec-dec-roofline

Charles 🎉 Frye @ ICML@charles_irlTECH

If you're interested in speculative decoding, take some time to grok this chart! And read the article from @haoailab.https://modal.com/llm-almanac/spec-dec-roofline https://x.com/charles_irl/status/2074288166576742510/photo/1 https://twitter.com/haoailab/status/2072472882014486610

Charles 🎉 Frye @ ICML@charles_irlTECH

@haozhangml @finn_fergus also fyi, in our shared interest

LLM Engineer's Almanac - Spec Dec Roofline Model (Speedup ratio)(Modal)viaCharles 🎉 Frye @ ICML

Digg Deeper

Ask a question below.

Published answers will appear here.

Related Stories

Commentary on X

Digg Deeper