Hugging Face shares a livestream on speculative decoding for faster local LLMs
vLLM says making speculative decoding easier to put into practice benefits the inference community.
TLDR
Hugging Face shared a livestream about using speculative decoding to get multipliers in local LLM speed-ups. vLLM described it as a practical look at the technique and said making it easier to put into practice benefits the inference community.
Combined views
2.4K
2 Sources, first seen ago
31 likes1 comments9 saves4 reposts
