Hugging Face has scheduled a livestream for Monday, October 19, about using speculative decoding to speed up inference for local large language models.
A participant in the session reminded followers about the event on October 17 and suggested that beginners who do not want to configure the optimization themselves use Llama. The app’s product page says it enables speculative decoding automatically when a selected model supports it.
What Llama handles automatically
Llama is a free, open-source desktop app from the llama.cpp team and Hugging Face. It runs open models locally on Mac and Windows, recommends models that fit the computer and selects settings including quantization, context size, image input and batch size.
The app stores each model once in the Hugging Face cache. Its page says models load when another app requests one and unload after five minutes of inactivity, reducing idle memory use.
A local API as well as chat
Llama includes a browser-based chat interface and a local API at localhost:9931/v1. The product page says the API offers OpenAI- and Anthropic-compatible endpoints, with support for streaming, tool calling, structured output and vision.
That lets compatible coding agents, editors and other apps share the same local engine and model files. The scheduled session is focused specifically on speculative decoding and faster local inference; the attached announcement does not include benchmarks or promise a particular speed increase.
