DeepSeek announced the open-source release of DeepInfer on Oct. 20, 2026. The company describes it as the inference framework powering DeepSeek-V4.1-Flash in its production services. DeepSeek says the framework delivers high throughput and low latency at scale, and that sharing it is meant to help the community lower compute costs and energy use.
What's in the release
The DeepInfer README lists DeepSeek-V4.1-Flash as its supported model. It describes optimized kernels, asynchronous scheduling, batch-invariant inference, deterministic sampling and structured outputs. DeepSeek's announcement thread also lists distributed inference and speculative decoding, with plans to add inference code for future models. The README names NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs as supported hardware.
DeepSeek also open-sourced deepseek-recipe, a separate token encoding and decoding component it says is intended to bring the official DeepSeek API's processing behavior to community deployments. Its README describes Rust libraries and Python bindings for converting requests, rendering prompts, encoding tokens and converting model output into responses. The caller must provide model inference, tool execution and HTTP transport.
What a complete service still needs
The DeepInfer README says this release provides offline inference, not all the systems needed for a complete inference service. It lists a KV cache system, VE/prefill/decode disaggregation and load balancing across prefill and decode replicas as required but not included. The README adds that most of the engine functionality needed for disaggregation is already implemented.
Comments