What was released
DeepSeek has published DeepInfer, the inference framework it says powers its own large-scale production services, under the Apache 2.0 licence. The repository describes the release as providing “offline inference with the core features needed for serving at scale”.
Inference is the unglamorous half of running a language model: the serving software that turns a trained set of weights into an API that answers requests quickly and cheaply. Labs rarely open it. DeepSeek has now opened a version of its own, and the detail that matters is which version.
One model, two accelerators
DeepInfer supports exactly one model — DeepSeek-V4.1-Flash — and downloads the DeepSeek-V4.1-Flash-1017 checkpoint by default. That is the model DeepSeek swapped in behind its own Flash API earlier this month.
The hardware list is the more interesting line. DeepInfer runs on Nvidia SM100 GPUs and on Huawei Ascend 950 NPUs, and the build script takes nvidia or ascend as its target. The prerequisites differ accordingly: CUDA 13 for the Nvidia path, Huawei’s CANN 9.2 for the Ascend path. A Chinese lab shipping first-class support for a domestic accelerator alongside Nvidia’s newest is a statement about where it expects its users to be running.
What is deliberately missing
The repository is unusually candid about the gap between this and what DeepSeek runs internally. Under a heading for what is “required for a complete inference service (not included in this release)”, it lists three items: a KV cache system, prefill/decode disaggregation — for which it says “most required engine functionality is already implemented” — and load balancing across data-parallel replicas for both prefill and decode.
Those are not footnotes. They are the layers that turn an engine into a service, and they are where a lot of the cost advantage in a large deployment sits. What has been released is the part that executes; the part that routes, caches and balances has not been.
The numbers that are not there
DeepInfer’s README links prefill and decode charts and states the conditions — “measured on 4 SM100 GPUs for prefill and 64 for decode — each within one scale-up domain” — but the page carries no throughput or latency figures in text. The listed feature set is specific: attention data parallelism, balanced context parallelism, mixture-of-experts expert parallelism with load balancing, asynchronous scheduling, batch-invariant inference, deterministic sampling and structured outputs.
DeepSeek also credits its sources. The acknowledgements say DeepInfer “was originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM” — so this is a hardened fork of the open-source serving stack the rest of the industry already uses, not a separate lineage.
Timing
The release landed on the same day the US Department of Homeland Security’s order appeared in the Federal Register, excluding DeepSeek products from federal civilian procurement and requiring their removal from civilian agency IT — including third-party products “developed using DeepSeek implementations”. The notice was filed on 19 October and published on 20 October.
The Decoder, which first reported the DeepInfer release, says DeepSeek framed the goal as helping the community lower compute costs and energy use, and that a companion library, deepseek-recipe, reproduces the token handling of DeepSeek’s official API so self-hosted deployments behave the same way.
What to watch
Two things. Whether DeepSeek releases the missing cache, routing and balancing layers, which would turn DeepInfer from an engine into a deployable stack. And whether anyone outside DeepSeek publishes reproduced numbers on Ascend 950 hardware — the repository ships a sweep script for exactly that, and no independent result has appeared yet. By the morning of 21 October the repository had about 14,400 stars, 1,400 forks and 124 open pull requests, with two contributors listed.
