DeepSeek open-sources the inference engine behind V4.1-Flash

DeepSeek has released DeepInfer, the software it uses to run its own language models in production, as open source. The code is licensed under Apache 2.0 and can be used, modified and built into other systems. The release lets organisations run DeepSeek's model on their own hardware with the same engine DeepSeek uses, but the package is not a complete service.

DeepSeek announced the release from its official X account on 20 October at 09:30 UTC. The company says DeepInfer powers DeepSeek-V4.1-Flash at scale with high throughput and low latency, and that it is sharing its production stack to help others lower compute costs and reduce energy use.

What DeepSeek has made available

The source code sits in a public repository under DeepSeek's GitHub organisation. According to the documentation, this release provides offline inference with the core features needed for serving at scale. Inference is the work a model does when it answers requests, and it is where the running cost of AI in production builds up.

The documentation lists optimised compute kernels, several forms of parallelism across many GPUs, and a scheduler that overlaps CPU and accelerator work. DeepInfer can also return identical output for identical input under fixed settings, and constrain output to a set format such as a JSON schema. DeepSeek says the identical-output feature works on NVIDIA only.

DeepSeek states that DeepInfer was originally built primarily on the open vLLM project, with inspiration from SGLang and NVIDIA's TensorRT-LLM.

Narrow support and heavy requirements

The release supports one model: DeepSeek-V4.1-Flash. It supports two kinds of hardware: NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. It requires Linux, Python 3.12 and either NVIDIA's CUDA 13 or CANN 9.2 for Ascend.

DeepSeek itself notes that three parts needed for a complete inference service are not included: a KV cache system for the model's working memory, disaggregation of the different phases of generation onto separate machines, and load balancing across multiple copies of the model. The performance figures in the documentation were measured on 4 GPUs for the first phase and 64 for generation itself. They are shown only as charts, without figures in the text.

What DeepSeek does not say

The main documentation page gives no cost figures, no commercial support offering and no timeline for the missing parts. Nor does it say whether more models or other hardware will follow. How much an organisation actually saves therefore depends on its own hardware, its own operations and how much engineering work is needed to fill the gaps.

What it means for business leaders

Assessment: The release lowers the bar for running DeepSeek's model in an organisation's own data centre or with a European hosting provider, instead of sending data to DeepSeek's own services. That matters most where data cannot leave the organisation's control.

• Anyone considering self-hosting open models should calculate the full cost: hardware in this class, power, and the skills to build what DeepSeek has not included. The package is a building block for an inference platform, not a finished service.
• Support for Huawei Ascend alongside NVIDIA shows DeepSeek is building for two parallel hardware tracks. Assessment: for procurement, the choice of model and the choice of hardware are increasingly linked.
• Open code under Apache 2.0 can be reviewed by in-house staff or a third party. Organisations with their own policies on Chinese suppliers should assess both the model and the code against them before adopting either.

Sources and media

• DeepSeek on X: DeepInfer announcement, published 20 October 2026 at 09:30 UTC
• DeepSeek: DeepInfer on GitHub, with documentation and Apache 2.0 licence
• Sources were checked on 20 October 2026.
• Illustration: AI-generated with Grok Bot / hogby.ai