Chinese lab DeepSeek published its production inference framework DeepInfer as open source on GitHub on October 20, 2026. Released under the Apache 2.0 license, the system is a high-performance framework powering the company's large-scale production services, and the initial release covers offline inference for the DeepSeek-V4.1-Flash model. According to a Bloomberg report on October 21, the release reveals how DeepSeek keeps artificial intelligence (AI) inference costs in check and gives cloud and hosting companies running DeepSeek models a way to cut their own costs.
What DeepInfer Does
DeepInfer is a high-performance inference engine built to run large language models in production. DeepSeek describes it as the backbone of its large-scale production services, and says the goal of opening the stack is to help the community lower compute costs, reduce energy use, and make AI more affordable. The company announced the release on October 20 on X:
“We're sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.” — DeepSeek, X (2026-yil 20-oktabr)
Open sourcing means any cloud provider or hosting company can now run the optimized inference layer that DeepSeek uses in its own services on their own infrastructure. Bloomberg notes that this is exactly what makes the release significant for the market: the technical mechanism behind the company's low-price policy is now open to everyone.
Technical Stack: Kernels, MoE, and Speculative Decoding
The initial release focuses on offline inference for DeepSeek-V4.1-Flash and includes several layers of optimization. First, optimized kernels — the model's compute operations are accelerated at the hardware level. Second, attention and expert parallelism are supported: the release includes EP (expert parallelism) and EPLB (expert load balancing) mechanisms for the mixture-of-experts (MoE) architecture.
Third, there is DSpark, a speculative decoding system — a technique that speeds up the model's response generation without sacrificing quality. Fourth, structured outputs are supported: the ability to get the model's responses in a predefined format (for example, a JSON schema) matters for production integrations. The official repository on GitHub has gathered more than 16,000 stars — a strong early sign of interest in open-source AI infrastructure.
NVIDIA and Huawei Chips Supported
DeepInfer supports two hardware platforms: NVIDIA's SM100-architecture GPUs and Huawei's Ascend 950 neural processors (NPUs). This is especially noteworthy because for Huawei it is the first production-grade support from a leading AI lab.
In practice, this means DeepInfer can be run not only in the NVIDIA ecosystem but also on Huawei chips. As reliance on domestic chips grows in the Chinese market amid US export restrictions, DeepSeek's treatment of the Ascend 950 as a first-class citizen is being seen as an important step toward technological independence.
Why It Matters: Costs and an Open Ecosystem
DeepSeek's market strategy has been built on low prices from the start: the company offers APIs and model licenses far cheaper than competitors, sharply driving down the cost of AI services. Open-sourcing DeepInfer exposes the technical foundation of that affordability — Bloomberg emphasizes that the release discloses how the company controls its inference costs.
The open framework has impact in two directions. On one hand, cloud and hosting companies offering DeepSeek models to their customers can now use the same optimizations to cut their operating costs. On the other, an open-source, high-performance inference engine raises compute efficiency across the entire AI ecosystem — a pressing issue in an industry where energy consumption and infrastructure costs keep rising.
As a reminder, the DeepSeek-V4.1-Flash model itself was updated on October 17, as we reported earlier. DeepInfer's release shows the company is now handing its infrastructure layer — alongside its model releases — to the open ecosystem.
The choice of the Apache 2.0 license is also an important detail: the license permits commercial use, so cloud and hosting companies can integrate DeepInfer into their products without friction. The three goals in DeepSeek's X announcement — lowering compute costs, reducing energy use, and making AI more affordable — are now being pursued through a tool that belongs not just to the company but to the whole community. This interest confirms that demand for open-source AI infrastructure keeps growing.
