DeepSeek Releases Open Source Inference Engine to Cut AI Costs

DeepSeek released its DeepInfer inference engine as open-source software on October 20, 2026, giving developers access to technology designed to reduce the computing costs and energy consumption associated with running artificial intelligence models. The Hangzhou-based company said the release supports its V4.1-Flash model and provides compatibility with NVIDIA GPUs and Huawei Ascend accelerators, increasing pressure on AI providers to make their services more affordable.

According to DeepSeek's announcement, the engine is available under the Apache 2.0 license without additional commercial revenue restrictions. The company's DeepSeek-V4.1-Flash-1017 model weights are distributed under the MIT license, offering developers separate licensing terms for the model and its inference software.

The release comes as businesses reconsider the cost of operating advanced AI systems. Gizmodo reported that Accenture had instructed many employees to limit their monthly spending on Claude to $29, illustrating how rising AI bills are prompting organizations to control usage.

DeepSeek described its objective in a post on X: "We're sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable." The company also said future models would be incorporated into the framework as they become publicly available.

DeepInfer includes several components used in DeepSeek's inference infrastructure, including FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect and custom TileLang kernels. The repository also features confidence-scheduled decoding, bounded replay, a batch scheduler designed to avoid additional overhead, and structured output capabilities.

According to Forkast News, the same codebase supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. Developers can switch between accelerator platforms using the DEEPINFER_ACCELERATOR environment variable, allowing the software to target hardware from both manufacturers.

However, DeepInfer does not provide every component needed to operate a complete production inference service. Its repository identifies key omissions, including a key-value cache system, separation of prefill and decode processing, and load balancing across data-parallel replicas. Organizations deploying the engine must develop or obtain those capabilities separately.

DeepSeek's documentation acknowledges that the project was "originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM." Those open-source projects provide infrastructure that developers may use to address some of the components missing from DeepInfer.

A companion project, deepseek-recipe, provides Rust and Python components for converting API requests into DeepSeek prompts and parsing model outputs. Its documentation specifies that developers must supply the model inference, tool execution and HTTP transport themselves.

Performance claims remain subject to independent verification. DeepSeek describes the engine as offering "high throughput and low latency," while its repository includes benchmark charts based on four SM100 GPUs for prefill and 64 GPUs for decoding within a single scale-up domain. Forkast News reported that the repository did not publish numerical speed improvements and that independent benchmarks were not yet available.

Pricing comparisons further illustrate the competitive pressure. According to OpenRouter figures cited by Gizmodo, DeepSeek V4.1-Flash costs $0.035 per million input tokens and $0.12 per million output tokens. OpenAI's GPT-6 Astra was listed at $10 per million input tokens and $50 per million output tokens.

Inference, the process through which trained models generate responses, has become an increasingly significant expense as AI systems handle more complex tasks and coordinate multiple agents. A McKinsey & Company report cited by Gizmodo projected that most data center computing capacity would be devoted to inference rather than model training by the end of the decade.

The release also coincided with new US restrictions on government use of Chinese AI technology. Gizmodo reported that the Trump administration had barred federal civilian agencies, including the State Department, from using DeepSeek AI tools on October 19, 2026. The restriction did not prohibit private companies or other organizations from using the technology.