DeepSeek makes DeepInfer open source to cut AI costs

DeepSeek on Tuesday made its in-house proprietary AI inference framework, DeepInfer, open source as it seeks to position itself as the open source provider of choice while appealing to cost-conscious customers.

DeepInfer runs on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs to provide offline inference. The vendor’s choice to run on hardware from Chinese chipmaker Huawei and U.S. AI giant Nvidia comes amid a push from Chinese vendors to rely less on Nvidia. By using both chipmakers, DeepSeek can appeal to its Chinese and Western customers.

The engine is an optimization layer for DeepSeek V4.1 Flash, known for its high-end agentic and coding capabilities and enables the vendor to run the model more efficiently on supported chips. The engine enables the DeepSeek models to generate responses faster and reduce compute costs.

The release of DeepInfer comes as enterprises seek to cut AI inference costs, with many AI models now powering AI agents that require large numbers of tokens, a cost that can be prohibitive.. However, the vendor’s move to open up the inference framework could help its enterprise customers limit those expenses.

Optimizing cost

“There is a lot of discussion around token optimization,” said Lian Jye Su, an analyst at Omdia, a division of Informa TechTarget. “Token consumption has been going through the roof, and the idea is that if you can turn around a task really quickly and at a low cost, it … helps to optimize your AI infrastructure investment.”

DeepInfer is helpful to enterprises because it accelerates inference processing for DeepSeek models, reducing the number of GPUs required for an inference workload.

“You can turn around your workload faster,” Su said.

While the open source engine will benefit businesses running DeepSeek on private, on-premises infrastructure, enterprises that use other models have access to other open source inference engines, such as Nvidia TensorRT-LLM, which speeds up large language models running on Nvidia GPUs.