Open R1: Update #4

Welcome DeepSeek-V3 0324

This week, a new model from DeepSeek silently landed on the Hub. It’s an updated version of DeepSeek-V3, the base model underlying the R1 reasoning model. There isn’t much information shared yet on this new model, but we do know a few things!

What we know so far

The model has the same architecture as the original DeepSeek-V3 and now also comes with an MIT license, while the previous V3 model had a custom model license. The focus of this model release was on improving the instruction following as well as code and math capabilities. Let’s have a look!

How good is it?

The DeepSeek team has evaluated the model on a range of math and coding tasks and we can see the model’s strong capabilities compared to other frontier models:

Clearly, the model plays in the top league: often on par with GPT-4.5 and generally stronger than Claude-Sonnet-3.7.

To summarise the model has seen significant improvements across benchmarks

MMLU-Pro: 75.9 → 81.2 (+5.3) (A good benchmark for overall understanding)
GPQA: 59.1 → 68.4 (+9.3)
AIME: 39.6 → 59.4 (+19.8) (proxy for MATH capabilities)
LiveCodeBench: 39.2 → 49.2 (+10.0) (indicator of coding abilities)

Specifically, in the model card the DeepSeek mentions targeted improvements in the following areas:

Front-End Web Development
- Improved executability of the code
- More aesthetically pleasing web pages and game front-ends
Chinese Writing Proficiency
- Enhanced style and content quality
  - Aligned with the R1 writing style
  - Better quality in medium-to-long-form writing
- Feature Enhancements
  - Improved mutli-turn interactive rewriting
  - Optimized translation quality and letter writing
Chinese Search Capabilities
- Enhanced report analysis requests with more detailed outputs
Function Calling Improvements
- Increased accuracy in Function Calling, fixing issues in previous V3 versions