How NuminaMath Won the 1st AIMO Progress Prize

This year, Numina and Hugging Face collaborated to compete in the 1st Progress Prize of the AI Math Olympiad (AIMO). This competition involved fine-tuning open LLMs to solve difficult math problems that high school students use to train for the International Math Olympiad. We’re excited to share that our model — NuminaMath 7B TIR — was the winner and managed to solve 29 out of 50 problems on the private test set 🥳!

In this blog post, we introduce the Numina initiative and the technical details behind our winning solution. If you want to skip straight to testing out the model with your hardest math problems, check out our demo.

Let’s dive in!

1. Introducing Numina - an open AI4Maths initiative
2. The AI Math Olympiad (AIMO) prize
3. Our winning solution for the 1st progress prize
4. The training recipe
5. Good data is all you need
6. Taming the variance with Self-Consistency and Tool Integrated Reasoning (SC-TIR)
7. Avoiding the curse of overfitting
8. Other ideas we tried
9. Numina’s future - looking for contributors and partners!
10. Acknowledgements

Introducing Numina - an open AI4Maths initiative

There is something very special about mathematics.

Mathematics is a domain accessible to everyone, even to children long before they can read. One of the greatest mathematicians of all time Srinivasa Ramanujan was self-taught, being born into a modest family in India in 1887. While for others it varies from recreation to profession and everything in between.

Mathematics is essential to humanity, being the backbone upon which we have built everything from commerce to iPhones and nuclear power plants. Yet even solving maths problems for a critical application can be a playful experience.

Pure mathematics transcends intelligence like an endless ocean only the mind can sail.

With the team in place, it was time to tackle the AIMO challenge!

The AI Math Olympiad (AIMO) prize

Given these constraints and rules, strategic choices were essential to develop our winning solution.