Solving math word problems

We’ve trained a system that solves grade school math problems with nearly twice the accuracy of a fine-tuned GPT‑3 model. It solves about 90% as many problems as real kids: a small sample of 9-12 year olds scored 60% on a test from our dataset, while our system scored 55% on those same problems.

Why it matters

This is important because today’s AI is still quite weak at commonsense multistep reasoning, which is easy even for grade school kids. We achieved these results by training our model to recognize its mistakes, so that it can try repeatedly until it finds a solution that works.

Introduction

Large language models like GPT‑3 have many impressive skills, including their ability to imitate many writing styles, and their extensive factual knowledge. However, they struggle to perform tasks that require accurate multistep reasoning, like solving grade school math word problems. Although the model can mimic the cadence of correct solutions, it regularly produces critical errors in logic.

To match human performance in complex logical domains, our models must learn to recognize their mistakes and to choose their steps carefully. To that end, we train verifiers to evaluate whether or not a proposed solution is correct. To solve a new problem, we use verifiers to select the best among many proposed solutions. We collected the new GSM8K dataset to evaluate our methods, and we are releasing this dataset to facilitate research.

In the ten examples below, we show solutions generated by our new method, verification, and our baseline method, fine-tuning.

GSM8K dataset

Solutions in GSM8K are written as natural language rather than as pure math expressions. By sticking to natural language, model-generated solutions are more readily interpretable by humans, and our methods remain relatively domain agnostic.

Training verifiers: Models that learn from their mistakes

Conclusion

• Transformers
• Language
• Reasonings & Policy

Authors

Karl Cobbe, Vineet Kosaraju, John Schulman