AI models are getting better at grade school math — but a new study suggests they may be cheating

Adobe Firefly
(Image credit: Adobe Firefly 3/Future generated AI image)

Large language models (LLMs) that power chatbots like ChatGPT may be getting better at answering benchmark questions that measure mathematical reasoning. But this may actually be a bad thing.

A pre-print research paper released on Wednesday by researchers at Scale AI detailed how LLMs have been achieving impressive results on math benchmark tests but that there’s growing concern that dataset contamination is fuelling high grades. 

This is when data resembling benchmark questions leaks into training data. The LLM then may end up training in a way that prioritizes passing these standardized tests rather than truly understanding the mathematical problem it’s trying to solve.

This is similar to if you’re preparing for a math quiz by memorizing answers, rather than learning how to solve the problem. This issue is called overfitting.

However, the authors of the paper say their results don’t support this theory, suggesting that it doesn't mean the AI is bad at reasoning, just that it might not be as good as the benchmarks suggest..

Developing a new math benchmark

In the paper the authors wrote: “The fact that a model is overfit does not mean that it is poor at reasoning, merely that it is not as good as the benchmarks might indicate it to be." They found that many of the most overfit models can still reason and solve problems they’ve never encountered before in their training sets.

To run these evaluations, they developed their own math benchmark test (GSM1k) which they say tests the AIs ability to understand the problem, not just the answer.

The fact that a model is overfit does not mean that it is poor at reasoning, merely that it is not as good as the benchmarks might indicate it to be.

Study authors

The questions are at grade school math level and a typical GSM1k question would look like: Jim wants to spend 15% of his monthly earnings on groceries. He makes $2500 a month. How much money will he have left over? The correct answer is $2125.

While such questions closely resemble those in the industry gold standard test (GSM8k) in difficulty, they're different enough to test whether the LLMs can handle math puzzles they haven’t seen before.

Using their new test, the research team at Scale AI reported accuracy drops of up to 13% when they evaluated leading open- and closed-source LLMs. Other models on the frontier such as Gemini, GPT, and Claude showed minimal signs of overfitting.

What's next?

This ‘issue’ may end up resolving itself over time as the authors predict that by 2025 grade school math will likely no longer be difficult enough to benchmark new LLMs. Still, they say that improving reasoning in LLMs “is one of the most important directions of current research.”

Senior Research Scientist at NVIDIA Jim Fan said on X that academic benchmarks are losing their potency.

He said that three types of LLM evaluations that will matter in the future would be privately held tests like that of Scale AI, public comparative benchmarks like Chatbot Arena where you can test models side-by-side, and privately curated benchmarks for each company’s own use cases.

More from Tom's Guide

Back to MacBook Air
Storage Size
Screen Size
Any Price
Showing 10 of 96 deals
Load more deals
Christoph Schwaiger

Christoph Schwaiger is a journalist who mainly covers technology, science, and current affairs. His stories have appeared in Tom's Guide, New Scientist, Live Science, and other established publications. Always up for joining a good discussion, Christoph enjoys speaking at events or to other journalists and has appeared on LBC and Times Radio among other outlets. He believes in giving back to the community and has served on different consultative councils. He was also a National President for Junior Chamber International (JCI), a global organization founded in the USA. You can follow him on Twitter @cschwaigermt.