The U.S. says China's AI progress is down to 'distillation.' But is it that clear cut?

This report is from this week's The Tech Download newsletter. Like what you see? You can subscribe here.

In the battle for AI supremacy between the U.S. and China, "distillation" has become a hot topic.

The term broadly refers to the training of an AI model using the output or answers from another more advanced model. In some instances the process has been deemed controversial because a less advanced lab could use the product of a model that has taken millions or billions of dollars to create to design a competitive product.

As China has continued to pump out highly advanced models which compete in some areas with their American rivals, U.S. companies and Washington have accused Chinese labs of using distillation to gain ground, denouncing their efforts as theft.

But Aidan Gomez, the CEO of Cohere, says not all of China's progress is down to this technique.

watch now

Gomez, who spoke to me on the latest episode of The Tech Download, is an authoritative figure in AI. He was a co-author of a 2017 research paper called "Attention Is All You Need," which underpins many of the AI services we use today such as ChatGPT and Claude.

Gomez described Chinese AI models as "world class," adding that the lead U.S. labs have is "evaporating very quickly."

"There was like a lot of talk about the Chinese are just copying, they're just distilling, they're cheating. A lot of these tropes were applied, and that was definitely true to an extent," Gomez told me.

"However, they have developed an exceptional capability independent of distillation, and it's proven by the fact that the latest models that are coming out, actually on some benchmarks, on some axis capabilities, beat the best American models. And you can't copy or distill to better. You can close the gap and reduce the gap by copying, but you can't outperform."

This runs slightly counter to what has been said by U.S. AI labs and Washington.

The rise of Chinese AI

Distillation doubts

watch now