Data Authenticity Is Now The Real Currency Of AI Research

By now you’ve likely heard that Amazon closed Mechanical Turk to new customers on July 30, and the platform stopped taking on any customers on September 30. It just so happens that SageMaker’s human-in-the-loop tools, Ground Truth and Augmented AI, are also being wound down, and transaction histories disappear from the platform in January 2027.

Taken individually, these developments look like ordinary product housekeeping. Taken together, they tell a different story: A company that helped invent the market for paid human data collection, the one that made “click work” a viable way to run a study or train a model, is stepping out of that market entirely.

The easy read is that MTurk simply fell behind on quality and got outcompeted. There is truth in that. But the more interesting questions are why quality became so hard to sustain there in the first place, and the extent to which the answer says something about the whole category of online data collection.

Researchers have historically measured the online data collection category with two blunt instruments: cost per response and sample size. Platform reputation filled in the rest. Those numbers were never designed to catch a trust problem that predated AI. Survey researchers have warned for years about respondents who cycle through multiple panels chasing incentive payments. They’ve also flagged low-effort completions that clear an attention check (a question built into a survey specifically to catch people who aren’t reading closely) without any real engagement with the question being asked.

“We’re at the point where, ‘Is this data even human?’ is a legitimate methodological question researchers have to ask before they ask anything else,” said Frankie Bryant, Vice President of Prolific for Research. “Platforms that can’t answer that with certainty are a liability. Any researcher publishing findings today needs to be able to defend where their data came from, not just what it says.”