This Startup Wants To Catch AI’s Mistakes

A flight recorder for wayward AI systems. The space industry’s record investments. AI spending warnings. All that and more in this week’s Prototype. To get it in your inbox, sign up here.

As an experiment, I built an AI agent trained on my taste that, every Friday, goes through the week’s new album releases and reviews to recommend new music to me. It doesn’t stop me from seeking out human reviews on my own, but I’ve found it to be a nice jumping-off point. (It recommends long reads about music to me, too.) Today, I was excited to see albums that were not on my radar at all from some of my favorite bands. New music from The Molotovs, The Lemon Twigs and the Sheepdogs all in one week? It sounded too good to be true… because it was. My entire briefing this week was completely made up because it experienced an error trying to search the web, couldn’t find new albums, and so made a bunch up based on its training data.

All that is a long way of saying: An AI system can be subject to drift, error and attack, but will often try to keep doing what it’s supposed to do anyway. In music recommendation, an error really only brings disappointment. But for autonomous drones, cars and factories, that kind of failure can be fatal.

That’s where Ashley Reeves comes in. He’s the CEO and founder of South Korea-based ArbaLabs, which is developing AI verification hardware that can record what embedded physical AI systems are actually doing–versus what they think they are doing. “We look at it as a kind of ‘black box’ for physical AI,” he told me, referring to the flight recorders found on planes that can be used to determine what happened in an accident.

“The goal of this company is to be a standardized de facto layer for the entire industry,” he said.

Got questions? Comments? Story tips? Human-generated music recommendations? Email me at: theprototype@forbes.com