Slowing Down AI Won't Save The Companies Already Wired To It

Tom Shea, CEO, OneStream Software.

Recently, the people building the most powerful AI models told us that AI could end humanity if we are not careful. Anthropic’s Dario Amodei called for the industry to slow the development of frontier systems. Sam Altman and Elon Musk backed him.

They are right to raise the alarm on catastrophic risk, but their proposed remedy—slowing down the frontier labs—misses where the most immediate danger lies. The reality is the risk is already here, and organizations everywhere are amplifying it by connecting today’s models to the systems that run their businesses.

Think about an LLM as the frontal lobe of the human brain. By itself, it can reason, but it cannot accomplish much in the physical world.

Now connect that frontal lobe to a nervous system and a body. Through Model Context Protocol (MCP), APIs, agentic frameworks, databases, enterprise applications, operating systems, industrial systems and eventually physical machines, we have already been giving AI models the ability to observe, decide and act, changing the risk equation completely.

The right way to think about AI risk is in three distinct categories—errors of commission, errors of omission and errors of ignorance—and they require a unique control system to minimize that risk.

Errors Of Commission

Errors of commission involve the intentional misuse of AI. Hackers, criminal organizations, hostile governments and other bad actors can use LLMs to dramatically increase their technical capabilities.

This is where the current regulatory conversation runs into a structural problem. Most of the proposals have been designed to govern how frontier labs build and release their models. That is reasonable, but it assumes the danger is contained inside the companies being regulated.

Errors Of Omission

That is the pattern that should concern us most, not a model becoming more capable, but a capable model being handed authority faster than the controls around it could keep pace.