AI Makes Traditional Business Controls Fashionable Again

I spent most of my career in the tech industry, helping large enterprises implement their most critical systems. In that world, releasing unsafe or unreliable products is unthinkable. Failures could trigger significant financial liabilities, possibly damaging a firm’s reputation beyond repair. Strict controls have built trust in information technology, assuring its place at the center of the modern economy. The current panic about rogue AI overlooks these basic principles. Powerful systems demand adequate oversight by those who develop and use them. Instead of a narrative of self-motivated sentient systems, the AI industry should make business controls fashionable again.

The language used to describe recent security breaches at OpenAI, Anthropic and others deflects the attention from quality and controls. To speak of agents scheming or seeking freedom is to give software the appearance of independent motives, pushing the responsibility of those who decide how to develop and release these products into the background. People give AI agents goals and they, in turn, carry out tasks over multiple steps, choosing the best path toward that goal. They get there through a sequence of mathematical optimization steps inside opaque networks. People design and execute the training and testing of these networks, or models. While their unexpected behavior raises serious engineering questions, describing it in human terms does little to explain who should have prevented the damage.

Pachocki’s metaphor of an alien mind being cultivated and grown carries responsibilities. Loving parents watch closely, set limits and step in when needed. The business equivalent requires leaders who know what their teams are testing and implement the controls to stop an experiment when needed.