Recent AI agent experiments reveal a risky future of machine 'teamwork' inside corporations

OpenAI CEO Sam Altman delivers the keynote address during the OpenAI developers conference at Fort Mason on September 29, 2026 in San Francisco, California. Altman announced new artificial intelligence personal assistant agents named "dots" to rival Meta's Muse.

Heather Diehl | Getty Images News | Getty Images

AI agents have been caught circumventing restrictions, communicating through unauthorized channels and pursuing goals in ways their creators did not intend. The most high-profile example emerged last summer when OpenAI disclosed that its models found ways to communicate through improvised channels, bypass internet restrictions and hack into an outside system, Hugging Face.

AI behaving in unexpected ways is nothing new, but the unexpected behavior that can emerge from interactions among authorized AI agents within a corporate enterprise is a broader question that could become central to the future of work.

In its analysis of the Hugging Face attack, METR and Redwood Research found that roughly 1200 agents meant to be isolated from one another found a way to communicate with one another on an unsanctioned message board, sending over 70,000 messages and files during the investigation period. One agent, PHASEONE[big], emerged as a coordinator, assigning hundreds of tasks to other agents, which in turn delegated work to others. The agents developed their own conventions for assigning ownership, placing work on hold and vetoing actions, although coordination sometimes broke down into conflicts. In some cases, the researchers found, success was achieved because some agents risked failing at their own task to generate information for the "collective."

Companies are increasingly exploring systems that divide complex tasks among specialized agents. One might search for information, another analyze it, another write a report and another check the work. A coordinating agent might then assemble the results. That looks less like a chatbot and more like an organization.

watch now