Creating agent and human collaboration with GPT 4o

Dr. Robert Yang has spent half his life building AI inspired by the brain. 

In 2023, when OpenAI’s language model became broadly available, Yang quit his job as an assistant professor at MIT to start Altera.AL⁠(opens in a new window), a research lab focused on building what they call “digital humans”: a new way for people to interact with agents that will have fundamental human qualities.

Yang, now Altera’s CEO, envisions a future where AI agents don’t just assist; soon, he believes, they’ll interact and collaborate with humans, and even experience emotions. Along with his three co-founders, Dr. Andrew Ahn, Nico Christie, and Shuying Luo, Yang has built Altera’s first product with GPT‑4o: the first autonomous agents that can play Minecraft⁠(opens in a new window) with you, just like a friend.

Tackling data degradation for long-term autonomy

Just as automation helps increase human capacity by supporting repetitive tasks, digital humans—the Altera team believes—will be able to collaborate productively and even form bonds with people. This may look like:

• Digital “coworkers” who can collaborate for days or weeks to proactively solve problems 

• Long-duration multi-agent worlds where we can measure responses to economic policies, advertisements, and more

But Altera ran into unique challenges as they set out to build. The most pressing issue was the phenomenon of data degradation, a problem that plagues all AI models making autonomous decisions in extended timeframes. 

“AI agents interact with the world, making decisions in real-time, but as their own output becomes their future input, the data quality degrades over time,” Yang explains. “This is an issue most AI Agent systems encounter, but for our digital humans, who are meant to live autonomously for hours or even longer, this becomes one of the most pressing issues to solve.”

Harnessing GPT-4 to enhance cognitive and emotional AI

Dr. Robert Yang, CEO of Altera