I Saw the Future of Robotics in a Virtual Duck That Chases My Cat

Allow me to introduce you to the robot cat butler I’m developing with help from AI.

Using OpenAI’s GPT-6 Sol, I taught a virtual version of Microduck—a cute robot from Pollen Robotics and Hugging Face—to walk around a room and identify any bengal cats in its vicinity. (The clip above shows the robot waddling toward a target feline; its eye view is shown top right.)

I’m still waiting patiently for a physical Microduck—please hurry up and ship it, guys—but simulated behavior is relatively simple, so it should transfer over to the real thing. I plan to have the robot automatically dispense a treat for my pet bengal, Leona, perhaps by bowing its head to release one from a 3D-printed, head-mounted storage chamber.

I was able to code and train the virtual robot in just a few hours, which is, quite frankly, incredible. Previously, this kind of thing might have taken me weeks to figure out. It was possible because OpenAI’s latest models are remarkably good at understanding and manipulating 3D representations of the physical world and then translating that into useful code, agentic actions, and robot training.

Real roboticists are using the latest models to do some amazing stuff. When OpenAI released Astra this past September, University of California, Berkeley professor Ken Goldberg and his students saw demos that showcased its ability to render 3D images. They wondered if it could help with the simulations required to train different kinds of robots. These simulations need to be incredibly detailed and extremely accurate to ensure that sophisticated behavior—a humanoid that can dance or a robot hand that can grasp—will transfer faithfully to the real world.

One member of Goldberg’s lab gave Astra access to an Nvidia tool called Newton, showed it a video clip of a real robot in action, and asked if it could create a simulation of the scene.

This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.