Robot company bets touch, sound can give bots an edge over vision AI

Germany’s Neura Robotics has unveiled a foundation model to help robots make decisions using touch, sound and other sensory inputs alongside vision, in a bid to improve the machines’ perception capabilities.

The new model, Neuron AI, combines 3D vision, tactile and force data, audio, robot state and language in a perception-language-action model. Neura said the design also incorporates “memory,” enabling robots to retain information about objects and previous actions even when those objects are no longer visible.

Neura is planning a live demonstration of Neuron AI at Nvidia’s GTC conference in Berlin later this week. At the demo, Neura’s humanoid robot 4NE1 will perform an eleven-minute laundry-handling sequence, trained using the new model.

The approach differs from traditional physical AI systems that primarily rely on vision-language-action (VLA) models, which use visual information to learn to perform tasks.

According to Neura, Neuron AI uses three sources of data to train robots: real-world robot interactions, simulated tasks and environments, and human demonstrations captured with sensor-equipped suits.

The company collects the sensory and movement data at its Neura Gyms, which provide infrastructure for robot training models.

Yet while combining multiple sensory inputs could help robots handle tasks that cameras alone cannot, the approach poses a training challenge: collecting sufficient suitable data to teach models to understand and respond to those inputs.

“Most physical AI stacks are focused on VLA models, which learn from ‘seeing,’” said Rueben Scriven, senior research manager at industrial automation and robotics research market firm Interact Analysis. “What Neura is betting on is that models that can be trained on different sensory inputs will perform better.”

The approach could also complement Neura’s vertically integrated sensor stack, potentially giving the company an advantage over competitors.