Asynchronous Robot Inference: Decoupling Action Prediction and Execution

TL;DR Robotic policies are increasingly bulky, and predict chunks of future actions rather than a single next action. This results in the robot being idle while awaiting new actions to perform, introducing noticeable lags at execution, and lacking responsiveness. Asynchronous inference tightens the control loop, removing lags at runtime and resulting in more adaptive control by decoupling action prediction from action execution. In this blog post, we cover the basics behind async inference, and how it can be used to improve the performance of robotic policies in the real-world.

Table of Contents

• Getting started
• Async inference: a deep dive
• 1. Why sequential inference falls short
• 2. Asynchronous inference, in a nutshell
• 3. System Architecture
  • Robot Client
  • Policy Server
• 4. Analyzing async inference
• 5. Using async in your setup
• Conclusions

Getting started

Get started with async inference by following our tutorial.

Sequential inference (first) versus async inference (second). Allowing for replanning and a tighter control loop, async inference results in (1) attempts at recovery, and (2) a ~2x speedup in task completion. Sequential inference keeps acting out the current action chunk even after failure to grasp the object, while async inference can replan and act the new action chunk. Both setups use the same policy!

Async inference: a deep dive

With async inference, we decouple action execution from action prediction. This is particularly relevant considering the tendency of currently popular models like [ACT], [OpenVLA], [PI0], and [SmolVLA] to be outputting chunks of actions at:t+Ha_{t:t+H} rather than single actions ata_t given an observation oto_t. Convince yourself of this by running all these models using LeRobot.

In particular, we design a 2-component system where policy inference and action execution are performed in two different processes, possibly on two different machines connected through the network: