Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimation

arXiv:2605.11840v2 Announce Type: replace Abstract: Radar camera depth estimation turns a very sparse but metric radar signal into a dense depth map. Existing methods combine radar and image features outside the sequence operator of the backbone, through concatenation, confidence gating, sparse supervision, or graph based extraction. Mamba variants for multimodal fusion also leave the selection mechanism itself unimodal. We argue that selection is the right place for radar to enter. We introduce Radar-Modulated Selection (RMS), which injects radar inside the selective scan of Mamba. Radar adds learned perturbations, initialised at zero, to the step size $\Delta$ and the readout $\mathbf{C}$. The input projection $\mathbf{B}$ and the state matrix $\mathbf{A}$ never see radar. At initialisation the block equals a pretrained image Mamba, so radar gains influence only where it improves accuracy. The construction also couples the two modalities at linear cost and falls back to the image backbone when radar is missing. We deploy RMS in a Multi-View Scan Pyramid (MVSP) that matches the fusion operator to the reach of radar at each scale. On nuScenes, SemoDepth reduces MAE by 34.0%, 29.9%, and 29.9% over the previous best within 50, 70, and 80 m. It also has the lowest single frame latency among methods with a public release (26.8 ms). A feature blend placed before the scan adds no accuracy on top of RMS, which supports the view that selection inside the scan can replace fusion outside it. When twice as many spurious radar returns as true ones are injected, the MAE@80 of SemoDepth rises by 1.4%, against 33.6% for a Mamba baseline that fuses radar before the scan.