
JS
Johann Schmidt, Tom Siegl, Martin Becker, Sebastian Stober
· 1 min read
ResearcharXiv cs.CV
Geometrically Constrained and Token-Based Probabilistic Spatial Transformers
arXiv:2509.11218v3 Announce Type: replace
Abstract: Spatial transformations such as rotation and scale obscure the morphological cues needed for accurate image classification. Careful consideration is required for reliable use in high stakes settings. A model should stay robust under such transformations, expose why a correction was applied, and signal when its input is ambiguous. While geometrically equivariant architectures provide a mathematically grounded solution, they often limit model flexibility through strict symmetry constraints and incur significant computational overhead. Spatial Transformer Networks (STNs) offer a data-driven, flexible alternative for learning pseudo-equivariances to affine transformations. However, STNs have historically been restricted to convolutional architectures and suffer from training instability. To address this, we introduce a novel STN framework. It leverages the global modeling capabilities of transformers to regress the affine transformation acting on the input. For this, we decompose affine transformations into interpretable primitives, regressed under adaptable geometric constraints, thereby preventing the training instability typically caused by degenerate transformations. By sharing weights between the localization network and the classification backbone, the framework requires minimal computational overhead. Extensive experiments on challenging insect biodiversity and medical imaging benchmarks demonstrate that our approach achieves superior predictive performance under diverse spatial transformations while maintaining high efficiency. Code is available at https://github.com/johSchm/TokenSTN.
Original source
This story was published by arXiv cs.CV and written by Johann Schmidt, Tom Siegl, Martin Becker, Sebastian Stober. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


