SyncAI.news, a Varaisys broadcasting
Aperture: Merge-Consistent Rotary States for Compressed Tokens
YD

Yuhao Du, Shunian Chen

· 1 min read

ResearcharXiv cs.AI

Aperture: Merge-Consistent Rotary States for Compressed Tokens

arXiv:2609.36781v1 Announce Type: new Abstract: Token compression combines content from several positions, yet rotary position embeddings usually assign the merged token one coordinate. We ask what positional information must survive later merges. Aperture stores Fourier moments of the token's weighted support at the model's rotary frequencies. We prove that these moments have minimal real dimension among continuous states sufficient for the selected expected rotary interactions. Represented mass makes updates additive; attention normalisation remains a separate readout choice. Uniform intervals give a centre rotation times a sinc gain. We characterise when centres determine interval widths and construct matched examples where they do not. Numerical checks verify the weighted-support implementation. In trained temporal readers, compression transfer varies with gain calibration and feature placement. In a prespecified native video question-answering comparison, stored support reaches $65.63\%$ accuracy versus $67.12\%$ for the deployed merging rule. These results separate exact positional preservation under compression from downstream benefit.

Original source

This story was published by arXiv cs.AI and written by Yuhao Du, Shunian Chen. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News