SyncAI.news, a Varaisys broadcasting
Representational Change Under Self-Consuming Training: A Sparse Autoencoder Analysis
SG

Suchit Gupte, Zhongteng Cai, Xueru Zhang, Mohammad Mahdi Khalili

· 1 min read

ResearcharXiv cs.AI

Representational Change Under Self-Consuming Training: A Sparse Autoencoder Analysis

arXiv:2611.21348v1 Announce Type: cross Abstract: Large language models are increasingly exposed to model-generated text, raising the possibility that future models will be trained recursively on the outputs of their predecessors. Prior work on such self-consuming training loops has largely focused on output quality and downstream performance, leaving their effects on internal representations poorly understood. We study how representations evolve across five generations of self-consumption and whether sparse autoencoders (SAEs) remain reliable under this shift. At each generation, we compare a lineage trained on synthetic data with a matched control trained on real data under the same training settings. We evaluate how well SAEs trained before self-consumption transfer to later generations, measuring reconstruction and sparsity across SAE widths, architectures, and model layers. We then fine-tune the original SAE on each generation and use sparse probing to test whether task-relevant information remains concentrated in a small number of latents or becomes more distributed across the representation. We find that zero-shot SAE performance degrades progressively under self-consumption, with growing reconstruction and sparsity gaps that vary substantially across layers. Fine-tuning largely restores reconstruction, but not feature-level usefulness: task-relevant information remains accessible in the full latent space while becoming less concentrated in individual features. These results show that self-consumption induces representational drift and that reconstruction fidelity alone is insufficient to establish that an adapted SAE remains equally useful for interpretation.

Original source

This story was published by arXiv cs.AI and written by Suchit Gupte, Zhongteng Cai, Xueru Zhang, Mohammad Mahdi Khalili. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News