SyncAI.news, a Varaisys broadcasting
The Scaling Properties of Implicit Deductive Reasoning in Transformers
EV

Enrico Vompa, Tanel Tammet

· 1 min read

ResearcharXiv cs.AI

The Scaling Properties of Implicit Deductive Reasoning in Transformers

arXiv:2605.04330v3 Announce Type: replace Abstract: We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers. By discouraging the reliance on statistical shortcuts via counterfactual data augmentation, and promoting the learning of shared reasoning primitives across direct and CoT modes, we find that in sufficiently deep models with a bidirectional prefix mask, implicit reasoning approaches explicit CoT performance across graph topologies and problem widths, though CoT remains necessary for depth extrapolation. These findings represent a step toward achieving better compositional reasoning in Transformers. The code and models to reproduce this work are available at: https://github.com/envomp/Implicit-Deductive-Reasoning-in-Transformers

Original source

This story was published by arXiv cs.AI and written by Enrico Vompa, Tanel Tammet. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News