
QH
Qinhan Hou, Jing Tang
· 1 min read
ResearcharXiv cs.CL
Many Solutions, One Answer: Search Order Shapes the Default Solutions of Language Models
arXiv:2611.25916v1 Announce Type: new
Abstract: When a constraint puzzle has several correct answers, language models tend to give the same one.We study four open-weight models from three families on symbolic puzzles and multi-solution ZebraLogic variants with exactly known solution sets.Repeated samples concentrate on one default solution far beyond uniform sampling, and different models often share it.Yet models usually produce even rarely sampled solutions when added clues uniquely identify them.On held-out symbolic puzzles, we compare visiting entities in name order with visiting those with the fewest initial candidates first. Both searches try values in the stated order. Where their pre-specified predictions differ, the latter matches a larger share of responses in all four models, by 26 to 40 percentage points.Reversing the stated value order or specifying a search order shifts responses towards the solution that the new order favours, on both testbeds. Randomising the presentation or supplying a random search order raises breadth (the effective share of solutions in use) by 26 to 42 percentage points on the symbolic task and 11 to 26 on ZebraLogic. The largest observed validity losses are 3 and 4 percentage points, respectively. Randomised presentation broadens answers more than a random-choice instruction in each of the three main models.
Original source
This story was published by arXiv cs.CL and written by Qinhan Hou, Jing Tang. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


