
MZ
Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle, Eneko Agirre
· 1 min read
ResearcharXiv cs.CL
Do not be greedy, Think Twice: Sampling and Selection for Document-level Information Extraction
arXiv:2601.18395v3 Announce Type: replace
Abstract: Document-level Information Extraction (DocIE) aims to produce an output template with the entities, relations, and events of interest occurring in the given document. Standard practices include prompting decoder-only LLMs using greedy decoding to avoid output variability. Rather than treating this variability as a limitation, we show that sampling can produce substantially better solutions than greedy decoding, especially when using reasoning models. We thus propose ThinkTwice, a sampling and selection framework in which the LLM generates multiple candidate templates for a given document, and a selection module chooses the most suitable one. We introduce both an unsupervised method that exploits agreement across generated outputs, and a supervised selection method using reward models trained on labeled DocIE data. To address the scarcity of golden reasoning trajectories for DocIE, we propose a rejection-sampling-based method to generate silver training data that pairs output templates with reasoning traces. Our experiments show the validity of unsupervised and supervised ThinkTwice, consistently outperforming greedy baselines and the supervised state-of-the-art.
Original source
This story was published by arXiv cs.CL and written by Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle, Eneko Agirre. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


