SyncAI.news, a Varaisys broadcasting
Twice-Told Critiques: Claim-Level Homogenization of Peer Review in the LLM Era
IG

Ilya Gusev

· 1 min read

ResearcharXiv cs.CL

Twice-Told Critiques: Claim-Level Homogenization of Peer Review in the LLM Era

arXiv:2611.26286v1 Announce Type: new Abstract: Peer review relies on several readers finding different problems. We measure critique overlap in conference and journal reviews. At ICLR, measured overlap rose from 12.0% in 2026 to 21.3% in 2027, in reviews from the same stage of the review process, with year-on-year rises at NeurIPS, ICML and the F1000 and PLOS journals. A third ICLR review of a paper added 5.46 new problems to the other two in 2027, against 6.12 in 2026. We apply MELD, a detector of LLM-generated text, to all reviews. Within a paper, two reviews that MELD flags overlap more than two unflagged reviews: by 3.0 points at ICLR in 2024-2025, by 4.5 points in 2026-2027, and by 5.4 points across five journal sources. These results provide evidence that LLM-flagged reviews of one paper are less distinct from each other than unflagged reviews, and suggest that the diversity of critiques among reviews of a single paper is falling.

Original source

This story was published by arXiv cs.CL and written by Ilya Gusev. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News