SyncAI.news, a Varaisys broadcasting
Authors' Response to AI-Based Reviews: A Mixed-Methods Pilot Study
CL

C\'esar Leblanc, Lukas Picek

· 1 min read

ResearcharXiv cs.AI

Authors' Response to AI-Based Reviews: A Mixed-Methods Pilot Study

arXiv:2605.16623v2 Announce Type: replace-cross Abstract: Background: Large language models are increasingly used to assist scholarly peer review, but there is limited evidence about how authors judge their feedback and use it to revise their papers. Methods: We studied authors' experiences of receiving an auxiliary AI review alongside human reviews in three computer-science cohorts: LifeCLEF 2025, LifeCLEF 2026, and CVWW 2026. Post-review questionnaires yielded 92 responses concerning 71 papers. We summarized closed-ended answers using descriptive statistics and examined free-text responses through thematic analysis. Results: Most responses rated the AI review useful (82/92 (89%), and 82/92 (89%) reported using at least some of its suggestions in revision. Authors reported that AI identified issues missed by human reviewers ((76/92 (83%). However, 44/92 (48%) trusted AI feedback less than human feedback, and 58/92 (63%) reported incorrect or misleading comments. Authors valued suggestions that improved clarity and reproducibility, but also described overlooked evidence, inappropriate demands, and difficulty finding useful comments within lengthy reviews. They emphasized the need to check AI feedback and retain control over how their manuscripts were processed. Future review by both humans and AI was supported by 85/92 (92%); 83/92 (90%) preferred advance notice and 73/92 (79%) supported explicit consent. Conclusions: Authors often found AI reviews useful for revision without trusting them as much as human reviews. These exploratory findings favor a supplementary role for AI, with human oversight, clear disclosure, and author consent. They do not establish that AI can replace human reviewers or improve manuscript quality.

Original source

This story was published by arXiv cs.AI and written by C\'esar Leblanc, Lukas Picek. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News