SyncAI.news, a Varaisys broadcasting
Vestrum: Improving Agent Harnesses by Adapting Their Verification, Structure and Memory
JP

Jayant Parashar, Eugene F. Douglass, William C. Bastian, Suchendra M. Bhandarkar

· 1 min read

ResearcharXiv cs.AI

Vestrum: Improving Agent Harnesses by Adapting Their Verification, Structure and Memory

arXiv:2609.33822v1 Announce Type: new Abstract: An agent harness controls how a language model accesses information, uses tools, preserves memory, and checks its work. Improving this software is costly when each evaluation requires a long interaction with an environment. We introduce Vestrum, a framework that turns failures in execution traces into scoped harness changes without training the task model. Its organizing overhypothesis is that tasks of a shared kind may exhibit recurring failures whose remedies transfer within that kind. Vestrum expresses failures as recognizable classes, proposes changes across verification, retrieval, decomposition, and knowledge synthesis, and screens their scope before evaluating them as a bundle. A persistent lessons file informs subsequent proposals. Across five settings and two baseline harnesses, the frozen harnesses improve held-out performance: UltraHorizon rises from 47.6 to 59.8 over GAM, Terminal-Bench 4 Hard from 63.7% to 70.3% of checks passed over Claude Code on eight held-out tasks at 1.03x test cost, and cell-type annotation agreement from 67.5% to 77.8% on held-out sections of one slide, alongside gains on LoCoMo and AMA-Bench. Across our searches, verification grounded in evidence helped both intermediate steps and final answers, at lower cost at intermediate steps, while critics asked to rebuild finished answers broke more than they repaired. On the three memory benchmarks, Vestrum also scores above the evaluated GEPA configurations in every paired evaluation.

Original source

This story was published by arXiv cs.AI and written by Jayant Parashar, Eugene F. Douglass, William C. Bastian, Suchendra M. Bhandarkar. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News