SyncAI.news, a Varaisys broadcasting
PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval
BK

Bin Kang, Jiarui Ouyang, Li Jiang, Bin Chen, Zhuotao Tian

· 1 min read

ResearcharXiv cs.AI

PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval

arXiv:2609.36923v1 Announce Type: new Abstract: Existing reactive Graphical User Interface (GUI) agents often fail in long-horizon, dynamic scenarios, where unexpected disturbances trigger attention-diverting and cascading failures. To address this, we propose PrecogUI, a pre-cognitive architecture that shifts the paradigm from reactive execution to proactive decision-making. Specifically, we design a Proactive Experience Pool (PEP), which caches recurring anomaly and success patterns as "state-action-result" tuples in a dual-memory repository. Furthermore, we introduce a Proactive Simulation Executor (PSE) that learns to forecast the next symbolic UI layout given a candidate action, enabling early anomaly avoidance and ranking candidate actions by predicted reliability. Finally, a Pre-cognitive Execution Controller (PEC) fuses these priors and predictions, prioritizes handling of foreseen anomalies, and ensures execution robustness through a closed-loop error correction mechanism. For robust evaluation, we develop AutoTraj, an automatic data-generation engine, to construct InterfereBench, a benchmark for long-horizon tasks with strong disturbances. Experiments demonstrate that PrecogUI surpasses state-of-the-art methods on InterfereBench while maintaining competitive performance on public benchmarks. The code will be publicly available.

Original source

This story was published by arXiv cs.AI and written by Bin Kang, Jiarui Ouyang, Li Jiang, Bin Chen, Zhuotao Tian. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News