SyncAI.news, a Varaisys broadcasting
TRACER: Trace-Regularized Training Against Indirect Prompt Injection
KM

Kshitij Mishra, Salem Lahlou, Nils Lukas

· 1 min read

ResearcharXiv cs.LG

TRACER: Trace-Regularized Training Against Indirect Prompt Injection

arXiv:2610.19133v1 Announce Type: new Abstract: Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior. We show that defense robustness is channel-conditioned: the same attack succeeds 18% of the time as a follow-up message but 53% when placed in retrieved context. We propose TRACER (Trace-Regularized Adversarial Channel Exposure with Reward-shaping), a channel-aware adversarial training method that samples presentation wrappers and combines an answer-anchored, evidence-gated reward with ATRC, which aligns attacked reasoning with clean evidence when an injection is invalid. On GSM8K with Qwen2.5-3B-Instruct, TRACER reduces RAG-framed ASR from 65.5% to 6.1% while preserving 78-82% clean accuracy. Joint RAG+tool training achieves 13.9% average ASR across framings at 3B scale, while a single epoch at 7B reaches 9.4%. These results demonstrate that robust defenses must explicitly train and evaluate across deployment channels.

Original source

This story was published by arXiv cs.LG and written by Kshitij Mishra, Salem Lahlou, Nils Lukas. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News