SyncAI.news, a Varaisys broadcasting
When the Cheapest Feasible Router Fails: Auditing Deployment Commitments in Cost-Aware LLM Routing
MH

Minxuan Hu, Qishi Zhan, Ziheng Chen, Zhengtao Yao, Zihan Dong, Jiayu Yi, Wenxi Sun

· 1 min read

ResearcharXiv cs.LG

When the Cheapest Feasible Router Fails: Auditing Deployment Commitments in Cost-Aware LLM Routing

arXiv:2611.20417v1 Announce Type: new Abstract: Cost-aware LLM routing often selects the cheapest policy meeting empirical risk and coverage constraints. Yet a router that passes an audit can violate its fixed service contract on new requests, even when candidate policies are constructed independently. We study this deployment-commitment decision with disjoint construction, audit, and test samples. Across 19 RouterBench task families, the cheapest audit-feasible router deploys in 91.82% of opportunities but violates the held-out contract in 32.22% of deployments. We introduce Menu-Aware Routing Commitment (MARC), which qualifies policies using audit evidence and a serving-cost tolerance to retain the apparent winner, deploy a supported alternative, or defer. Its practical policywise variant (MARC-P) reduces observed conditional failure to 0.73% at 85.79% deployment, with higher serving cost; fixed-margin and formal risk-control procedures provide competitive alternatives. Nested audits show how added evidence changes deployable policy sets, and learned semantic routing exhibits the same commitment failure. These results establish deployment commitment as a distinct operational decision connecting router evaluation, service reliability, and inference cost.

Original source

This story was published by arXiv cs.LG and written by Minxuan Hu, Qishi Zhan, Ziheng Chen, Zhengtao Yao, Zihan Dong, Jiayu Yi, Wenxi Sun. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News