SyncAI.news, a Varaisys broadcasting
Measuring and Charging for Retained Context in Long-Context LLM Services
YL

Yi Lu, Qianshi Lin, Yumeng Yang, Hongxia Wang

· 1 min read

ResearcharXiv cs.CL

Measuring and Charging for Retained Context in Long-Context LLM Services

arXiv:2611.24071v1 Announce Type: cross Abstract: Long-context large language model (LLM) services charge for repeated context, yet billing weights need not reflect its contribution to answer quality. We present a three-layer framework separating content relevance, observable proxies and auditable positional weights. Context-ablation experiments show that relevance dominates position and that distance-decay families do not outperform a relevance-only baseline. A measured serving stack shows that cache hits reduce repeated-prefix compute cost by over two orders of magnitude at 65,536 tokens, while cold resends constrain sustainable discounts. Cross-provider measurements separate cache value from tokenisation and list-price differences. Calibrated simulations reduce the 99th percentile of session charges by 31-42%, although revenue-matched nonlinear tariffs achieve similar reductions and median charges increase. The framework provides a bounded per-token accounting rule and measured deployment constraints, rather than a validated utility estimator.

Original source

This story was published by arXiv cs.CL and written by Yi Lu, Qianshi Lin, Yumeng Yang, Hongxia Wang. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News