
YL
Yi Lu, Qianshi Lin, Yumeng Yang, Hongxia Wang
· 1 min read
ResearcharXiv cs.CL
Measuring and Charging for Retained Context in Long-Context LLM Services
arXiv:2611.24071v1 Announce Type: cross
Abstract: Long-context large language model (LLM) services charge for repeated context, yet billing weights need not reflect its contribution to answer quality. We present a three-layer framework separating content relevance, observable proxies and auditable positional weights. Context-ablation experiments show that relevance dominates position and that distance-decay families do not outperform a relevance-only baseline. A measured serving stack shows that cache hits reduce repeated-prefix compute cost by over two orders of magnitude at 65,536 tokens, while cold resends constrain sustainable discounts. Cross-provider measurements separate cache value from tokenisation and list-price differences. Calibrated simulations reduce the 99th percentile of session charges by 31-42%, although revenue-matched nonlinear tariffs achieve similar reductions and median charges increase. The framework provides a bounded per-token accounting rule and measured deployment constraints, rather than a validated utility estimator.
Original source
This story was published by arXiv cs.CL and written by Yi Lu, Qianshi Lin, Yumeng Yang, Hongxia Wang. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


