A differentially private framework for gaining insights into AI chatbot use

Large language model (LLM) chatbots are used by hundreds of millions of people daily for tasks ranging from drafting emails and writing code to planning vacations and creating menus for cafes. Understanding these high-level use cases is incredibly valuable for platform providers looking to improve services or enforce safety policies. It also offers the public insights into how AI is shaping our world.

But this raises a critical question: How can we gain valuable insights when the conversations themselves might contain private or sensitive information?

Existing approaches, like the CLIO framework, attempt to solve this by using an LLM to summarize conversations while prompting it to strip out personally identifiable information (PII). While a good first step, this method relies on heuristic privacy protections. The resulting privacy guarantee is difficult to formalize and may not hold up as models evolve, making these systems difficult to maintain and audit. This limitation led us to ask if it is possible to achieve similar utility with formal, end-to-end privacy guarantees.

In our paper, "Urania: Differentially Private Insights into AI Use," presented at COLM 2025, we introduce a new framework that generates insights from LLM chatbot interactions with rigorous differential privacy (DP) guarantees. This framework uses a DP clustering algorithm and keyword extraction method to ensure that no single conversation overly influences the result (i.e., the output summaries do not reveal information about any single individual's conversation). Here we explain the algorithm and demonstrate that this framework is indeed providing better privacy guarantees than prior solutions.

Privacy-preserving framework for insights mining

DP uses a privacy budget parameter, ε, to measure the maximum allowed influence of any single user's contributions to the final output of a model. Our framework is designed to rely on two key properties of DP: