On the Privacy of LLMs: An Ablation Study

arXiv:2605.02255v2 Announce Type: replace-cross Abstract: Large language models (LLMs) face diverse privacy threats, yet existing studies typically evaluate individual attacks in isolation, offering limited guidance on which system factors drive privacy vulnerabilities. This paper presents a unified threat model and a systematic ablation study of four representative attacks: Membership Inference (MIA), Attribute Inference (AIA), Data Extraction (DEA), and Backdoor Attacks (BA). Our findings reveal attack-specific sensitivities: membership inference depends strongly on dataset characteristics and attack strategy, attribute inference is influenced by model and training characteristics, data extraction increases with model scale and training-data repetition, and backdoor attacks achieve consistently high success rates. Based on these findings, we derive practical recommendations for privacy-aware LLM deployment, including prioritizing membership leakage assessments in retrieval-augmented systems, minimizing repeated personal information in training data, monitoring attribute inference risks, and safeguarding fine-tuning pipelines against backdoors.