GS-Chaff: Multi-Agent Prompt-Level Semantic Chaffing for Privacy-Preserving LLM Inference

Abstract

Cloud-based large language model (LLM) services are increasingly used to process natural-language queries that may contain private or sensitive information. Conventional privacy-preserving approaches, such as cryptographic protection and text sanitization, often introduce substantial computational overhead or disrupt the semantic integrity of the original query, resulting in a trade-off between privacy protection and task utility. To address this limitation, we propose generative semantic chaffing (GS-Chaff), a training-free multi-agent framework for privacy-preserving LLM inference over natural-language text queries. Rather than explicitly masking sensitive content, GS-Chaff hides the user’s true intent among semantically plausible chaff queries. The framework is implemented through two small language model (SLM)-based agents: a privacy policy agent that adaptively determines the required semantic abstraction level and chaffing factor for each text query, and a generative semantic chaffing agent that produces semantically aligned dummy queries. After cloud-side inference, the response corresponding to the protected real query is recovered locally using a stateless index, without modifying the cloud-based LLM. Experimental results on text-based benchmarks demonstrate that GS-Chaff reduces the attacker’s real-query identification rate to 22.5%, close to random guessing, while maintaining inference utility on the evaluated benchmarks. In addition, GS-Chaff reduces local preprocessing time by 1.85× compared with a fixed chaffing configuration using β=5.