TellTail shows that retrieval-based systems leak the identity of their embedding model---exposing intellectual property and enabling reconnaissance ahead of model-specific attacks such as corpus poisoning.
Abstract
Dense embedding models are core to modern text retrieval, enabling systems ranging from web search to retrieval-augmented generation (RAG). Yet, retrievers are usually deployed within opaque systems, exposing only ranked results, cited sources, or generated answers. This opacity prevents users from verifying which retrievers providers serve and may create a false sense of robustness against retrieval attacks. We show that attackers can infer retrievers'identity only through queries. We introduce TellTail, a fingerprinting attack for identifying retrievers behind black-box systems across a spectrum of access levels. Despite sharing many properties, retrievers can be steered to emit distinct results. When retrieved passages are exposed, TellTail compares retrieval overlap to deduce the underlying model. When only the final generated response is exposed, TellTail optimizes model-specific queries that induce a chosen retrieval behavior on the target retriever but transfer poorly to others. Interestingly, the poor transferability that limits attacks elsewhere is exactly what makes them useful for fingerprinting. We evaluate TellTail using 53 retrievers under three increasingly restrictive settings. TellTail perfectly identifies the deployed retriever from full retrieval rankings; in 94.3% of attempts with unordered top-3 results; and in 92.5% of cases from language-model-generated answers alone. Furthermore, fingerprinting succeeds against a popular RAG system (OpenWebUI). Overall, TellTail shows that retrieval-based systems leak the identity of their embedding model---exposing intellectual property and enabling reconnaissance ahead of model-specific attacks such as corpus poisoning.
Retrieval-Augmented Generation (RAG) enables large language models to access external knowledge during inference, but its reliance on external knowledge bases also introduces new attack surfaces. Existing data extraction attacks against RAG systems rely on either prompt injection, which heavily depends on the generator...
Yuchen Chen, Wei Cheng, Yuan Xiao et al.· 0 citations
Modern Information Retrieval (IR) systems, including small and medium-scale ones, often search across millions of documents. In many cases, data owners outsource retrieval to a third-party host to keep the data accessible. Furthermore, IR has also moved toward semantic matching with dense embeddings; although these emb...
F. D. De Faveri, Guglielmo Faggioli, Nicola Ferro· Proceedings of the 35th ACM...· 0 citations
Consistency-based defenses for retrieval-augmented generation (RAG) rest on an assumption that a sparse adversary breaks: a poisoning campaign may modify a negligible fraction of the corpus, yet retrieval concentrates its passages into the reader's context. Across four Natural Questions retrieval seeds, at least one in...
The results suggest that private dense retrieval is already practical for moderately sized, privacy-sensitive corpora when minute-scale latency is acceptable, and formulates a two-round cryptographic protocol that provides both guarantees.
Louis Tremblay Thibault, Sofiane Azogagh, Marc-Olivier Killijian et al.· arXiv.org· 1 citation
Vector databases require embedding geometry for similarity search, but that geometry can also support inference. We evaluate keyed orthogonal obfuscation against a snapshot adversary. The practical Walsh–Hadamard construction has no established confidentiality guarantee; its empirical evaluation covers only the tested...
Ionuț Cernica, Nirvana Popescu· Applied Sciences· 0 citations
Open-weight large language models (LLMs) can be copied, modified, and redeployed behind black-box APIs, making post-release ownership verification difficult. Existing black-box fingerprints often rely on secret query-key pairs that reproduce predefined responses, and can therefore be easily disrupted by fine-tuning, pr...