This work presents a black-box secret extraction framework for commercial, API-based LLMs under output-only access, and improves recovery effectiveness and real-key rates over representative baselines while reducing extraction latency.
Abstract
Large language models (LLMs) increasingly power autonomous coding agents such as Codex and Claude Code, yet their training corpora may contain confidential credentials exposed in public repositories or collected from private development artifacts, creating risks of memorization and subsequent leakage. Existing extraction audits, however, largely assume access to model weights or token probabilities. In this work, we present a black-box secret extraction framework for commercial, API-based LLMs under output-only access. It comprises (i) \emph{Cross-Validated Secret Knowledge Distillation}, which uses semantics-preserving prompt variants, response cross-validation, and provider-specific format filtering to distill secret-relevant behavior into a local white-box proxy; and (ii) \emph{Proxy-Guided Secret Extraction and Candidate Filtering}, which combines truncated top-$p$ sampling with local token entropy, $N$-gram frequency profiling, and provider-specific structural priors. On controlled API-key benchmarks, our framework improves recovery effectiveness and real-key rates over representative baselines while reducing extraction latency. A responsible real-world evaluation further recovers masked provider-specific credentials from three independently deployed black-box LLM systems spanning OpenAI and Claude Code, showing that memorized secrets can be exposed under output-only access.
Modern large language models (LLMs) are trained on massive, largely unfiltered datasets, including content scraped from nearly every accessible website and user inputs. As a result, LLMs often memorize and reproduce personally sensitive information (PSI) such as birth dates, phone numbers, and home addresses. This lead...
Anna Yoo Jeong Ha, Ronik Bhaskar, Hai-Tao Zheng et al.· 0 citations
Proprietary language models and their reasoning traces are vulnerable to API-based distillation, where an adversary trains a student model on the teacher model's outputs. To obtain evidence of distillation, model providers can watermark the teacher's outputs, aiming to leave a radioactive signal in the student's output...
Hong-Yan Chang, Hady ElSahar, Tom Sander et al.· 0 citations
This work introduces \method{}, a framework for jailbreaking through text-only continuation interfaces that permit repeated sampling and assistant-prefix continuation, and achieves the highest mean score most comparisons against baselines.
Jesson Wang, Shawn Li, Wei Yang et al.· arXiv.org· 2 citations
: Large language model (LLM)-based phishing detectors can produce decisions and explanations, but adversarial webpages violate the assumption that all input text is trustworthy evidence. Hidden comments, metadata, scripts, Cascading Style Sheets (CSS), and retrieved snippets can act as instruction-injection channels, w...
Yue-Kui Xu, Xiong-Zhang Luo· Computers, Materials & C...· 1 citation
A residual-stream covert-channel attack is introduced that maps messages to codewords and injects them into an intermediate residual stream through a compromised runtime hook and is scaled with the local residual norm using the signal-to-residual-norm ratio.
Large Language Models (LLMs) offer new capabilities for software security by interpreting source code and natural-language specifications, and generating plausible code. However, their probabilistic behavior, hallucinations, limited security knowledge, and inconsistent outputs make their direct use non-trivial. This di...
Varun Gadey· Proceedings of the 2026 ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.