Skip to content

Practical Secrets Extraction against Black-box LLMs

Sep 2026 · arXiv.org · Vol abs/2609.36941 · 0 citations · 16 references
Computer Science

TL;DR

This work presents a black-box secret extraction framework for commercial, API-based LLMs under output-only access, and improves recovery effectiveness and real-key rates over representative baselines while reducing extraction latency.

Abstract

Large language models (LLMs) increasingly power autonomous coding agents such as Codex and Claude Code, yet their training corpora may contain confidential credentials exposed in public repositories or collected from private development artifacts, creating risks of memorization and subsequent leakage. Existing extraction audits, however, largely assume access to model weights or token probabilities. In this work, we present a black-box secret extraction framework for commercial, API-based LLMs under output-only access. It comprises (i) \emph{Cross-Validated Secret Knowledge Distillation}, which uses semantics-preserving prompt variants, response cross-validation, and provider-specific format filtering to distill secret-relevant behavior into a local white-box proxy; and (ii) \emph{Proxy-Guided Secret Extraction and Candidate Filtering}, which combines truncated top-$p$ sampling with local token entropy, $N$-gram frequency profiling, and provider-specific structural priors. On controlled API-key benchmarks, our framework improves recovery effectiveness and real-key rates over representative baselines while reducing extraction latency. A responsible real-world evaluation further recovers masked provider-specific credentials from three independently deployed black-box LLM systems spanning OpenAI and Claude Code, showing that memorized secrets can be exposed under output-only access.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Mitigating Private Data Leakage in LLMs with Whiteout

Modern large language models (LLMs) are trained on massive, largely unfiltered datasets, including content scraped from nearly every accessible website and user inputs. As a result, LLMs often memorize and reproduce personally sensitive information (PSI) such as birth dates, phone numbers, and home addresses. This lead...

Anna Yoo Jeong Ha, Ronik Bhaskar, Hai-Tao Zheng et al. · 0 citations
Preprint Oct 2026

Watermark Basins: Black-Box Detection of Distilled LLMs

Proprietary language models and their reasoning traces are vulnerable to API-based distillation, where an adversary trains a student model on the teacher model's outputs. To obtain evidence of distillation, model providers can watermark the teacher's outputs, aiming to leave a radioactive signal in the student's output...

Hong-Yan Chang, Hady ElSahar, Tom Sander et al. · 0 citations

Controlled Decoding Attacks on Black-Box LLMs

This work introduces \method{}, a framework for jailbreaking through text-only continuation interfaces that permit repeated sampling and assistant-prefix continuation, and achieves the highest mean score most comparisons against baselines.

Jesson Wang, Shawn Li, Wei Yang et al. · 2 citations
Open access 2026

Auditable and Robust LLM-Based Phishing Detection via Provenance-Guided Evidence Contracts

: Large language model (LLM)-based phishing detectors can produce decisions and explanations, but adversarial webpages violate the assumption that all input text is trustworthy evidence. Hidden comments, metadata, scripts, Cascading Style Sheets (CSS), and retrieved snippets can act as instruction-injection channels, w...

Yue-Kui Xu, Xiong-Zhang Luo · 1 citation
Sep 2026

Your Model Is Leaking: Covert Information Transfer through LLM Residual Streams

A residual-stream covert-channel attack is introduced that maps messages to codewords and injects them into an intermediate residual stream through a compromised runtime hook and is scaled with the local residual norm using the signal-to-residual-norm ratio.

Ming-Yuan Li, Yan-Na Jiang, Guang-Sheng Yu et al. · 0 citations
Book Open access Nov 2026

Securing Software in the Age of Large Language Models

Large Language Models (LLMs) offer new capabilities for software security by interpreting source code and natural-language specifications, and generating plausible code. However, their probabilistic behavior, hallucinations, limited security knowledge, and inconsistent outputs make their direct use non-trivial. This di...

Varun Gadey · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.