From Multiplicity to Vulnerability: Privacy Amplification Risk from One-Dataset-Multiple-Model Exposure

Privacy-preserving methodologies against privacy attacks on deep learning: a survey

Training Dataset Exposure Through Adversarial Model Query Analysis: A Black-Box Privacy Breach Framework

This paper presents a systematic framework for membership inference attacks, in which an adversary with only black-box query access to a deployed classifier determines whether a specific individual's record was part of its training set.

Predicting Privacy Leakage from Weight Spectral Density

Membership inference attacks (MIAs) are widely used to audit the privacy disclosure risk of machine learning models, however current state-of-the-art attacks require training computationally expensive shadow models, making large-scale privacy evaluation impractical. In this work, we investigate whether inexpensive spectral metrics derived from the heavy-tailed self-regularisation framework can serve as proxies for MIA vulnerability. We evaluate several WeightWatcher spectral metrics on image and tabular classification tasks and compare their relationship with MIA privacy leakage against conventional measures of generalisation. Across datasets, stable rank exhibits a strong positive correlation with overall MIA success, while Log alpha-Norm shows a consistent negative correlation with MIA vulnerability at the low false-positive regime. These associations are observed to be stronger than those obtained using the generalisation gap. The results indicate that neural network spectra may contain information about privacy leakage that is not fully captured by conventional measures of overfitting, motivating spectral analysis as a promising direction for scalable privacy auditing.

Embedding-Space Anonymization for Privacy-Preserving AI Systems

This paper studies embedding-space privacy as a representation-level learning problem. Rather than altering raw records directly, the proposed framework applies embeddingspace transformation to full-record representations through Gaussian perturbation and adversarial representation sanitization. The method is evaluated through ablation across utility metrics, linkage attacks, attribute-inference attacks, and membership-inference tests. The primary empirical evaluation uses a synthetic fusion recommendation benchmark built from MovieLens [1], [2] 32M behavior and Adult-derived demographics [3], while a secondary synthetic medical benchmark is used to examine cross-domain transferability under more constrained conditions. The strongest results appear in the recommendation experiments. Under grouped demographic privacy evaluation, the combined condition preserves recommendation utility with $N D C G {@} K=0.6312$ while reducing exact and entity linkage from 0.7090/0.7204 to 0.0001/0.0000. Sensitive-target attacker performance remains near the majority baseline, supporting the claim of empirical privacy improvement without visible ranking degradation in that benchmark. The healthcare experiments also demonstrate meaningful embedding transformation and linkage reduction, though the current benchmark remains datalimited and therefore less conclusive for utility-focused evaluation. Overall, the findings support the conclusion that embeddingspace transformation can preserve downstream utility while substantially reducing linkage risk and sensitive-information recoverability under explicit attacker evaluation. The findings support embedding-space transformation as a practical privacypreserving strategy for embedding-driven AI systems under explicit attacker evaluation.

Don't Trust the AI Ecosystem: Analyzing Privacy Leakage in Compromised Open-Source Components

GradLock is introduced, a novel training-time injection attack that stealthily injects sensitive training data directly into the model parameters and employs dynamic gradient locking to prevent payload degradation during the optimization process.

Privacy-Preserving Framework for Multi-Institutional Medical Time-Series Analysis via Homomorphic Encryption: Design and Development Study

Abstract Background The development of robust medical AI for knowledge discovery and decision support commonly necessitates large-scale datasets from multiple institutions. However, such data aggregation is severely constrained by privacy regulations and the inherent risk of sensitive information leakage, making it difficult to navigate the utility-privacy trade-off. Objective We aimed to design a secure multiparty deep learning system that enables privacy-preserving modeling from distributed medical time-series data without centralizing raw information or exposing model parameters. Our goal was to achieve predictive accuracy comparable to nonsecure models while providing strong security and efficiency. Methods We developed a framework using threshold homomorphic encryption to securely train recurrent neural networks on distributed longitudinal data. To improve the efficiency, we proposed an optimized encrypted matrix multiplication scheme, a secure ciphertext refresh protocol, and used lightweight encryption parameters and low-degree approximated activation polynomials. The system was evaluated on 4 real-world intensive care unit datasets for tasks like mortality and sepsis prediction. Results The system demonstrated practical efficiency, requiring approximately 1 minute per training iteration for processing 125 local batches over 39 variables and 48 time steps, and scaling well with data size and participant number. Securely trained models achieved predictive performance that was comparable to, and in some cases superior to, nonsecure centralized models, highlighting their ability to learn generalizable patterns in different unseen data distributions. For example, on the PhysioNet Challenge 2012 dataset, our secure model achieved an area under the curve (AUC) of 0.8480, outperforming the nonsecure baseline AUC of 0.8404. Conclusions This work provides a viable and efficient solution for cross-institutional, privacy-preserving analysis of longitudinal medical data. The framework successfully bridges the utility-privacy gap, facilitating safer collaborative research and enabling robust knowledge discovery and decision support while adhering to strict data protection standards.