Jul 2026· Anais do LIII Seminário Integrado de Software e Hardware (SEMISH 2026)· pp. 914-919· 0 citations· 6 references
TL;DR
ParameterEfficient Fine-Tuning (PEFT) mitigates data leakage in Llama and Gemma and Qwen remain highly vulnerable across all adaptation methods (p < 0.05), while Gemma and Qwen remain highly vulnerable across all adaptation methods.
Abstract
Dataset Inference provides a robust framework for auditing data ownership by aggregating statistical signals that traditional Membership Inference Attacks (MIAs) fail to capture. While proven for LLM pre-training, its efficacy during fine-tuning is largely unexplored. We evaluate Dataset Inference on Gemma, Llama, and Qwen models using Full Fine-Tuning (FFT), LoRA, and QLoRA. Our findings reveal a stark architectural divergence: ParameterEfficient Fine-Tuning (PEFT) mitigates data leakage in Llama (AUC ≈ 0.50, p > 0.05), while Gemma and Qwen remain highly vulnerable across all adaptation methods (p < 0.05). Additionally, higher learning rates accelerate data absorption, and QLoRA provides only marginal regularization compared to standard LoRA.
This work identifies a phenomenon termed token-level memorization asymmetry through theoretical analysis of diffusion training dynamics and proposes Q-Skew, a quantile-weighted skewness-based indicator for membership inference on finetuned DLMs.
Sheng-Fang Zhai, Leo Marchyok, Yu-Ling Shi et al.· 1 citation· ⚡1
This is the first study to target worst-case demographic personas with LoRA fine-tuning for cross-cultural bias mitigation, and country-level decomposition reveals that fine-tuning redistributes rather than removes bias.
It is argued that WG is more plausible as an adversarial threat-requiring careful data engineering-rather than as a significant hazard inherent to routine fine-tuning.
Benchmark scores are reported as properties of a model, yet the inference framework used to produce them, such as HuggingFace, vLLM, or Ollama, are considered non-influential and their names and versions are almost never disclosed. In this work we investigate how much this choice can influence the model output. In a fu...
Shahed Masoudian, Passant Shafaei, Monorama Swain et al.· 0 citations
Membership inference attacks (MIAs) try to determine whether a specific record was used to train a model, a privacy risk that matters in natural language processing (NLP), where training data can contain sensitive user text. This paper presents a controlled benchmark of membership inference vulnerability for text class...
TopoTuner is competitive with full fine-tuning while training only 1-2% of the model parameters, and outperforms LoRA in 7 out of 9 model-dataset settings, which can change up to 39.57% of the projection parameters.
