Skip to content
Log in

Similar papers

Preprint Jul 2026

When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training

To overcome data scarcity and privacy constraints in data collection, it has become standard practice across academia and industry to augment real training data with text-to-image (T2I)-generated synthetic data, a paradigm we term Real-Synthetic Mix-Training (RSMT). While substituting synthetic data for sensitive real samples is widely regarded as a means to mitigate privacy exposure of the substituted data, the risk to the remaining real samples that actively participate in training has remained largely unexamined. This work reveals, for the first time, that RSMT can substantially amplify privacy leakage of these real training samples. We establish a theoretical framework, RSMT Memorization Amplification, proving that incorporating synthetic data displaces real samples toward peripheral regions of the mixed feature space, in turn forcing the model to memorize them more aggressively. Guided by this foundation, we propose RSMixLeak to systematically assess this risk through membership inference attacks (MIAs). RSMixLeak comprises two variants depending on the adversary's capability. The non-adversarial variant audits a benign RSMT pipeline with an honest T2I provider, establishing a lower bound on the leakage induced by the intrinsic gap between real and T2I-generated data. The adversarial variant considers an adversary who controls the T2I model or contributes crafted data to the T2I provider, and deliberately enlarges this distributional gap on a target class via either high-level semantic attribute binding or imperceptible pixel-level coating, further amplifying leakage on real training data while improving downstream model utility. Motivated by these findings, we further propose a lightweight leakage propensity indicator computable from real data alone that reliably identifies high-risk datasets unsuitable for entering RSMT, as a self-assessable mitigation.

Na Li, Boyu Kuang, Hongsheng Hu et al. · 0 citations
Conference Open access 2026

Robust Membership Inference for Large Language Models under Adversarial Generative Corruption

Membership inference attack (MIA) has emerged as a promising tool for auditing the training data of LLMs, supporting data privacy and copyright protection. Most existing MIA methods rely on the assumption that LLMs assign higher confidence scores to training samples than to non-training ones. However, since LLMs generate text by sampling high-confidence tokens, they naturally produce AI-generated texts (AIGTs) that also satisfy this assumption. In this work, we empirically confirm that such AIGTs, regardless of whether they are generated by the target LLM, can lead existing MIAs to assign even higher membership likelihoods than those of true training samples, thereby significantly un-dermining their reliability. To address this challenge, we propose a robust membership inference framework for reliably identifying training data. Our method adopts a mixture-of-experts formulation to jointly model interactions across complementary features derived from multiple MIA methods and AIGT detectors, which can remain robust against ad-versarially generated samples. Furthermore, by leveraging expert components, our method provides explainable insights into the characteristics of member data. Experiments on various datasets and LLMs show that adversarial samples substantially degrade the performance of baselines, whereas our method preserves performance close to that of the unattacked setting. Codes and datasets are released at https://github.com/kong-hyh/MoMIA .

Yuanhong Huang, Huili Wang, Xueying Bai et al. · 0 citations
Preprint Jul 2026

Inference-Time Mitigation of Adversarial Political Bias in Large Language Models

As Large Language Models (LLMs) become the mainstay for information retrieval and summarization tasks, ensuring that they are always non-partisan and invulnerable to political bias is a critical step towards safer and more trustworthy Artificial Intelligence (AI). Current model alignment paradigms, such as reinforcement learning from human feedback (RLHF), make LLMs follow overarching safety instructions. However, this instruction tuning can be exploited via adversarial prompt injection and be used to generate unsafe content. In particular, political bias has not been specifically targeted by modern alignment techniques as harmful and biased content. To address this vulnerability of LLMs, we propose mitigation strategies using Chain of Thought (CoT) prompting and Direct Preference Optimization (DPO). Using a public dataset of legislative videos, we generate summaries using LLMs, inject bias via adversarial prompting and evaluate their performance on a four axis scale designed for political summarization. In this paper, we present different methods to shield LLMs against the injection of political bias. Our results demonstrate that the proposed Recursive Self-Correction approach raises model performance from a Political Neutrality Likert scale baseline of 2.14 to 4.56, averaged across all models, demonstrating effective inference-time mitigation of political bias in LLM-generated summaries.

Tejaswi V. Panchagnula, Bruce Coburn, Bryce J. Dietrich et al. · 0 citations
Book Open access Jul 2026

Investigating Robustness in Vision-Language Models via Adversarial Prompt Illumination

Trained on large corpora of image-text pairs, vision-language models (VLMs) have proven broadly useful across many applications. However, they can still make errors that humans rarely do, particularly when exposed to adversarial inputs crafted to mislead them. Traditional approaches to uncovering such vulnerabilities typically optimize a single input, such as a text prompt, to induce incorrect predictions while remaining plausible to human readers. These methods tend to identify only one or a few high-impact adversarial examples, offering a narrow view of model weaknesses. In contrast, we argue that a Quality-Diversity (QD) perspective is more informative. Rather than searching for a single best attack, QD explicitly aims to generate many high-quality adversarial prompts spanning diverse behaviors and characteristics. This allows us not only to diagnose model weaknesses, but also to characterize which prompts are robust and which are especially fragile. Our experiments show that CVT-MAP-Elites, a QD method integrated into our pipeline, discovers a richer and more diverse set of meaningful adversarial samples than quality-only optimization. Consequently, our approach achieves broader search-space coverage and provides deeper insight into VLM failure modes on text-to-image retrieval tasks in both general and medical domains.

Thai Huy Nguyen, Khoa Tran, Quan Minh Phan et al. · 0 citations
Review Open access 2026

A Comprehensive Survey of Generative AI: Applications and Future Directions Across Domains

Generative Artificial Intelligence (GenAI) has become a foundational paradigm for learning complex data distributions and synthesizing realistic content across text, image, audio, tabular, scientific, and multimodal data. Despite rapid progress, the literature remains fragmented across model families and application domains, making it difficult to compare methodological choices, evaluation practices, and domain-specific deployment constraints. This survey addresses this gap by reviewing GenAI from both an algorithmic and cross-domain perspective. Following a PRISMA-inspired search and screening procedure, we organize representative and technically relevant studies across major GenAI families, including variational autoencoders, generative adversarial networks, diffusion models, autoregressive models, flow-based models, and recent foundation and multimodal models. We synthesize applications across healthcare, agriculture, manufacturing, transportation, earth and environmental systems, computer science and networks, materials science, finance and economics, education, business and services, and creative industries. The review shows that diffusion and autoregressive foundation models increasingly dominate high-fidelity image, language, and multimodal generation, while GANs, VAEs, and flow-based models remain important in data-limited, structured, scientific, and privacy-aware settings. Beyond summarizing applications, the paper compares domain-specific data structures, evaluation practices, robustness concerns, failure modes, and ethical risks. The survey contributes a unified reviewed abstraction of GenAI workflows, a comparison with prior surveys, a domain-aware synthesis of open gaps, and a future roadmap for trustworthy, validated, and responsible GenAI deployment.

A. Javadpour, F. Ja’fari, T. Taleb et al. · 0 citations
Conference Open access 2026

Stop Hardening Everything: A Training-Free Neuron-Level Defense for Neural Ranking Models

While neural ranking models (NRMs) have achieved state-of-the-art performance in information retrieval, they remain highly vulnerable to imperceptible adversarial perturbations. Existing defenses are predominantly data-centric , exemplified by adversarial training, which requires constructing large collections of adversarial examples. By treating NRMs as black boxes and indiscriminately optimizing all model parameters, these methods incur substantial computational cost and often degrade performance on clean data due to over-fitting. In this paper, we advocate that adversarial vulnerability is not uniformly distributed across model parameters, but instead originates from specific internal units. We propose a paradigm shift toward a model-centric defense that addresses vulnerability at its architectural source, without requiring costly retraining or adversarial data generation. Specifically, we introduce Search in the Model , a novel training-free framework that performs fine-grained identification and rectification of vulnerable neurons directly within the model. By formulating neuron identification as a ranking problem, we develop a maximum marginal vulnerability criterion to precisely locate the top-K neurons most responsible for model vulnerability, and apply targeted neuronal inverse perturbation to correct them. Extensive experiments on MS MARCO and TREC 19 show

Yuansan Liu, Ruqing Zhang, Hongru Song et al. · 0 citations