Skip to content
Preprint

GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification

Jul 2026 · 0 citations · 49 references
Computer Science

TL;DR

GenSyn10 is introduced, a CIFAR-10-aligned synthetic image dataset of 60,000 images generated using three architecturally diverse state-of-the-art models, enabling controlled and systematic evaluation of out-of-distribution (OOD) generalization to novel generators.

Abstract

The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before. We introduce GenSyn10, a CIFAR-10-aligned synthetic image dataset of 60,000 images (10 classes, 32$\times$32, 50k/10k split) generated using three architecturally diverse state-of-the-art models: FLUX.2-dev (Rectified Flow Transformer), HunyuanImage-3.0 (MoE Transformer), and Qwen-Image-2512 (Multimodal Diffusion Transformer), to advance research in AI-generated image detection. A central challenge in this domain is that detectors perform well on known generators but degrade on unseen ones. GenSyn10 addresses this limitation by curating data from multiple contemporary architectures under a standardized generation protocol, enabling controlled and systematic evaluation of out-of-distribution (OOD) generalization to novel generators. Images are generated using a template-based prompt engine and downsampled to ensure consistency. We evaluate 17 image classification models under a four-stage protocol: real-data baseline, zero-shot transfer, fine-tuning, and retention. Despite a measurable domain gap, CIFAR-10-trained models achieve up to 96.86\% zero-shot accuracy on GenSyn10, increasing to 99.88\% after fine-tuning. In binary real-vs-synthetic classification, fine-tuned models achieve 97-99.9\% accuracy on seen generators but drop to 79-96\% on images from an unseen generator, highlighting persistent limitations in OOD generalization. These results establish GenSyn10 as a controlled benchmark for studying synthetic image detection beyond single-generator settings, supporting research on robustness, domain adaptation, and cross-generator generalization.

View source

Similar papers

Conference Jul 2026

Deep Learning-Based Detection of Real and AI-Generated Images With Multiple Generators and Multi-Scaling Features

The pervasiveness those involving computer generated images, especially in the last decade has seen massive generational leaps, with the likes of GANs, autoencoders, and diffusion models now advanced enough to produce images so real that distinguishing between such and existing pictures becomes an uphill task. This paper serves a significant role in the improvement of modern deep learning models that can tackle different AI image generators and their performance in simulated images. The authors provide data pools with collected datasets to close previous research on gap existence such as limited cross-generator generalization, lack of focus on fine-grained relic detection, and absence of accurate alterations. The abundant dataset being discussed is a tangle of real imaginations, faked media Recognition video sequences, StyleGAN-made pictures, ProGAN/PGGAN outputs, and Stable Diffusion artificial visuals which were collected from Kaggle. Each of the dataset classes was leveled in quantity and then made rugged with different types of real-world alterations like solidity pieces, noise in low sunlit, occlusions, impression, and so on. A variety of other artifacts occurring in the output were ultra local and required more specific information for elimination. Plans were further made to create a model that would perform better with structured information based on the global context such as the frame-level status with fine-grained artifacts around and the proposed improvement focuses on the recognition of textured elements in the image on the level of individual parts. Extensive and exhaustive tests assessing the estimation features based on F1-score, precision, accuracy, recall, ROC-AUC metric as well as cross-generator estimation which involve the very behavior depreciation as a weakness revealed that the mentioned framework can be applied to new unseen generative models while preserving the proposed optimality with the diffusion-based datasets included.

Prashali Billore, Vineeta Shrivastava, Kirti Verma et al. · 0 citations
Preprint Open access Aug 2026

Robustness of AI-Art Detectors under Generator Shift

Text-to-image generative models have advanced rapidly, with modern Diffusion Transformer architectures producing images that are increasingly difficult to distinguish from human-created artwork. This development has raised significant concerns regarding copyright protection, misinformation, fraud, impersonation, and the authenticity of digital content. Most AI-art detectors are trained and evaluated on the same generator family, leaving robustness to newer architectures underexplored. In this chapter, we analyze generator shift based on a Stable Diffusion 3.5 Medium (SD3.5m) artwork dataset spanning ten art styles through reverse prompting of held-out human artwork samples. Five detectors are trained on U-Net-based latent diffusion artwork and evaluated in a zero-shot cross-generator setting on the SD3.5m dataset. Deep learning models perform strongly in-distribution but degrade under generator shift, misclassifying many SD3.5m images as human while human false positives remain low. The CLIP ViT-L/14 model performs best overall, while Grad-CAM analysis reveals weaker and more diffuse activation on false negatives. These findings highlight a generalization gap in current AI-art detectors and motivate the development of detectors as one component of a layered defense that remains reliable across rapidly evolving generative architectures.

Shivank Singh Thakur, Meien Li, Mark Stamp · 0 citations
Book Open access Jul 2026

Radial Residual Frequency: A Semantically Aligned Benchmark and Spectral Detector for AI-Generated Images

A data-generation pipeline that captions real photographs with a vision–language model and regenerates them with modern text-to-image systems, producing semantically aligned real/synthetic pairs that isolate generative artifacts from image content is described.

Guang Yang, Qian Zhang, Fengchen Liu · 0 citations
Open access Jul 2026

Attention-Based Deep Learning Pipeline for AI-Created Image Recognition

The advent of generative AI models that can generate some of the most realistic images including DALL·E, Midjourney, and Stable Diffusion has posed a significant threat to digital forensics and media authenticity verifications. Conventional methods of forgery detection, which focus on splicing or copy-move manipulations, are not very efficient in the case of AI-generated images, which do not contain any explicit tampering traces. This paper presents Attention-Based Deep Learning Pipeline of AI-Created Image Recognition, intended to resolve the increasing challenge of separating AI-synthesized and authentic imagery. The proposed architecture incorporates three integrated branches, including low-level statistical feature extraction, high-level semantic representation learning, and attention-based feature refinement mechanism. The features that are extracted are combined and fed through a fully connected classifier to generate final predictions. The Synthbuster dataset that consists of images of several generative models, including DALL·E 2, DALL·E 3, Firefly, Glide, and versions of Stable Diffusion, were experimentally evaluated. The suggested framework score had an average AUC of 97.4%, as well as high accuracy, precision, recall, and F1-score. These findings support the robustness and generalization ability of the proposed model in detecting AI-generated images in a variety of generators and conditions.

Nadia Ali · 0 citations
Preprint Aug 2026

MLLM-Routed Heterogeneous Ensembles for Robust Cross-Dataset Image Classification

Modern image classification models excel when trained on single task-specific datasets but often struggle to generalize across domains and difficulty levels. We propose ARMDIL, an Adaptive Router for Multi-Domain Image Classification with LLMs. ARMDIL is an ensemble that uses a multimodal large language model (MLLM) agent to dynamically route each image to the most suitable vision backbone. Our diverse ensemble employs convolutional neural networks (ResNets), self-supervised representation learners (SSL), and vision language models (VLMs), each trained on a unified label space constructed from multiple image datasets with differing distributions and characteristics. Empirical evaluations illuminate the distinct capabilities and vulnerabilities of each architecture across disparate visual domains. Crucially, we show that ARMDIL effectively navigates these tradeoffs, performing competitively with specialized training-based routers. Furthermore, it drastically improves adaptability by allowing new information to be integrated via simple prompt modifications, while enhancing interpretability through natural language reasoning traces. These advances in cross-dataset image classification pave the way for more reliable general-purpose vision systems such as AI assistants and autonomous robots.

Daniel A. Perkins, John Squires, Janou Milligan et al. · 0 citations
Preprint Aug 2026

Generated Images Are Easier to Forget: A Machine Unlearning Perspective for Synthetic Image Detection

This work establishes a new paradigm for generated image detection by recasting the detection task as a problem of machine unlearning, and introduces two detection methods: data-free detection, which prunes model parameters to induce unlearning without data access, and data-driven detection, which optimizes LVMs to unlearn knowledge tied to generated images.

Jun Nie, Yonggang Zhang, Tongliang Liu et al. · 0 citations