A configuration-resolved benchmark of differential abundance analysis methods for human gut 16S rRNA microbiome data

Abstract

Tools for differential abundance testing of 16S rRNA data are conventionally treated as discrete methods, and benchmarks have accordingly sought to determine which tool performs best. However, each tool offers an array of configurations based on different normalisation, transformation, reference choice, and sensitivity filtering methods, and the specific impact of these configurations on performance has rarely been systematically investigated. We benchmarked five widely used tools (MaAsLin 2, MaAsLin 3, edgeR, ALDEx2, and ANCOM-BC2) across 18 configurations, using simulated communities and human gut profiles with implanted signals, at two taxonomic resolutions and across several design factors. Configuration accounted for as much performance variation as the choice of tool itself, with the ranking of two tools depending on which of their settings are compared. Individual parameters behaved as switches between opposite error regimes rather than as graded adjustments, and the settings carrying this weight are identifiable in advance. These behaviours were reproducible across data sources and resolutions. Our results define a configuration-aware framework for matching a tool and its settings to the cohort, study design, and feature resolution, establishing that a differential abundance result is interpretable only if the configuration used for the analysis is reported. Importance Most microbiome studies rely on a single software run with default settings to identify disease-associated bacteria, yet the settings inside a tool can alter results as much as switching tools. By systematically testing eighteen combinations of normalisation, transformation, and filtering options across human gut microbiome data, we provide a practical guide that prevents false biological conclusions in clinical microbiome research. The framework is immediately applicable to inflammatory bowel disease and dysbiosis studies, where sample size, community composition, and the expected proportion of differential taxa vary widely.