A Systematic Literature Review of Emerging Big Data Analytics Using Hadoop: Applications, Trends, and Implications for Sustainability

Abstract

The rapid expansion of Big Data has transformed data-driven decision-making across diverse sectors, including healthcare, cybersecurity, smart cities, and the Internet of Things (IoT). However, traditional data management platforms struggle to address the increasing volume, velocity, variety, and veracity of modern data ecosystems, resulting in scalability constraints and inefficiencies. These challenges necessitate distributed, cloud-native, and energy-efficient architectures capable of supporting Artificial Intelligence (AI) and Machine Learning (ML)–driven analytics. This paper presents a systematic literature review (SLR) of emerging Big Data analytics frameworks centered on the Hadoop ecosystem and its modern extensions, including Apache Spark, Flink, Kafka, and data lakehouse technologies. Following the PRISMA 2020 guidelines, the review analyzes 138 primary studies published between 2015 and 2025, with 60 studies selected through rigorous inclusion and quality assessment criteria. The analysis evaluates architectural evolution from batch-oriented MapReduce systems to unified batch-stream processing and cloud-native data platforms, emphasizing performance optimization, resource efficiency, and sustainable computing practices. The findings reveal significant progress in integrating AI and ML pipelines with Hadoop-based infrastructures for applications such as real-time fraud detection, anomaly detection, predictive healthcare analytics, renewable energy forecasting, and intelligent urban management. Particular attention is given to explainable AI (XAI), federated data processing, and green computing strategies that enhance transparency, trust, and environmental sustainability. This review proposes a structured taxonomy of Hadoop-based Big Data systems, categorizes emerging research trends, benchmarks modern frameworks, and identifies open research challenges. The study provides theoretical, practical, and policy-level implications to guide researchers, system architects, and decision-makers in selecting scalable, low-latency, and energy-aware Big Data solutions aligned with next-generation digital transformation and sustainable development objectives. Not applicable, as this study is not a clinical trial.