Real-Time Big Data Streaming and Event-Driven Architectures for Next-Generation Enterprise Systems

Abstract

Real-time enterprise computing is transitioning from periodic data movement and tightly coupled request-response integration onto continuous event streams, independently scalable services, and stateful processing that adapts as business conditions change. This review integrates research published between 2020 and 2025 on big data streaming, event-driven architectures, microservices, transactional stream processing, observability, and serverless execution. Its goal is to explain how these domains converge into a integrated architecture for next gen enterprise systems, rather than treating streaming as an isolated analytics component. A structured integrative review was carried out using DOI-verifiable, peer-reviewed sources from major computing publishers and journals. Thirty studies were retained based on their focus on architectural design, streaming semantics, scalability, reliability, consistency, deployment, or operational governance. Key factors such as event-time semantics, watermarks, durable logs, checkpointed state, idempotent consumption, schema evolution, distributed transaction models, and end-to-end observability collectively determine a system’s ability to deliver timely and trustworthy outcomes. Comparative evidence demonstrates that no stream-processing framework is universally optimal; workload characteristics, state requirements, deployment topology, and cost constraints considerably influence engineering decisions. While event-driven microservices enhance autonomy and extensibility, they also bring complexity in information consistency, asynchronous failure recovery, tracing, and governance. Serverless and edge patterns can reduce operational cost and network latency, but cold starts and distributed state management remain major challenges. This review proposes a layered enterprise event fabric and a correctness-resilience loop that integrate transport, stream computation, domain choreography, data governance, and operational assurance. The resulting research agenda highlights adaptive state management, portable semantics, energy-aware scheduling, automated data contracts, and verifiable real-time service-level objectives.