Benchmarking Fault-Tolerance Characteristics of Actor-Based Runtimes

Abstract

Fault tolerance is a fundamental requirement of distributed systems, and actor-based runtimes provide a widely adopted approach for building resilient and highly concurrent applications. Although several actor ecosystems offer mechanisms for supervision, failure detection, and recovery, comparative studies frequently focus on performance metrics rather than fault-tolerance behaviour. This paper presents a language-independent benchmarking framework for evaluating fault tolerance in actor-based runtimes. The framework was implemented using three representative ecosystems: Elixir/BEAM, Scala/Akka, and Go/Proto.Actor. A distributed chat-based benchmark application was used to measure throughput, reconnection latency, and failure-detection latency under recurring transient failures. All implementations followed an equivalent architecture and were executed under identical experimental conditions. The study deliberately targets a single, well-defined fault model: the supervised crash recovery of in-memory, effectively stateless actor services, in which chat actors are abruptly terminated and restarted by their supervisors while clients rediscover and reconnect to them. Stateful recovery (actor state, mailbox contents, in-flight or persistent messages), as well as multi-node network effects, are explicitly out of scope. Accordingly, the benchmark characterises supervised crash–recovery behaviour for largely stateless actor services rather than providing a comprehensive evaluation of actor-based fault tolerance. The results reveal distinct trade-offs among the evaluated ecosystems. Elixir achieved the highest throughput and the lowest throughput variability under fault conditions, while Scala/Akka consistently provided the lowest reconnection and failure-detection latencies, particularly at large scale. Go/Proto.Actor remained competitive in throughput-oriented scenarios but showed greater degradation in recovery-related metrics as concurrency increased. The results indicate that no single runtime dominates all evaluated dimensions of recovery behaviour. Beyond the runtime comparison, this work contributes a reproducible benchmarking framework that provides a foundation for future empirical studies of actor-based runtime recovery under controlled fault conditions.