Investigating Parallel Scaling Bottlenecks Across Rust, Julia, Haskell, and Python: Workload–Runtime Signatures

Abstract

Parallel performance depends not only on programming language and runtime design, but also on how the dominant execution bottleneck changes as parallelism increases. We present a controlled cross-language study of Rust, Julia, Haskell, and Python using Merge Sort, Closest Pair of Points, and Numerical Sum in a multicore environment. For each of the three workloads, we evaluate four language-based implementations at five worker counts p∈{1,2,4,8,16} using two input sizes and 10 untrimmed trials per configuration, yielding 3 × 4 × 5 × 2 × 10 = 1200 benchmark observations. We propose a decomposition-based diagnostic framework built on three measured components: slowest-worker computation (Cp), algorithmic merge/combine work (Bp), and residual parallel overhead (Rp). Their normalized fractions, together with observed speedup, form a Workload–Runtime Scaling Signature (WRSS). Tracking WRSS across worker counts identifies Bottleneck Transition Points (BTPs). We additionally apply a standardized 20% component-reduction sensitivity analysis to express how strongly total parallel-region time depends on each measured component under an explicit ceteris-paribus assumption. Across the 3 × 2 × 4 = 24 workload–size–implementation conditions, each tracked over p∈{1,2,4,8,16}, 10 (41.67%) exhibit a bottleneck transition: six of eight Merge Sort conditions and four of eight Closest Pair conditions, whereas none of the eight Numerical Sum conditions exhibits a transition. At p=16, Merge Sort reaches only 2.02–3.19× median speedup because merge work dominates several configurations; Numerical Sum reaches 7.61–12.72× while remaining almost entirely computation-dominant. A separate 100-observation Python shared-memory ablation reduces Merge Sort residual overhead substantially, but leaves the merge stage dominant. The results show that useful parallelism depends on how workload structure and runtime mechanisms shape the evolution of the dominant bottleneck as worker count increases.