GPU-Accelerated Parquet Compaction and Index Construction for High-Cardinality Time-Series Databases

Abstract

Time-series databases commonly employ LSM-Tree with Parquet columnar storage, where compaction and index construction act as major performance bottlenecks in high-cardinality IoT and cloud monitoring scenarios. Traditional CPU-only compaction suffers from low parallelism and high computation overhead, while mainstream GPU solutions such as cuDF fail to adapt to time-series features. They load complete Parquet files without optimization for string primary keys and lack an end-to-end accelerated index building workflow. This work presents a CPU-GPU collaborative framework for Parquet compaction and index construction. It partitions Parquet row groups into overlapping and non-overlapping segments by primary key ranges to avoid unnecessary global sorting. A global dictionary alignment scheme transforms variable-length string keys into integers to eliminate GPU warp divergence and enable efficient radix sort. Combined with lazy materialization, the framework only processes sorting keys and delays metric field parsing, significantly reducing GPU memory consumption and data transfer cost. A two-phase adaptive cost model enables effective heterogeneous task scheduling. Additionally, a parallel GPU pipeline accelerates tag inverted index construction, leaving only FST and bitmap generation to the CPU. Evaluated on the TSBS benchmark, our framework outperforms CPU baselines and cuDF substantially. It cuts compaction time by nearly an order of magnitude, speeds up index construction by 3.8×–5.2×, and boosts end-to-end throughput by 7.0%–16.0% with stable query latency.