Reaching the Pinnacle of TPC-DS: Co-Design of Architecture, Executor, and Storage in TDSQL
Enterprise data explosion and the urgent industry demand for realtime complex multidimensional analytics require OLAP databases to be highly scalable, efficient, and cost-effective. Though diverse solutions (shared-nothing MPP databases, cloud-native decoupled systems, in-process analytical engines) exist with respective strengths, they all have critical inherent flaws. In response, this paper presents TDSQL, a distributed OLAP database system developed by Tencent. We leverage its native architectural advantages, analyze the merits and drawbacks of state-of-the-art systems, and elaborate on the rationale behind our technical solution selection and proprietary innovations tailored for TDSQL. Specifically, built upon the traditional MPP execution framework, TDSQL incorporates the Forward Node mechanism to enhance scalability. By optimizing parallel execution, runtime filter strategies, and designing and implementing a vectorized execution engine, TDSQL achieves rapid response to large-scale complex queries. Experimental results based on the TPC-DS benchmark demonstrate that TDSQL ranks first among publicly reported systems. In a cluster configuration for 10,000 GB data size, TDSQL achieves a score of 72.6 million QphDS, which is 1.81 times and 3.82 times the scores of the second- and third-highest-performing database systems in public TPC-DS results, respectively, while offering a 79% and 37% lower cost per 1000 QphDS.