CC-GT to GFLCC: From Synchronous to Asynchronous Conjugate Compressed Gradient Tracking for Distributed Optimization

Abstract

In distributed optimization, communication bottlenecks and computational heterogeneity are two critical challenges that undermine system scalability and efficiency. To address these issues, we propose a synchronous algorithm, CC-GT (Conjugate Compressed Gradient Tracking), which integrates conjugate gradient acceleration with dual error-compensated compression to reduce communication overhead and accelerate convergence. Theoretically, we establish its linear convergence under strongly convex and smooth assumptions, and extensive experiments on regression, classification, and neural-network tasks validate the fast convergence and communication efficiency of the synchronous CC-GT. Building on this foundation, we develop GFLCC, an asynchronous extension that eliminates global synchronization barriers while preserving the core mechanisms of CC-GT, thereby inherently mitigating the straggler effect and enabling dynamic load balancing. For the asynchronous GFLCC, extensive comparisons against state-of-the-art asynchronous baselines under harsh network interference demonstrate its strong empirical convergence speed, communication efficiency, and robustness. These results confirm that the proposed framework offers a practical and theoretically grounded solution for communication-constrained and heterogeneous distributed optimization systems.