The rapid scaling of large language models (LLMs) has made communication overhead a critical bottleneck in distributed model-parallel training. Although existing low-precision compression methods can effectively reduce communication costs, further reducing the communication footprint to ultra-low-bit formats such as FP...
Man Liu, Xing-Chen Liu, Hai-Rui Zhao et al.· 0 citations
CC-Bench is presented, a lightweight, extensible, and application-oriented benchmark suite for evaluating communication compression under realistic execution conditions, and uses declarative application-environment modeling to decouple profiling logic from communication libraries, datasets, and fidelity metrics, enabli...
Hao Fan, Wei Wang, Xing-Chen Liu et al.· IEEE International Symposium...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.