PowerScale: Energy-Efficient Geo-Distributed Model Training with Federated Datacenter Power
PowerScale is presented, a hierarchical aggregation system that exploits the latency hierarchy of wide-area networks and shortens synchronization barriers and replaces per-site WAN transmissions with fewer, pre-aggregated transmissions at a lower frequency.