Mixture-of-experts inference introduces fine-grained, dynamically changing many-to-many communication for token dispatch and collection, making serving latency highly sensitive to the interconnect. Optical circuit switching (OCS) offers an optically transparent data plane, but its benefits for MoE depend critically on circuit reconfiguration latency, which has not been quantified for inference workloads. We develop a performance model and an OMNeT++ simulation methodology to quantify the reconfiguration-latency budget of an OCS-based switching fabric for MoE inference. Results reveal a sharp regime transition: nanosecond-scale reconfiguration preserves favorable latency, throughput, jitter, and task completion time, whereas microsecond-scale reconfiguration collapses throughput and inflates completion time by orders of magnitude. Trace-driven replay using measured DeepSeek-V3 inference communication traces collected from an 8-H20 GPU server confirms that the same latency regimes persist under measured MoE traffic, supporting the representativeness of the model-generated workload. With Tb/s link bandwidth and increasing oversubscription, the budget tightens to the tens-of-nanoseconds regime (e.g., <inline-formula><tex-math notation="LaTeX">$44.3 \,{\mathrm{ns}}$</tex-math></inline-formula> at <inline-formula><tex-math notation="LaTeX">$1.6 \,{\mathrm{Tbps}}$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$17.2 \,{\mathrm{ns}}$</tex-math></inline-formula> at <inline-formula><tex-math notation="LaTeX">$3.2 \,{\mathrm{Tbps}}$</tex-math></inline-formula> in the evaluated configuration). Finally, we demonstrate <inline-formula><tex-math notation="LaTeX">$43.4 \,\mathrm{n}\mathrm{s}$</tex-math></inline-formula> end-to-end circuit reconfiguration on a multi-endpoint prototype, validating feasibility at the implied timescales.
Shuo Li, Hua-Xi Gu, Yi-Xuan Hao et al.· Journal of Lightwave Technol...· 0 citations
Optical-circuit-switched interconnects have become one of the core components for AI training due to their flexible topology reconfiguration. In contrast to the applications carried by traditional data center networks, large-scale language model training is highly sensitive to network failures, where frequent disruptions will cause gradient synchronization delays, leading to training interruptions and wasted computational resources. Existing schemes are primarily focused on specific communication patterns, without considering the fault probability distribution. As a result, unreliable links remain on critical paths. Furthermore, passive fault response mechanisms lead to inefficient topology reconfigurations, preventing network protocol convergence and making it difficult to meet the stringent stability requirements of large-scale model training. To address reliability challenges in optical-circuit-switched interconnect, we propose TopoCrafter, which leverages dual-agent deep reinforcement learning to proactively mitigate network failures. The “Topo-Agent” estimates link failure probabilities to determine reconfiguration timing and then employs a lightweight heuristic algorithm to create failure-avoidant topology that matched to traffic pattern. Concurrently, the “Route-Agent” optimizes traffic distribution. Through their strategic interaction, the agents learn holistic policies that optimally balance network reliability and communication efficiency. To improve generalization, a progressive training approach is employed, allowing the agents to adapt to complex failure environments while accelerating convergence. Under link failure scenarios, TopoCrafter maintains reliability, reducing end-to-end latency by up to 50% and maximum link utilization by approximately 20% compared to FatTree. In addition, progressive training algorithm ensures a performance degradation of less than 10% when adapting to new failure environments, and it maintains stable high performance as the network scales.
Liang Qin, Xing-Yu Liu, Wen-Ting Wei et al.· IEEE Transactions on Cogniti...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
