LBoDSN: An In-Network Load Balancing Mechanism for Lossless Data Center Networks Based on Direct Switch Notification

Qingyu Shi, Fangxue Jiang, Huang Huang, Xiaocui Li, Chuang Li, Wenzhi Cao, Limei Liu

Abstract: With the extensive deployment of Remote Direct Memory Access (RDMA) in lossless data center networks (DCNs), enhancing the RDMA load balancing performance for distributed AI training and HPC applications becomes particularly critical. However, existing load balancing schemes either suffer difficulty in responding to congestion in sub-RTT time, or cannot accurately detect and reroute flows that are on the verge of creating path congestion for Priority-based Flow Control (PFC) enabled lossless DCNs. In this paper, we propose LBoDSN, an in-network load balancing mechanism for lossless DCNs based on direct swtich notification, to address above challenges. LBoDSN monitors ingress queue length evolution at destination switches to predict the triggering time of PFC pause, and accurately identifies congested flows based on the congestion contribution level before PFC pause, then further proactively sends the flow congestion notification (FCN) to source switches for fast rerouting. The FCN contains a flow ID that identifies the congested flow, and a path ID that identifies the target path to which the congested flow will be rerouted. And after rerouting, the Congestion Notification Packet (CNP) of the old path is selectively discarded at source switches to improve transmission performance, while out-of-order packets are reordered at the destination switch. The experimental results show that under realistic workloads, LBoDSN achieves 9–64% and 20–80% better than CONGA for the average and tail Flow Completion Times (FCTs), respectively. Compared with ConWeave, LBoDSN achieves around 8% better performance for the average and tail FCTs while significantly reducing switch queue usage for reordering.