DualPipe Bidirectional Overlap Telemetry
Interleaves forward and backward micro-batches across dual symmetric pipelines, partitioning backward propagation into B_input and B_weight to completely hide cross-node All-to-All network latency behind matrix computation kernels.
Pipeline Bubble: Under 2% Ratio
Model Flops Util: 63%+ MFU Frontier
Network Overlap: 98%+ All-to-All Hidden