DEVOURED
NVIDIA Spectrum-X — the Open, AI-Native Ethernet Fabric — Sets the Standard for Gigascale AI, Now With MRC
NVIDIA, Microsoft, and OpenAI open-sourced MRC, the network protocol running OpenAI's Blackwell training clusters, which reroutes traffic in microseconds to keep thousands of GPUs synchronized.
What: NVIDIA released MRC (Multipath Reliable Connection) as an open specification through the Open Compute Project. The protocol enables a single RDMA connection to distribute traffic across multiple network paths, load-balancing to maximize GPU utilization and detecting failures in microseconds. It's deployed in production at OpenAI's Blackwell clusters, Microsoft's Fairwater data center, and Oracle's Abilene data center, all running on NVIDIA Spectrum-X Ethernet hardware.
Why it matters: NVIDIA open-sourcing a protocol they developed with Microsoft and OpenAI, after deploying it in production, signals they're positioning Spectrum-X Ethernet as an industry standard rather than a proprietary stack. This also reveals network reliability as a critical bottleneck in frontier AI training—important enough for AI labs to co-develop infrastructure with hardware vendors.
Deep dive
- Joint development by NVIDIA, Microsoft, and OpenAI, now released as an open specification through OCP to enable industry-wide adoption
- Solves a fundamental problem at scale: thousands of GPUs must stay synchronized during training, and any network disruption can halt the entire job
- Routes a single RDMA connection across multiple network paths, dynamically balancing load and avoiding congested paths in real time
- Detects failures in microseconds and reroutes at hardware speed, keeping GPU utilization high even during transient network issues
- Deployed in production at OpenAI's Blackwell training runs, Microsoft's Fairwater, and Oracle's Abilene—some of the largest AI factories built for frontier LLMs
- OpenAI's head of industrial compute Sachin Katti: "MRC enabled us to avoid much of the typical network-related slowdowns and maintain efficiency of frontier training runs at scale"
- Supports multiplanar network architectures where multiple independent fabrics provide redundant paths, with hardware-accelerated load balancing across planes
- Built on NVIDIA Spectrum-X Ethernet (ConnectX SuperNICs and Spectrum-X switches), demonstrated scaling to hundreds of thousands of GPUs
- Spectrum-X offers multiple RDMA transports (MRC, Adaptive RDMA, custom protocols) rather than forcing a single approach, positioning as a composable platform
- Broader collaboration includes AMD, Broadcom, Intel—signal that this is intended as an industry standard, not a proprietary NVIDIA-only protocol
Decoder
- RDMA (Remote Direct Memory Access): Network protocol that allows computers to exchange data directly in memory without involving the CPU, reducing latency. Critical for high-performance AI training where microsecond delays compound across thousands of GPUs.
- Multiplanar network: Architecture with multiple independent network fabrics, each providing an alternate communication path between GPUs for redundancy and scale.
Original article
Multipath Reliable Connection (MRC) is an RDMA transport protocol that enables a single RDMA connection to distribute traffic across multiple network paths. This improves throughput, load balancing, and availability for large-scale AI training fabrics. MRC delivers high levels of GPU utilization by load-balancing traffic across all available paths. It gives administrators fine-grained visibility and control over traffic paths to simplify operations and accelerate troubleshooting at scale.
Explore all 73 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi