Optimizing HPC clusters with 10 Gigabit Ethernet iWARP technology
Date
:
10/1/2010
Today, some of the world’s most powerful supercomputers are composed of x86-based clusters with performance delivered on the network side using high-speed InfiniBand connectivity. But for high-performance computing (HPC) experts striving toward reduced fabric costs and simplified use, InfiniBand can present several challenges. Now, Internet Wide Area RDMA Protocol (iWARP) provides a way to deliver Remote Direct Memory Access (RDMA) clustering on 10 Gigabit Ethernet (10GbE) network adapters—an approach that enables cluster users to overcome fabric challenges and continue to push the supercomputing envelope.
A full implementation of currently available iWARP technology helps avoid virtually all processor networking overhead—bypassing the kernel, avoiding intermediate buffer copies, and accelerating transport (TCP/IP) processing to help reduce processor overhead, memory bandwidth utilization, and latency. Using mainstream Ethernet connectivity for compute clusters can now provide highly favorable performance, efficiency, and scalability, enabling organizations to obtain the low latency that HPC clusters need while capitalizing on the ease of use and familiarity of Ethernet. One 4,032-core cluster based on Dell™ PowerEdge™ servers with iWARP-enabled Intel® 10GbE server cluster adapters ranked at number 208 on the June 2010 TOP500 list, and reranking by efficiency shows that this cluster comes in at number 84—a level more efficient than many InfiniBand clusters, and the highest of the listed Ethernet solutions.
Using Ethernet connectivity as a unified fabric for cluster interconnects, LANs, and storage can help lower total cost of ownership by significantly reducing the number of switches and cables required. As 10GbE products and technology—including switches with high port density and technologies to further drive down latency—continue to evolve, future work can provide additional value in building supercomputing platforms on Dell PowerEdge servers interconnected with iWARP-enabled Intel 10GbE adapters.
A full implementation of currently available iWARP technology helps avoid virtually all processor networking overhead—bypassing the kernel, avoiding intermediate buffer copies, and accelerating transport (TCP/IP) processing to help reduce processor overhead, memory bandwidth utilization, and latency. Using mainstream Ethernet connectivity for compute clusters can now provide highly favorable performance, efficiency, and scalability, enabling organizations to obtain the low latency that HPC clusters need while capitalizing on the ease of use and familiarity of Ethernet. One 4,032-core cluster based on Dell™ PowerEdge™ servers with iWARP-enabled Intel® 10GbE server cluster adapters ranked at number 208 on the June 2010 TOP500 list, and reranking by efficiency shows that this cluster comes in at number 84—a level more efficient than many InfiniBand clusters, and the highest of the listed Ethernet solutions.
Using Ethernet connectivity as a unified fabric for cluster interconnects, LANs, and storage can help lower total cost of ownership by significantly reducing the number of switches and cables required. As 10GbE products and technology—including switches with high port density and technologies to further drive down latency—continue to evolve, future work can provide additional value in building supercomputing platforms on Dell PowerEdge servers interconnected with iWARP-enabled Intel 10GbE adapters.