/
InfiniBand network

InfiniBand network

NHR Infiniband Lewin.png

Notes:

  • Both, the GPU (green) and Genoa (red) sub-network have a Fat Tree topology.

  • 2 spine switches (grey) are linked to 6 storage gateways of the “IBM Spectrum Scale” (GPFS)https://nhr-zib.atlassian.net/wiki/spaces/PUB/pages/426848: home, scratch, sw.

  • A pair of two Genoa compute nodes share one 400Gb/s NIC. Hence, per node the effective bandwidth is 200Gb/s (like in the GPU cluster).0

  • 14 LNET gateway servers provide access to the OPA network of the CLX cluster.

  • Mellanox (acquired by Nvidia) designs/manufactures InfiniBand components.