Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device. Atlassian cookies and tracking notice, (opens new window)
/
PVC MPI Usage
Updated März 26, 2024

PVC MPI Usage

For the PVC partition, Intel MPI is the preferred GPU-aware MPI implementation. Load an impi environment module, to make it available.

To enable GPU support, set the environment variable I_MPI_OFFLOAD to "1" (in your jobscript). In case you make use of GPUs on multiple nodes, it is strongly recommended to use the psm3 libfabric provider (FI_PROVIDER=psm3)

Depending on your application’s needs set I_MPI_OFFLOAD_CELL to either tile or device to assign each MPI rank either a tile or the whole GPU device.

It is recommended to check the pinning by setting I_MPI_DEBUG to (at least) 3 and I_MPI_OFFLOAD_PRINT_TOPOLOGY to 1.

Refer to the Intel MPI documentation on GPU support for further information.

Example Job Script:

#!/bin/bash # example to use use 2 x (2 x 4) = 16 MPI processes, each assigned # to one of the two tiles (stacks) of an PVC GPU #SBATCH --partition=gpu-pvc #SBATCH --nodes=2 #SBATCH --ntasks-per-node=8 #SBATCH --job-name=pin-check # required for usage of Intel GPUs module load intel # required for MPI, apparently module load impi/2021.11 # required for GPU usage with MPI export FI_PROVIDER=psm3 # to enable GPU support in Intel MPI export I_MPI_OFFLOAD=1 # assign each rank a tile of a GPU export I_MPI_OFFLOAD_CELL=tile # for checking the process pinning export I_MPI_DEBUG=3 export I_MPI_OFFLOAD_PRINT_TOPOLOGY=1 mpirun ./application

 

User Manual

User Manual
Results will update as you type.
  • Application Guide
  • Status of System
  • Usage Guide
  • Compute partitions
    • CPU CLX partition
    • CPU Genoa partition
    • GPU A100 partition
    • GPU PVC partition
      • PVC AI Tools and Frameworks
      • PVC GPU Programming
      • PVC MPI Usage
      • Interactive PVC
    • Next-Gen Technology Pool
  • Software
  • FAQ
  • NHR Community
  • Contact

    You‘re viewing this with anonymous access, so some content might be blocked.
    {"serverDuration": 9, "requestCorrelationId": "30d2552a2a444de697da406ded29a1f8"}