Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device. Atlassian cookies and tracking notice, (opens new window)
User Manual

User Manual
Results will update as you type.
  • Application Guide
  • Status of System
  • Usage Guide
  • Compute partitions
    • CPU CLX partition
    • CPU Genoa partition
    • GPU A100 partition
      • Slurm partition GPU A100
      • OpenMP for GPU A100
      • CUDA
      • Apptainer
    • GPU PVC partition
    • Next-Gen Technology Pool
  • Software
  • FAQ
  • NHR Community
  • Contact

    You‘re viewing this with anonymous access, so some content might be blocked.
    /
    CUDA

    CUDA

    Juni 26, 2024

    CUDA (Compute Unified Device Architecture) is an interface to program Nvidia GPUs. It offers support to the languages such as C, C++, and Fortran.

    To build and execute code on the GPU A100 partition, please login to

    • a GPU A100 login node, like bgnlogin.nhr.zib.de.
    • see also GPU A100 partition

    Note, that codes written in the cross-industry standard language SYCL can be executed on Nvidia (and AMD) hardware.

    Code build

    For code generation we recommend the software package NVIDIA hpcx which is a combination of compiler and powerful libraries, like e.g. CUDA, cublas, and MPI.

    CUDA and with cublas
    bgnlogin1 $ module load nvhpc-hpcx/23.1
    bgnlogin1 $ module list
    Currently Loaded Modulefiles: ... 4) hpcx   5) nvhpc-hpcx/23.1
    bgnlogin1 $ nvc -cuda -gpu=cc8.0 cuda.c -o cuda.bin
    bgnlogin1 $ nvc -cuda -gpu=cc8.0 -cudalib=cublas cuda_cublas.c -o cuda_cublas.bin

    CUDA can be used in combination with MPI.

    CUDA with MPI
    bgnlogin1 $ module load nvhpc-hpcx/23.1
    bgnlogin1 $ mpicc -cuda -gpu=cc8.0 -cudalib=cublas mpi_cuda_cublas.c -o mpi_cuda_cublas.bin

    Code execution

    All available slurm partitions for the A100 GPU partition you can see on Slurm partition GPU A100.

    Job script for CUDA
    #!/bin/bash
    #SBATCH --partition=gpu-a100:shared
    #SBATCH --gres=gpu:1
    #SBATCH --nodes=1
    #SBATCH --ntasks-per-node=72
    
    ./cuda.bin
    ./cuda_cublas.bin
    
    Job script for CUDA with MPI
    #!/bin/bash
    #SBATCH --partition=gpu-a100
    #SBATCH --gres=gpu:4
    #SBATCH --nodes=2
    #SBATCH --ntasks-per-node=72
    
    module load nvhpc-hpcx/23.1
    mpirun --np 8 --map-by ppr:2:socket:pe=1 ./mpi_cuda_cublas.bin
    
    , multiple selections available,
    {"serverDuration": 10, "requestCorrelationId": "8f8ad1efdb994f1ba8c3dd627123d26c"}