...

The migration to the new OS is organised in three consecutive phases. It is expected to be complete by the end of July September.

1. The first phase starts with 2 login nodes and 112 compute nodes already migrated to Rocky Linux 9 for testing. The other nodes remain available under CentOS 7 for continued production.

2. After the test phase, a major fraction of nodes will be switched to Rocky Linux 9 to allow for general job production under the new OS.

3. During the last phase, only a few nodes still remain under CentOS 7. At the very end, they will be migrated to Rocky Linux 9, too.

During the migration phase the use of Rocky Linux 9 "clx" compute nodes will be was free of charge.

Current migration state: complete

Latest news

...

( ● available ● closed/not available yet )

Jobs submitted without a partition name are placed in the default partition. The old default was standard96, the new default is cpu-clx.

Software and environment modules

...

• node hardware and node names

• communication network (Intel Omnipath)

• file systems (HOME, WORK, PERM) and disk quotas

• environment modules system (still based on Tcl, a.k.a. “Tmod”)

• access credentials (user IDs, SSH keys) and project IDs

• charge rates and CPU time accounting (early migrators' jobs are were free of charge)

• Lise’s Nvidia-A100 and Intel-PVC partitions

...

• For users of SLURM’s srun job launcher:
  Open MPI 5.x has dropped support for the PMI-2 API, it solely depends on PMIx to bootstrap MPI processes. For this reason the environment setting was changed from SLURM_MPI_TYPE=pmi2 to SLURM_MPI_TYPE=pmix, so binaries linked against Open MPI can be started as usual “out of the box” using srun mybinary. For the case of a binary linked against Intel-MPI, this works too when a recent version (≥2021.11) of Intel-MPI has been used. If an older version of Intel-MPI has been used, and relinking/recompiling is not possible, one can follow the workaround for PMI-2 with srun as described in the Q&A section below. Switching from srun to mpirun instead should also be considered.

• Using more processes per node than available physical cores (PPN > 96; hyperthreads) with the OPX providerwhen defining FI_PROVIDER=opx:
  The OPX provider currently does not support using hyperthreads/PPN > 96 on the clx partitions. Doing so may result in segmentation faults in libfabric during process startup. If a high number of PPN is really required, the libfabric provider has to be changed back to PSM2 by setting re-defining FI_PROVIDER=psm2(which is the default setting). Note that the usage of hyperthreads may not be advisable. We encourage users to test performance before using more threads than available physical cores.
  Note that Open MPI’s mpirun/exec defaults to use all hyperthreads if a Slurm job/allocation is used that does not explicitely sets explicitly set --ntasks-per-node (or similar options).

...