...
WORK is the default shared file system for all jobs and can be accessed using the $WORK variable. WORK is accessible for all users and consists of 8 Metadata Targets (MDT's) with NVMe SSDs and 28 Object Storage Targets (OST's) on Lise and 96 100 OST's on Emmy handling the data. Both using classical hard drives.
...
Some workloads will benefit of striping. Files will be split between a number of 2 up OSTs.
Especially large shared file IO patterns will benefit from striping. Up to 28 OSTs on Lise and up to 96 100 OST's on Emmy can be used, recommeded are up to 8 OSTs for Lise and 32 OSTs on Emmy.
Especially large shared IO patterns will benefit from striping.
Access: create a new directory in $WORK and set lfs setstripe -c <stripsize> <dir>
...
| Lise: SSD | Lise: CAS | Emmy: SSD | |
|---|---|---|---|
| Access | via queue: using | via queue: using | via queue: using |
| Type and size | Intel NVMe SSD DC P4511 (2 TB) | Intel NVMe SSD DC P4511 (2 TB) using Intel Optane SSD DC P4801X (200 GB) as write-trough cache | Intel DC S4500 (400 GB) |
...
DDN Infinite Memory Engine (IME) based Burst Buffers and file system Cache is a fast NVMe Solid State Disk (SSD) based data tier between compute nodes and the WORK file system.
This helps avoid overload on the system when a program tries to write large amount of data within a short period of time to the global parallel file system.
IME servers consist of Solid State Disk (SSD) that acts as a cache and burst buffer to improve the global file system performance. . Like Lise's FastIO this is especially helpful for random IO (especially random read) on large files
Using the Burst Buffer for random helps to avoid overloading the global filesystem which results in slow job runtimes for all users. Beside the POSIX interface a native API and a MPI-IO module for further acceleration is available.
IME servers are currently available for use in EMMY.
Access: IME Burst Buffer, File System Cacheask support@hlrn.de for access
Size: 48 TiB
Finding the right File System
...