Versionen im Vergleich

Schlüssel

  • Diese Zeile wurde hinzugefügt.
  • Diese Zeile wurde entfernt.
  • Formatierung wurde geändert.


Auszug

an object-oriented open source CFD toolkit

...

Typically, OpenFOAM causes a lot of meta data operations. This default behavior jams no only your job but may slow down the shared parallel file system (=Lustre) for all other users. Also, your job is interrupted if the inode limit (number of files) of the quota system (show-quota) is exceeded.

If you can not use our local 2TB-SSDs (see Special Filesystems) #SBATCH --partition={standard,large,huge}96:ssd at $LOCAL_TMPDIR please refer to our general advices to reduce Metadata Usage on WORK (=Lustre).

To adapt/optimize your OpenFOAM job specifically for I/O operations on $WORK (=Lustre) we strongly recommend the following steps:

  • Always, to avoid that each processor writes in its own file please use collated file I/O.
    This feature was released 2017 for all OpenFOAM versions.
    [ESI www.openfoam.com/releases/openfoam-v1712/parallel.php]
    [Foundation www.openfoam.org/news/parallel-io]

    Codeblock
    OptimisationSwitches
    {
        fileHandler collated; // all processors share a file
    }

    to the /.OpenFOAM/v#/controlDict file or per-case override in the $FOAM_CASE/system/controlDict file.

  • Always, set

    Codeblock
    runTimeModifiable false;

    to reduce I/O activity. Only set "true" (default), if it is strictly necessary to re-read dictionaries (controlDict, ...) each time step.

  • Possibly, do not save every time step:
    [www.openfoam.com/documentation/guides/latest/doc/guide-case-system-controldict.html]
    [www.cfd.direct/openfoam/user-guide/v6-controldict]

    Codeblock
    writeControl	timeStep;
    writeInterval	100;


  • Possibly, save only the latest n time steps (overwrite older ones), such as:

    Codeblock
    purgeWrite	1000;


  • Typically, only a subset of variables is needed frequently (post-processing). The full set of variables can be saved less frequently (e.g., restart purposes). This can be achieved with [https://wiki.bwhpc.de/e/OpenFoam]:

    Codeblock
    writeControl    clockTime;
    writeInterval   21600; // write ALL variables every 21600 seconds = 6 h
    
    functions
    {
        writeFields
        {
            type writeObjects;
            libs ("libutilityFunctionObjects.so");
    
            objects
            (
    	    T U // specified variables
            );
    
            outputControl timeStep;
            writeInterval 100; // write specified variables every 100 steps
        }
    }


  • In case your HLRN run job accidentally generated thousands of small files, please pack them (at least the small-size metadata files) into a single file afterwards:

    Codeblock
    tar -xvzf singlefile.tar.gz -C /folder/subfolder/location/

    Thanks a lot for your contribution making HLRN the compute systems a great place for all...

Compiling Own Code Over OpenFOAM

...