Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device. Atlassian cookies and tracking notice, (opens new window)
User Manual

User Manual
Results will update as you type.
  • Application Guide
  • Status of System
  • Usage Guide
  • Compute partitions
  • Software
    • AI Frameworks and Tools
    • Bring your own license
    • Chemistry
    • Data Manipulation
      • AEC library
      • CDO
      • ECCODES
      • HDF5 libraries / binaries
      • libtiff
      • libz
      • NCO
      • netCDF
        • Install netCDF
        • Linking with the netcdf library
          • Example netcdf program
      • pigz
      • PROJ
      • R
      • Szip
      • UDUNITS2
    • Engineering
    • Environment Modules
    • Miscellaneous
    • Numerics
    • Virtualization
    • Devtools Compiler Debugger
    • Visualisation Tools
  • FAQ
  • NHR Community
  • Contact

    You‘re viewing this with anonymous access, so some content might be blocked.
    /
    Linking with the netcdf library

    Linking with the netcdf library

    Juli 28, 2020

    Problem

    After linking a program with a netcdf library some users report error messages like this:

      ./nctest: error while loading shared libraries: libnetcdff.so.7: cannot open shared object file: No such file or directory
    

    One my find this error message by searching the web. Hence, one is in good company. Here we present methods to use the tools available at HLRN to produce running binaries. First of all an important statement:

    Neither the netcdf nor the hdf5 module have any effect on your binary during runtime. It is useless to load them before running a program.

    The following example program can be used for testing only. Don't ask for a sophisticated science background. The example uses FORTRAN-90 notation.

    download the example

    To compile the code, the netcdf data- and function types must be included by a use statement. The compiler needs to know, where the fortran-modules are located. Instead of digging into installation details, we use the result of the script nf-config, which is part of the netcdf-suite. To make it available, load a netcdf module:

       module load gcc/8.3.0
       module load netcdf/gcc.8/4.7.3
    

    Note, compiler version and netcdf version must fit, otherwise the fortran netcdf module cannot be included. We can check this with

       nf-config --fc
    

    fortran .mod files from different major gfortran versions are incompatible.


    Dynamic linking with the fortran netcdf library

    Serial programms

    Now you can compile

       gfortran  -c -I`nf-config --includedir` test.f90
    

    Instead of digging even more into the installation details, we use the result of nf-config in the link step:

       gfortran -o nctest *.o `nf-config --flibs` -lm
    

    To understand the details, just issue the commands nf-config --includedir and nf-config --flibs separately.

    Now you may unload both, the compiler and the netcdf module as well. The binary runs anyway, since the path to the netcdf- and hdf5- libraries is stored in the binary. This information was extracted from the environment variable LD_RUN_PATH that is set when loading a netcdf module. Note, the path to the hdf5 libraries linked to the netcdf library is inherited by our binary. The effect of LD_RUN_PATH is broken, if the linker option rpath is used for other purposes. Say, your example program links another library located in /mybla. Checkout

       gfortran -o nctest -Wl,-rpath=/mybla *.o `nf-config --flibs` -lm  
       ldd nctest
    

    The netcdf-libraries as well as the compiler specific libraries are not found any more. Here we notice, the compiler module also uses LD_RUN_PATH to set the path to the compiler specific libraries. So handle LD_RUN_PATH with care not to disturb this important feature! The correct way would be:

       gfortran -o nctest -Wl,-rpath="/mybla:$LD_RUN_PATH" *.o `nf-config --flibs` -lm  
    

    Hint, you may use readelf to see more details on the internal binary structure. Watch out for the library sections.

    MPI programms

    Now we need to use the compiler wrappers to include mpi.h and link the mpi libraries automatically. Here, rpath is used internally, which breaks the automatic propagation of the information of the paths, where the netcdf library resides. This concerns also the location of compiler specific libraries like libgcc_s.so.1 or libquadmath.so.0 . From that there may arise problems, since libquadmath.so.0 coming with gcc.9 contains more symbols than the system version. (This allows usage of the netcdf library build with gcc.8 together with gcc.9 .) Hence, we have to use the rpath option to provide the information on the library path and propagate it to the linker. Load the module files:

      module load gcc/9.2.0
      module load netcdf/gcc.8/4.7.3
      module load openmpi/gcc.9/3.1.5
    

    and check LD_RUN_PATH:

      echo $LD_RUN_PATH
      /sw/comm/openmpi/3.1.5/skl/gcc/lib:/sw/dataformats/netcdf/gcc.8.3.0/4.7.3/skl/lib:/sw/compiler/gcc/9.2.0/skl/lib64/
    

    Compile:

      mpifort  -c -I`nf-config --includedir` test.f90
      mpifort -o nctest *.o `nf-config --flibs` -Wl,-rpath=$LD_RUN_PATH -lm
    

    To check the true content of the binary, unload the modules first

      module unload openmpi/gcc.9/3.1.5
      module unload gcc/9.2.0
      module unload netcdf/gcc.8/4.7.3
      ldd nctest
    

    With readelf -a more details can be explored:

      readelf -a nctest | less
    ....
    Dynamic section at offset 0x2d58 contains 37 entries:
      Tag        Type                         Name/Value
     0x0000000000000001 (NEEDED)             Shared library: [libnetcdff.so.7]
     0x0000000000000001 (NEEDED)             Shared library: [libnetcdf.so.15]
     0x0000000000000001 (NEEDED)             Shared library: [libgfortran.so.5]
     0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
     0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
     0x0000000000000001 (NEEDED)             Shared library: [libmpi_usempif08.so.40]
     0x0000000000000001 (NEEDED)             Shared library: [libmpi_usempi_ignore_tkr.so.40]
     0x0000000000000001 (NEEDED)             Shared library: [libmpi_mpifh.so.40]
     0x0000000000000001 (NEEDED)             Shared library: [libmpi.so.40]
     0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
     0x0000000000000001 (NEEDED)             Shared library: [libquadmath.so.0]
     0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
     0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
     0x000000000000001d (RUNPATH)            Library runpath: [/sw/comm/openmpi/3.1.5/skl/gcc/lib:/sw/dataformats/netcdf/gcc.8.3.0/4.7.3/skl/lib:/sw/compiler/gcc/9.2.0/skl/lib64/:/sw/tools/hwloc/1.1.13/skl/lib]
    ... 
    


    Static linking with the fortran netcdf library

    There is no simple way to find all required libraries, but information on the libraries can be gathered from the file libnetcdff.la made by libtool for dynamic linking.

       NETCDF_LIB=`nc-config --libdir`
       cat $NETCDF_LIB/libnetcdff.la | grep dependency_libs 
    

    We configure the paths to the relevan libraries by hand:

       NETCDF_LIB=`nc-config --libdir`
       HDF5_LIB=/sw/dataformats/hdf5/1.8.20/skl/gcc.8.2.0.hlrn/lib
       SZIP_LIB=/sw/dataformats/szip/2.1/skl/gcc.8.2.0.hlrn/lib
       
       LIBS="$NETCDF_LIB/libnetcdff.a $NETCDF_LIB/libnetcdf.a $HDF5_LIB/libhdf5_hl.a  $HDF5_LIB/libhdf5.a $SZIP_LIB/libsz.a -lpthread -lcurl -lz -lm -ldl"
    

    Now compile and link

      gfortran -fPIC -c -I`nf-config --includedir` -Bstatic test.f90
      gfortran -o nctest *.o $LIBS
    

    and have fun.

    , multiple selections available, Use left or right arrow keys to navigate selected items
    kb-how-to-article
    rpath
    {"serverDuration": 11, "requestCorrelationId": "cb2d6cc01248429ba4da582e27bdbfca"}