Anaconda package manager (predominantly Python) and mambaforge

Description

Anaconda is a package manager commonly used for Python. mamba (documentation) is a compatible package management tool that is designed to be faster and has a few extra features

Prerequisites

None

Modules

The anaconda3  module. See Python for the available versions. To load version 2020.11 for Python 3, you would do

module load anaconda3/2020.11

Now, in order to use it, you need to source the anaconda init shell script. You can either always run

source $CONDASH

after loading the module, or if you want to only use one version of anaconda3  and have it available every time you log in, you can enter

conda init SHELL

where SHELL  is the name of the shell that you use, which will most likely be bash , tcsh , or zsh .

Creating And Loading Environments

conda (and mamba) work with environments that can have their own packages completely separate from each other. When you create an environment, you can specify any packages you want to install (note, you can install more later). For example, it is common to specify the exact Python version you want in case you don't want the default. To create the environment FOO with Python 3.8 package along with various other packages, you would use

conda create -n FOO python=3.8 PACKAGE1 PACKAGE2 ...

To enter an environment (make it the active on in use), you activate it like

conda activate FOO

To leave the environment, you deactivate it like

conda deactivate

Setting up Mamba

This section introduces how to set up mamba (documentation). It was kindly provided by our GPU user Anwai Archit, which is lightly edited to fit the format of this documentation and the first item removed (pointed to instructions to setup conda).

1. To download mambaforge , follow these instructions (also listed below):

   1. Download mamba-forge

      wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh

   2. Activate mamba-forge (to override a previous installation, just add -u )

      bash Mambaforge-Linux-x86_64.sh

   3. Accept the license
   4. Important: Choose your scratch directory for installation, for example: /scratch/usr/<USERNAME>/mambaforge . The space in /home/<USERNAME>  directory is not sufficient for large environments.
   5. At the end of installation, type yes  to initialize mamba-forge.
   6. After following the steps, your conda/mamba installation is available on the cluster (automatically activated when you get a new shell) (if not, see next point)

   7. To enable the necessary configuration for your environment and activate the `base` environment,

      source ~/.bashrc

   8. Use mamba commands to create & activate environments, and install packages.

   9. Use

      mamba create

       
      for creating new environments. For example,

      mamba create -n test numpy

      
      will create a new environment called test  and install numpy into it.

   10. Once created, you can activate this new environment using

       mamba activate test

       
       and then install new packages using

       mamba install

   11. If the package you want to install is not available from the default repository, you can add custom repositories (called "channels") by adding -c <CHANNEL_NAME>  to the command. For example,

       mamba install -c pytorch pytorch torchvision

       
       will install the packages pytorch  and torchvision  from the pytorch channel.

   12. You can also use pip , the most commonly used python package manager, to install packages to the environments (Recommendation - preferrably install packages using conda/mamba). Just activate the correct environment and run

       pip install <PACKAGE>

2. For more details on how to use conda/mamba, check out the conda tutorial.