Anaconda (conda) and Mamba
Anaconda package manager (predominantly Python) and mambaforge
Description
Anaconda is a package manager commonly used for Python. mamba (documentation) is a compatible package management tool that is designed to be faster and has a few extra features
Prerequisites
None
Modules
The anaconda3
module. See Python for the available versions. To load version 2020.11 for Python 3, you would do
module load anaconda3/2020.11
Now, in order to use it, you need to source the anaconda init shell script. You can either always run
source $CONDASH
after loading the module, or if you want to only use one version of anaconda3
and have it available every time you log in, you can enter
conda init SHELL
where SHELL
is the name of the shell that you use, which will most likely be bash
, tcsh
, or zsh
.
Creating And Loading Environments
conda (and mamba) work with environments that can have their own packages completely separate from each other. When you create an environment, you can specify any packages you want to install (note, you can install more later). For example, it is common to specify the exact Python version you want in case you don't want the default. To create the environment FOO
with Python 3.8 package along with various other packages, you would use
conda create -n FOO python=3.8 PACKAGE1 PACKAGE2 ...
To enter an environment (make it the active on in use), you activate it like
conda activate FOO
To leave the environment, you deactivate it like
conda deactivate
Setting up Mamba
This section introduces how to set up mamba (documentation). It was kindly provided by our GPU user Anwai Archit, which is lightly edited to fit the format of this documentation and the first item removed (pointed to instructions to setup conda).
- To download
mambaforge
, follow these instructions (also listed below):Download mamba-forge
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
Activate mamba-forge (to override a previous installation, just add
-u
)bash Mambaforge-Linux-x86_64.sh
- Accept the license
- Important: Choose your scratch directory for installation, for example:
/scratch/usr/<USERNAME>/mambaforge
. The space in/home/<USERNAME>
directory is not sufficient for large environments. - At the end of installation, type
yes
to initialize mamba-forge. - After following the steps, your conda/mamba installation is available on the cluster (automatically activated when you get a new shell) (if not, see next point)
To enable the necessary configuration for your environment and activate the `base` environment,
source ~/.bashrc
- Use mamba commands to create & activate environments, and install packages.
Use
mamba create
for creating new environments. For example,mamba create -n test numpy
will create a new environment calledtest
and install numpy into it.Once created, you can activate this new environment using
mamba activate test
and then install new packages usingmamba install
If the package you want to install is not available from the default repository, you can add custom repositories (called "channels") by adding
-c <CHANNEL_NAME>
to the command. For example,mamba install -c pytorch pytorch torchvision
will install the packagespytorch
andtorchvision
from thepytorch
channel.You can also use
pip
, the most commonly used python package manager, to install packages to the environments (Recommendation - preferrably install packages using conda/mamba). Just activate the correct environment and runpip install <PACKAGE>
- For more details on how to use conda/mamba, check out the conda tutorial.