Conda on ARC

Background

Conda is a tool for managing and deploying applications, environments and packages.

Brief help message

$ conda --help
usage: conda [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

Options:

positional arguments:
  command
    clean        Remove unused packages and caches.
    config       Modify configuration values in .condarc. This is modeled
                 after the git config command. Writes to the user .condarc
                 file (/home/drozmano/.condarc) by default.
    create       Create a new conda environment from a list of specified
                 packages.
    help         Displays a list of available conda commands and their help
                 strings.
    info         Display information about current conda install.
    init         Initialize conda for shell interaction. [Experimental]
    install      Installs a list of packages into a specified conda
                 environment.
    list         List linked packages in a conda environment.
    package      Low-level conda package utility. (EXPERIMENTAL)
    remove       Remove a list of packages from a specified conda environment.
    uninstall    Alias for conda remove.
    run          Run an executable in a conda environment. [Experimental]
    search       Search for packages and display associated information. The
                 input is a MatchSpec, a query language for conda packages.
                 See examples below.
    update       Updates conda packages to the latest compatible version.
    upgrade      Alias for conda update.

optional arguments:
  -h, --help     Show this help message and exit.
  -V, --version  Show the conda version number and exit.

conda commands available from other packages:
  build
  convert
  debug
  develop
  env
  index
  inspect
  metapackage
  render
  server
  skeleton
  verify

Installing Conda

You can install a local copy of miniconda in your home directory on our clusters. It will give you flexibility to install packages needed for the workflow.

Before installing Conda, please review the article about managing software on ARC: in your personal home directory

Here are the steps to follow:

Once connected to the login node, in your SSH session, make sure you are in your home directory:

$ cd 

Create a "software" subdirectory for all custom software you are going to have:

$ mkdir software; cd software 

Download the software the latest Miniconda distribution file:

$wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Install the downloaded .sh file:

$ bash Miniconda3-latest-Linux-x86_64.sh

Follow the instructions (choosing ~/software/miniconda3 as the directory to create), agree to the license, decline the offer to initialize.

Every time you launch a new terminal and want to use this version of python, set the path as follows

$ export PATH=/home/<username>/software/miniconda3/bin:$PATH

Ensure it is using the python installed in your home directory

$ which python 
~/software/miniconda3/bin/python

Create a virtual environment for your project

$ conda create -n <yourenvname>

Install additional Python packages to the virtual environment

$ conda install -n <yourenvname> [package]

Activate the virtual environment

$ source activate <yourenvname>

At this point you should be able to use your own python with the modules you added to it.