User installed software

You can install software that doesn't require sudo privileges in any of these storage locations,

• Your home directory, /hpc/home/$USER. It has a 50 GB limit, and conda environments can fill it quickly.
• A project directory, /data/<projectname>, if your project has one. This is the best choice for software a group shares, or when a project wants to strongly control software versions and packages. Project space is requested through your institution contact.

For software that is hard to build, or that needs system packages, consider a container instead.

Python with Miniforge

Use conda environments to manage Python versions and packages for both SSH and OnDemand. We recommend Miniforge, a minimal conda installer that uses the community conda-forge channel by default and includes mamba. Miniconda and Anaconda default to Anaconda's defaults channel, which has licensing terms for large organizations.

Install Miniforge

Log in to NCShare, or open an OnDemand shell, then set PREFIX to the install location you chose and run the installer,

PREFIX=$HOME/miniforge3   # or /data/<projectname>/$USER/miniforge3
mkdir -p "$(dirname "$PREFIX")"
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p "$PREFIX"
"$PREFIX/bin/conda" init bash
rm Miniforge3-Linux-x86_64.sh

conda init adds a block to your ~/.bashrc. Log out and log back in, and your prompt should start with (base).

If you already have Miniconda or Anaconda installed, you can install Miniforge alongside it and recreate your environments there. Remove the old conda block from ~/.bashrc so only one installation is initialized.

Create an environment

In the example below, myenv is the name of the environment and python=3.12 is the Python version.

conda create -n myenv python=3.12
conda activate myenv

Your prompt shows the active environment,

(myenv) $

With the environment active, install packages with conda install (or mamba install) or pip install. Each environment has its own pip, so packages installed with it stay inside that environment. Use conda deactivate to leave the environment.

For more information, see Managing Environments in the conda user guide.

Use the environment in a batch job

Batch scripts don't read ~/.bashrc, so conda activate is undefined until you load conda's shell functions. Either source conda's setup script directly,

#!/bin/bash
#SBATCH --partition=common
#SBATCH --mem=4G

source "$(conda info --base)/etc/profile.d/conda.sh"
conda activate myenv
python my_script.py

Or, since conda init added conda to your ~/.bashrc, source that instead of conda.sh,

source ~/.bashrc
conda activate myenv

Sourcing ~/.bashrc also runs anything else in it, such as aliases. Use conda.sh if you want the job environment to depend only on conda.

Use the environment in Jupyter

To make an environment available as a kernel in Jupyter Lab on OnDemand, see Using your own Python environment.

Troubleshooting

conda is not recognized after installing Miniforge

If conda init didn't run, run it again with the PREFIX you installed to, then log back in,

$HOME/miniforge3/bin/conda init bash