Skip to content

Cluster Software

Basic linux software is installed on the cluster for general use. For scientific computation users are welcome to install software in their /hpc/home/${USER} space if sudo access is not required. Most commonly, users may self install Conda to use Python and other supported languages.

For all other software we recommend building and deploying your container to NCShare.

Warning

Do not install software in the /work space as this is a temporary storage location where files older than 75 days will be purged automatically.

Conda / Python Install

Users are encouraged to install their own instance of Conda in their home directory. This allows users to manage their own Python environments and install packages as needed without requiring admin access.

To install Conda, login to the cluster either through SSH, Open OnDemand shell, or an Open OnDemand Jupyter Terminal and run the following commands from your home directory,

  1. Download the Miniconda installer,

    wget -O Miniforge3.sh "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"

  2. Run the installation script,

    bash Miniforge3.sh
    and and then follow the instructions. It will offer to update your ~/.bashrc, (init), say no if you use multiple environments. Say yes if you will only be using this environment.

  3. Restart your shell,

    exec bash

  4. Finally, delete your installation file to save space,

    rm Miniforge3.sh

Now you should be an run conda install, pip install, create environments, etc.

To learn about managing conda environments on Open OnDemand JupyterLab instances, please read the Managing Conda Environments on Open OnDemand JupyterLab instances guide.

Software Containers

NCShare provides the following containers for use on the cluster:

These are hosted on the github repository ncshare/Apptainer.

Adding containers to NCShare

Pre-built Docker and Singularity containers are often available within scientific repositories and on GitHub. Images can be moved to the cluster using the same methods as any other file.

!! info Because many images are quite large, we recommend storing them in /opt/apps/containers/user.

Most Docker containers are fully supported with singularity and can be run using Apptainer.

Useful Singularity Container repositories

Pre-built containers are also available in external repositories and can be loaded to the DCC using singularity pull or singularity build commands.

Creating your own apptainer containers

Singularity containers cannot be created on the DCC directly because you need root access to the build system. Generally to build a container, you will need root access to a Linux machine. If you do not have a Linux VM, you can obtain one through Duke Virtual Computing Manager, or lab groups may have access to a RAPID VM through their Faculty/PIs allocation. Helpful documentation: