HOWTO: Use Docker and Apptainer/Singularity Containers at OSC

It is now possible to run Docker and Apptainer/Singularity containers on all clusters at OSC. Single-node jobs are currently supported, including GPU jobs; MPI jobs are planned for the future.

From the Docker website:  "A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings."

As of June 21st, 2022, Singularity is replaced with Apptainer, which is just a renamed open-source project. For more information visit the Apptainer/Singularity page

This document will describe how to run Docker and Apptainer/Singularity containers on OSC clusters. You can use containers from Docker Hub, Sylabs Cloud, or any other source. As examples we will use hello-world and ubuntu from Docker Hub.

If you encounter an error then check the Known Issues on using Apptainer/Singularity or Podman at OSC. If the issue can not be resolved, please contact OSC help. 

Contents

1. Getting help
2. Setting up your environment
3. Accessing a container
4. Running a container
5. File system access
6. GPU usage within a container
7. Build a container
8. References

Getting help

• For Apptainer/Singularity, use apptainer help.

• For Podman/Docker, use podman help.

• User guides and examples are available at Podman documentation and Apptainer documentation.

Setting up your environment for Podman or Apptainer/Singularity usage

No setup is required. You can use Podman or Apptainer/Singularity directly on all clusters.

Accessing a container

A container image is a file (e.g. .sif for Apptainer) or image stored in a registry (for Docker/Podman).

You can pull images from hubs: Docker Hub, Sylabs Cloud, or other registries.

Using Apptainer/Singularity

Examples:

# Pull the gcc:7.2.0 image from Docker Hub → gcc_7.2.0.sif
apptainer pull docker://gcc:7.2.0

# Ubuntu 18.04 → ubuntu_18.04.sif
apptainer pull docker://ubuntu:18.04

# Pull from Singularity Hub
apptainer pull shub://singularityhub/hello-world

Downloading Apptainer/Singularitycontainers from the hubs is not the only way to get one. You can, for example get a copy from your colleague's computer or directory. If you would like to create your own container you can start from the Build a container section below. If you have any questions, please contact OSC Help.

Using Podman/Docker

With Podman/Docker, you pull images to your local image store:

podman pull ubuntu:18.04
podman pull docker.io/library/gcc:7.2.0

Use podman images to list available images in the local registry:

REPOSITORY                TAG         IMAGE ID      CREATED      SIZE
docker.io/library/ubuntu  18.04       f9a80a55f492  2 years ago  65.5 MB
docker.io/library/gcc     7.2.0       81ffb25b1dec  7 years ago  1.73 GB

Running a container

You can run containers on OSC clusters either interactively or in batch jobs.

We note that the operating system on OSC cluster is Red Hat by running cat /etc/os-release:

NAME="Red Hat Enterprise Linux Server"
[..more..]

In the examples below we will often check the operating system to show that we are really inside a container.

Using Apptainer/Singularity

Run container like a native command

If you simply run the container image, it will execute the container’s runscript.

apptainer pull docker://hello-wolrd
./hello-world_latest.sif

You should see the following output:

Hello from Docker!
This message shows that your installation appears to be working correctly.

Use the “run” sub-command

The Apptainer “run” sub-command does the same thing as running a container directly as described above. That is, it executes the container’s runscript.

Example:  Run a container from a local file

apptainer run hello-world_latest.sif

Example:  Run a container from a hub without explicitly downloading it

apptainer run docker://hello-world

Use the “exec” sub-command

The Apptainer “exec” sub-command lets you execute an arbitrary command within your container instead of just the runscript.

Example:  Find out what operating system the ubuntu:18.04 container uses

apptainer pull docker://ubuntu:18.04
apptainer exec ./ubuntu_18.04.sif cat /etc/os-release

You should see the following output:

NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"

Use the “shell” sub-command

The Apptainer “shell” sub-command invokes an interactive shell within a container.

Example:  Run an Ubuntu shell.

apptainer shell ubuntu_18.04.sif

You should now see the prompt Apptainer>, indicating that you are logged into the container. You can verify the operating system version by running:

Apptainer> cat /etc/os-release

Output:

NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"

To exit the container, simply type exit.

Using Podman/Docker

With Podman or Docker:

• Run: runs the container’s default command

podman run hello-world

• Exec: execute arbitrary command inside running (or started) container

podman run -t -d --name my_ubuntu ubuntu:18.04
podman exec my_ubuntu cat /etc/os-release

• Interactive shell:

podman run -it ubuntu:18.04

File system access

Using Apptainer/Singularity

When you use a container you run within the container’s environment. The directories available to you by default from the host environment are

• your home directory
• working directory (directory you were in when you ran the container)
• /fs/ess
• /tmp

You can review our Available File Systems page for more details about our file system access policy.

If you run the container within a job you will have the usual access to the $PFSDIR environment variable with adding node attribute "pfsdir" in the job request (--gres=pfsdir). You can access most of our file systems from a container without any special treatment.

Using Podman/Docker

If using Podman/Docker, you may need to explicitly bind mount host directories into the container. For example:

podman run -it -v $HOME:$HOME -v /fs/ess:/fs/ess $ubuntu:18.04

GPU usage within a container

Using Apptainer/Singularity

If you have a GPU-enabled container you can easily run it on Pitzer just by adding the --nv flag to the apptainer exec or run command.  The example below comes from the "exec" command section of Apptainer User Guide.  It runs a TensorFlow example using a GPU on Pitzer.  (Output has been omitted from the example for brevity.)

[pitzer-login01]$ salloc -n 4 -G 1
...
[p0756]$ git clone https://github.com/tensorflow/models.git
[p0756]$ apptainer exec --nv docker://tensorflow/tensorflow:latest-gpu \
python ./models/tutorials/image/mnist/convolutional.py

In some cases it may be necessary to bind the CUDA_HOME path and add $CUDA_HOME/lib64 to the shared library search path:

[pitzer-login01]$ salloc -n 4 -G 1
...
[p0756]$ module load cuda
[p0756]$ export APPTAINER_BINDPATH=$CUDA_HOME
[p0756]$ export APPTAINERENV_LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CUDA_HOME/lib64
[p0756]$ apptainer exec --nv my_container mycmdn 

Using Podman/Docker

To use a GPU in a Docker container, you need to add the GPU device using the --device option.

For example, to request a GPU node with one GPU:

salloc -n 1 -G 1

After obtaining the node, you can test if the GPU device is available in a container by running:

podman run --rm --device nvidia.com/gpu=all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi

If successful, the nvidia-smi command will display details about the GPU, such as model, memory usage, and driver version.

Build a container

Using Apptainer/Singularity

OSC users can now build an Apptainer/Singularity container image from a definition file using the fakeroot feature:

apptainer build --fakeroot myimage.sif mydef.def

When building an image, it is recommended to change the cache folder (the default location is $HOME/.apptainer/cache) to improve build efficiency and avoid potential file system issues, especially when building a large container image.

Recommended steps

Request a compute node with sufficient memory. The following example requests a compute node with 32 GB of memory:

sinteractive -n 8

Change the cache folder to a temporary file system:

export APPTAINER_CACHEDIR=$TMPDIR

Build an image:

apptainer build --fakeroot myimage.sif mydef.def

These steps help optimize performance and prevent file system issues when building large container images.

Using Podman/Docker

OSC users can now build a Docker container image from a Dockerfile:

podman build -f Dockerfile --format docker -t tag_my_container .

The option --format docker ensures that the container format is compatible with the Docker schema for the manifest. The -t flag is used to tag the image, typically in the format name:version.

For example, if you set -t my_container:1.0, you should see the following after listing images with podman images:

REPOSITORY                TAG         IMAGE ID      CREATED      SIZE
local/my_container        1.0         f9a80a55f492  30 mins ago  65.5 MB

Note that our local registry is saved on a local disk. This means that if you build an image on a compute node, it will not be accessible from other nodes and will be removed once the job ends.

To ensure that your container image can be reused, you should create an account on Docker Hub (or another supported container registry) and tag your image with the registry URL. For example:

podman login
podman tag my_container:1.0 docker.io/<username>/my_container:1.0
podman push docker.io/<username>/my_container:1.0

References

• Ap\ptainer User Guide
• Singularity Tutorial
• Podan Documentation