Singularity

Singularity is a container system designed for use on High Performance Computing (HPC) systems. It allows users to run both Docker and Singularity containers.

From the Docker website: "A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings."

Availability and Restrictions

Versions

Singularity is available on Ruby, Owens and Pitzer clusters. Only one version is available at any given time. To find out the current version:

singularity version

Check the release page for the changelog: https://github.com/sylabs/singularity/releases

Access

Singularity is available to all OSC users.

Publisher/Vendor/Repository and License Type

SyLabs, Inc., 3-clause BSD License

Usage

Set-up

No setup is required. You can use Singulairty directly on all clusters.

Using Singularity

See HOWTO: Use Docker and Singularity Containers at OSC for information about using Singularity on Ruby, Owens and Pitzer clusters, including some site-specific caveats.  

Example:  Run a container from the Singularity hub

[owens-login01]$ singularity run shub://singularityhub/hello-world
INFO:    Downloading library image
Tacotacotaco

Workshop

Known Issues

Failed to pull a large Docker image

Update: 05/21/2019 
Version: all

You might encounter an error while pulling a large Docker image:

[owens-login01]$ singularity pull docker://qimme2/core
FATAL: Unable to pull docker://qiime2/core While running mksquashfs: signal: killed

The process could killed because the image is cached in the home directory which is a slower file system or the image size might exceed a single file size limit.

The solution is to use other file systems like /fs/scratch and $TMPDIR for caches, squashfs temp files and download

[owens-login01]$ qsub -I -l nodes=1:ppn=1
[o0001]$ cd $TMPDIR
[o0001]$ export SINGULARITY_CACHEDIR=$TMPDIR
[o0001]$ export SINGULARITY_TMPDIR=$TMPDIR
[o0001]$ singularity pull docker://qiime2/core:2019.1
[o0001]$ qiime2_core_2019.1.sif /where/to/keep/image/

Failed to run a container directly or pull an image from Singularity or Docker hub

Update: 03/08/2019 
Version: all

You might encounter an error while run a container directly from a hub:

[owens-login01]$ singularity run shub://vsoch/hello-world
Progress |===================================| 100.0%
FATAL: container creation failed: mount error: can't mount image /proc/self/fd/13: failed to find loop device: could not attach image file too loop device: No loop devices available

One solution is to remove the Singularity cached images from local cache directory $HOME/.singularity/cache. If the singularity version is prior to 3.0:

[owens-login01]$ cd $HOME/.singularity/cache
[owens-login01 cache]$ rm -rf *

If the singularity version is 3.1 and above:

[owens-login01]$ singulariy cache clean -a

Alternatively, you can change the Singularity cache directory to different location by setting the variable SINGULARITY_CACHEDIR. For example, in a batch job:

#PBS -N singularity_teset
#PBS -l nodes=1:ppn=1
#PBS -j oe

export SINGULARITY_CACHEDIR=$TMPDIR
singularity run shub://vsoch/hello-world

Further Reading

Supercomputer: