Introduction to Singularity
In this section we're going to be working with Singularity Community Edition (CE)
Wait, what is "Apptainer", and what is the difference between SingularityCE and Apptainer?
The Singularity project was split into multiple projects managed by different organizations since it was created in 2017.
In a nutshell:
- Greg Kurtzer founded the Singularity project while at the Lawrence Berkeley National Laboratory
- Kurtzer created Sylabs, a private company, around Singularity
- Kurtzer left Sylabs to focus on CIQ, another private company, and moved Singularity to HPCng (a Community Org)
- Sylabs forked Singularity for control and professionially licensed support creating Singularity Community Edition"
- HPCng gave the official project to Linux Foundation and renamed it "Apptainer"
- Apptainer is being marketed by CIQ
At the present time, Apptainer and Singularity CE have highly similar syntax and will run Singularity
.sif images interoperably
Docker vs SingularityCE & Apptainer
Apptainer and SingularityCE are 100% compatible with Docker but they do have some distinct differences
Docker containers run as
- This privilege is almost never supported by administrators of High Performance Computing (HPC) centers. Meaning Docker is not, and will likely never be, installed natively on your HPC cluster.
uses compressed layers to create one image
SingularityCE & Apptainer:
Same user and group identity inside as outside the container
User only has
root privileges if elevated with
sudo when the container is run
Can run and modify any existing Docker image
- These key differences allow Singularity to be installed on most HPC centers. Because you can run virtually all Docker containers in Singularity, you can effectively run Docker on an HPC.
Sylabs Singularity Community Edition (CE) homepage: https://www.sylabs.io/docs/
Apptainer Linux Foundation homepage: https://apptainer.org/
To Install Singularity follow the instructions for your specific OS:
Module loading on HPC¶
If you are interested in working with SingularityCE on HPC, you may need to contact your systems administrator and request they install SingularityCE. Because SingularityCE ideally needs setuid, your admins may have some qualms about giving SingularityCE this privilege. If that is the case, you might consider forwarding this letter to your admins.
Most HPC systems are running Environment Modules with the simple command
You can check to see what is available:
If Singularity is listed as being installed, load a specific version, e.g.:
Install in CodeSpaces¶
Let's use Conda (or optionally, Mamba)
Singularity’s command line interface allows you to build and interact with containers transparently. You can run programs inside a container as if they were running on your host system. You can easily redirect IO, use pipes, pass arguments, and access files, sockets, and ports on the host system from within a container.
help command gives an overview of Singularity options and
subcommands as follows:
$ singularity help pull Pull an image from a URI Usage: singularity pull [pull options...] [output file] <URI> Description: The 'pull' command allows you to download or build a container from a given URI. Supported URIs include: library: Pull an image from the currently configured library library://user/collection/container[:tag] docker: Pull a Docker/OCI image from Docker Hub, or another OCI registry. docker://user/image:tag shub: Pull an image from Singularity Hub shub://user/image:tag oras: Pull a SIF image from an OCI registry that supports ORAS. oras://registry/namespace/image:tag http, https: Pull an image using the http(s?) protocol https://library.sylabs.io/v1/imagefile/library/default/alpine:latest Options: --arch string architecture to pull from library (default "amd64") --dir string download images to the specific directory --disable-cache dont use cached images/blobs and dont create them --docker-login login to a Docker Repository interactively -F, --force overwrite an image file if it exists -h, --help help for pull --library string download images from the provided library --no-cleanup do NOT clean up bundle after failed build, can be helpful for debugging --no-https use http instead of https for docker:// oras:// and library://<hostname>/... URIs Examples: From Sylabs cloud library $ singularity pull alpine.sif library://alpine:latest From Docker $ singularity pull tensorflow.sif docker://tensorflow/tensorflow:latest From Shub $ singularity pull singularity-images.sif shub://vsoch/singularity-images From supporting OCI registry (e.g. Azure Container Registry) $ singularity pull image.sif oras://<username>.azurecr.io/namespace/image:tag For additional help or support, please visit https://www.sylabs.io/docs/
Just like with Docker, you can
search the Singularity container registries for images.
The easiest way to use a Singularity is to
pull an existing container
from one of the Registries.
Downloading pre-built images¶
library://- images hosted on Sylabs Cloud
docker://- images hosted on Docker Hub
localimage://- images saved on your machine
yum://- yum based systems such as CentOS and Scientific Linux
debootstrap://- apt based systems such as Debian and Ubuntu
arch://- Arch Linux
zypper://- zypper based systems such as Suse and OpenSuse
shub://- (archived) images hosted on Singularity Hub, no longer maintained
Pulling an image from Singularity Hub¶
Similar to previous example, in this example I am pulling a base Ubuntu container from Singularity-Hub:
You can rename the container using the --name flag:
$ singularity pull --name ubuntu_test.simg shub://singularityhub/ubuntu WARNING: Authentication token file not found : Only pulls of public images will succeed 88.58 MiB / 88.58 MiB [===============================================================================================] 100.00% 35.12 MiB/s 2s
The above command will save the alpine image from the Container Library
Pulling an image from Docker Hub¶
This example pulls an
ubuntu:16.04 image from DockerHub and saves it
to the working directory.
$ singularity pull docker://ubuntu:20.04 INFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob 8f6b7df711c8 done Copying blob 0703c52b8763 done Copying blob 07304348ce1b done Copying blob 4795dceb8869 done Copying config 05ac933964 done Writing manifest to image destination Storing signatures 2020/03/09 16:14:12 info unpack layer: sha256:8f6b7df711c8a4733138390ff2aba1bfeb755bf4736c39c6e4858076c40fb5eb 2020/03/09 16:14:13 info unpack layer: sha256:0703c52b8763604318dcbb1730c82ad276a487335ecabde2f43f69a6222e8090 2020/03/09 16:14:13 info unpack layer: sha256:07304348ce1b6d24f136a3c4ebaa800297b804937a6942ce9e9fe0dac0b0ca74 2020/03/09 16:14:13 info unpack layer: sha256:4795dceb8869bdfa64f3742e1df492e6f31baf9cfc36f1a042a8f981607e99a2 INFO: Creating SIF file... INFO: Build complete: ubuntu_20.04.sif
Pulling Docker images reduces reproducibility. If you were to pull a Docker image today and then wait six months and pull again, you are not guaranteed to get the same image. If any of the source layers has changed the image will be altered. If reproducibility is a priority for you, try building your images from the Container Library.
Pulling an image from Sylabs cloud library¶
Let’s use an easy example of
alpine.sif image from the container
You can use
singularity search <name> command to locate groups,
collections, and containers of interest on the Container Library
Interact with images¶
You can interact with images in several ways such as
For these examples we will use a
cowsay_latest.sif image that can be
pulled from the Docker Hub.
$ singularity pull docker://tswetnam/cowsay INFO: Downloading library image 67.00 MiB / 67.00 MiB [=====================================================================================================] 100.00% 5.45 MiB/s 12s WARNING: unable to verify container: cowsay_latest.sif WARNING: Skipping container verification $ sudo singularity run cowsay_latest.sif ________________________________________ / Expect a letter from a friend who will \ \ ask a favor of you. / ---------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
shell command allows you to spawn a new shell within your
container and interact with it as though it were a small virtual
The change in prompt indicates that you have entered the container (though you should not rely on that to determine whether you are in container or not).
Once inside of a Singularity container, you are the same user as you are on the host system.
shell also works with the library://, docker://, and shub:// URIs.
This creates an ephemeral container that disappears when the shell is
The exec command allows you to execute a custom command within a
container by specifying the image file. For instance, to execute the
cowsay program within the cowsay_latest.sif container:
exec also works with the library://, docker://, and shub:// URIs.
This creates an ephemeral container that executes a command and disappears.
Singularity containers contain
These are user defined scripts that define the actions a container
should perform when someone runs it. The runscript can be triggered with
run command, or simply by calling the container as though it were
singularity run lolcow_latest.sif _________________________________________ / You will remember, Watson, how the \ | dreadful business of the Abernetty | | family was first brought to my notice | | by the depth which the parsley had sunk | | into the butter upon a hot day. | | | \ -- Sherlock Holmes / ----------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
inspect command will provide information about labels, metadata, and environmental variables.
See Next Section for details of