Creates a permanent cache for singularity images on disk.
Project description
Singularity-permanent-cache creates a permanent cache for singularity images on disk. It takes a URI as argument and returns the location of the image. It utilizes a filelock to prevent cache corruption.
It will use the SINGULARITY_PERMANENTCACHEDIR or SINGULARITY_CACHEDIR environment variables to determine the location of the cache. Alternatively the cache dir can be set with the -d or --cache-dir flags on the command line.
The singularity-permanent-cache command can be used in scripts. It was designed with multiprocess usage in mind: a filelock will prevent corruption of the cache when multiple instances of singularity-permanent-cache are running. It can be used in a script like this:
#!/usr/bin/env bash
set -eu -o pipefail
export SINGULARITY_PERMANENTCACHEDIR=$HOME/.singularity/permanent_cache
MY_IMAGE_URI="docker://debian:buster-slim"
IMAGE_LOCATION=$(singularity-permanent-cache $MY_IMAGE_URI)
cluster_submit "singularity exec $IMAGE_LOCATION echo 'Hello world!'"Singularity-permanent-cache will download the debian buster slim image if it is not yet in the cache. It will not dowload anything if it is already in the cache.
Usage
Beside singularity-permanent-cache, also spc is added to PATH as a short-hand when installing the package. singularity-permanent-cache is also available as a stand-alone script singularity_permanent_cache.py.
singularity-permant-cache has no dependencies and only requires a modern python version (3.5 or higher).
usage: singularity-permanent-cache [-h] [-d CACHE_DIR] [-s SINGULARITY_EXE]
[--which-cache] [-v] [-q]
<IMAGE>
Creates a permanent cache on disk for singularity images. Returns the location
of the image in the cache. WARNING: This program will never check if a newer
image is available. Make sure unique tags or hashes are used!
positional arguments:
<IMAGE> The singularity URI to the image. For example:
'docker://debian:buster-slim'
optional arguments:
-h, --help show this help message and exit
-d CACHE_DIR, --cache-dir CACHE_DIR
Path to the cache location. Uses the
SINGULARITY_PERMANENTCACHEDIR, or SINGULARITY_CACHEDIR
environment variable by default.
-s SINGULARITY_EXE, --singularity-exe SINGULARITY_EXE
Path to singularity executable.
--which-cache Show which cache the program will use and exit.
-v, --verbose Increase log verbosity. Can be used multiple times.
-q, --quiet Decrease log verbosity. Can be used multiple times.Acknowledgements
Lots of thanks to @TMiguelT, @illusional and @vsoch for their constructive feedback on this PR for Cromwell which led to the development of this program.
The filelock implementation is based on py-filelock. Huge thanks to @benediktschmitt & contributors for this filelock example which they made Public Domain.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for singularity-permanent-cache-1.0.0a0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 8e91f14b08cbe4cce61b0b271f0768f0e4e1b5d4f9db1c000ea3bb5aceb37dea |
|
| MD5 | 611be54f193e5f7a054723d5764058e1 |
|
| BLAKE2b-256 | bf4a45e2079ad58ff8bca6152c49107e021052cb917332adf4c4706ab4c9e023 |
Hashes for singularity_permanent_cache-1.0.0a0-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 5a51e25446edf2adec3ab953c3e7e1c41f633bab0d4c6148e98c0a72d2699e23 |
|
| MD5 | c93f95c6b973f64f8166f0fac04dd4ab |
|
| BLAKE2b-256 | 328951c1ff2394e3c2dd77dce3d43affd36196a6d24756bd24fd5eacd77561a6 |
