ChRIS Automatic Workflows
Project description
ChRIS Automated Workflows
A command-line client for ChRIS for pipeline execution and data mangement.
Installation
The easiest option is via pip
.
Pip
pip install -U caw
# optional, for tab completion of subcommands
caw --install-completion
# optional, for secure password storage
pip install keyring
Alternatively, container images are also provided. See below.
Usage
caw [OPTIONS] COMMAND [ARGS]...
Container Usage
A Docker image is also provided. Podman and Docker work equally well.
docker run --rm --net=host -v $PWD/data:/data:ro -t fnndsc/caw:latest caw upload /data
podman run --rm --net=host -v $PWD/data:/data:ro -t fnndsc/caw:latest caw upload /data
Constainer isolation can make usage finicky.
Volumes must be mounted for the container to read data which exists on the host filesystem.
If the ChRIS backend is on a private network, the --net=host
option might be necessary to resolve
the server's hostname.
Alternatively, Singularity
is much easier to use because of its weaker container isolation and $HOME
is a bind path by default.
singularity exec docker://fnndsc/caw:latest caw upload ./data
Documentation
Details are provided by the --help
commaand.
caw --help
caw search --help
caw pipeline --help
caw download --help
caw upload --help
Logging In
Multiple ways of providing your credentials are supported. The most secure way is to
install the optional dependency keyring
and run caw login
.
# install optional dependency
pip install keyring
caw --address https://cube.chrisproject.org/api/v1/ --username 'a_crow' login
Alternatively, ChRIS user account credentials can be passed via command-line arguments or environment variables. It's safer to use environment variables (so that your password isn't saved to history) and also easier (no need to retype it out everytime).
# using cli arguments
caw --address https://cube.chrisproject.org/api/v1/ \
--username 'a_crow' \
--password notchris1234 \
search
# using environment variables
export CHRIS_URL=https://cube.chrisproject.org/api/v1/
export CHRIS_USERNAME=a_crow
export CHRIS_PASSWORD=notchris1234
caw search
Commands
search
: Search for pipelines that are saved in ChRIS.pipeline
: Run a pipeline on an existing feed.upload
: Upload files into ChRIS storage and runpl-dircopy
.download
: Download everything from a ChRIS url.
caw search
Search for pipelines that are saved in ChRIS.
Examples
# list all pipellines
$ caw search
https://cube.chrisproject.org/api/v1/pipelines/1/ Automatic Fetal Brain Reconstruction Pipeline
https://cube.chrisproject.org/api/v1/pipelines/2/ Infant FreeSurfer with Cerebellum Step
https://cube.chrisproject.org/api/v1/pipelines/2/ COVID-Net Chest CT Analysis and Report
# search for pipelines by name
$ caw search 'Fetal Brain'
https://cube.chrisproject.org/api/v1/pipelines/1/ Automatic Fetal Brain Reconstruction Pipeline
caw pipeline
Run a pipeline on an existing feed.
Examples
# specify source as a plugin instance ID
$ caw pipeline --target 3 'Automatic Fetal Brain Reconstruction Pipeline'
# specify source by URL
$ caw pipeline --target https://cube.chrisproject.org/api/v1/plugins/instances/3/ 'Automatic Fetal Brain Reconstruction Pipeline'
caw upload
Upload files into ChRIS storage and then run pl-dircopy, printing the URL for the newly created plugin instance.
Examples
# upload files and create a new feed by running pl-dircopy
$ caw upload something.txt picture.jpg
# upload a folder and create a new feed by running pl-dircopy
$ caw upload data/
# create a feed with a title and description
$ caw upload --name 'Caw caw, ima crow' --description 'A murder of crows' \
something.txt picture.jpg
# create a feed and run a pipeline after the pl-dircopy instance
$ caw upload --name 'In-utero study' \
--pipeline 'Automatic Fetal Brain Reconstruction Pipeline' \
data/T2_*.nii
caw download
Download files from ChRIS.
# download everything from a feed
$ caw download 'https://cube.chrisproject.org/api/v1/3/files/' results/
# download the output directory of a specific plugin instance
$ caw download 'https://cube.chrisproject.org/api/v1/plugins/instances/5/files/' results/
# download everything from a path 'chris/uploads/test'
$ caw download 'https://cube.chrisproject.org/api/v1/uploadedfiles/search/?fname=chris%2Fuploads%2Ftest' results/
# example results
$ tree results/
wow
└── uploads
└── test
├── a.txt
├── b.txt
├── c.txt
├── d.txt
└── e.txt
Development
python -m venv venv
source venv/bin/activate
pip install -e .
Testing
First, set up the ChRIS backend on http://localhost:8000/api/v1/
(say, using miniChRIS).
Next, install the example pipeline.
./testing/upload_reconstruction_pipeline.sh
Run all tests using the command
python -m unittest
The end-to-end test is disabled by default because it will create a ChRIS account and
affect caw
user settings. It is recommended to run it in a container instead.
docker build -t caw .
docker run --rm --net=host --userns=host \
-e CAW_TEST_FULL=y caw python -m unittest caw.tests.test_e2e
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.