Skip to main content

An opinionated Cromwell orchestration system

Project description

Oliver

Build Status PyPI PyPI - Downloads Conda Conda - Downloads License: MIT

Oliver Example

An opinionated Cromwell orchestration manager.

Getting Started

Conda

Oliver is distributed as a package using the community-curated Anaconda repository, conda-forge. We recommend that you first follow the instructions included in the conda-forge documentation to get everything set up!

conda install oliver -c conda-forge

Python Package Index

You can also install Oliver using the Python Package Index (PyPI).

pip install stjudecloud-oliver

Please refer to the guides in the docs/ folder for more information.

Guide Name Link
Advanced Usage Link
Configuration Link
Submitting Jobs Link

Usage

oliver -h

The following subcommands are currently supported.

Subcommand Short Command Description
aws All subcommands related to Cromwell on AWS.
azure All subcommands related to Cromwell on Azure.
aggregate a Aggregate all results to a local or cloud folder for a run.
batches b Explore batches of jobs submitted to Cromwell.
configure Configure Oliver with default options.
config Set or get a single config value from Oliver.
inputs Find all reported outputs for a given workflow.
inspect i Describe the state of a Cromwell workflow.
kill k Kill a workflow running on a Cromwell server.
logs l Find all reported logs for a given workflow.
outputs o Find all reported outputs for a given workflow.
retry re Resubmit a workflow with the same parameters.
runtime ru Get the runtime attributes used for a specific call.
status st Report various statistics about a running Cromwell server.
submit su Submit a workflow to the Cromwell server.

Development

If you are interested in contributing to the code, please first review our CONTRIBUTING.md document. To bootstrap a development environment, please use the following commands.

# Clone the repository
git clone git@github.com:stjudecloud/oliver.git
cd oliver

# Link the package with your current Python environment
python setup.py develop

# Ensure pre-commit is installed to automatically format
# code using `black`.
brew install pre-commit
pre-commit install
pre-commit install --hook-type commit-msg

Tests

Oliver provides a (currently patchy) set of tests — both unit and end-to-end. To get started with testing, you'll need to bootstrap a Docker test environment (one-time operation).

# Start development environment
docker image build --tag oliver .
docker-compose up --build  -d

alias docker-run-oliver="docker container run \
  -it \
  --rm \
  --network oliver_default \
  --mount type=bind,source=$PWD/seeds,target=/opt/oliver/seeds \
  --mount type=bind,source=$PWD/oliver,target=/opt/oliver/oliver \
  --mount type=bind,source=$PWD/scripts,target=/opt/oliver/scripts \
  --mount type=bind,source=$PWD/tests,target=/opt/oliver/tests \
  oliver"

# Seed development environment (make sure Cromwell is live first!)
docker-run-oliver bash seeds/seed.sh http://cromwell:8000 seeds/wdl/hello.wdl
docker-run-oliver pytest

To reset your entire docker-compose environment, you can run the following:

docker-compose down

docker image rm oliver:latest
docker image rm oliver_cromwell:latest
docker image rm mysql:5.7
docker volume rm oliver_mysql_data
docker network rm oliver_default

docker image build --tag oliver .
docker-compose up --build -d

Author

👤 St. Jude Cloud Team

🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check issues page. You can also take a look at the contributing guide.

📝 License

Copyright © 2020 St. Jude Cloud Team.
This project is MIT licensed.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

stjudecloud-oliver-1.0.7.tar.gz (33.8 kB view hashes)

Uploaded Source

Built Distribution

stjudecloud_oliver-1.0.7-py3-none-any.whl (44.6 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page