Covalent Workflow Tool
Project description
🤔 What is Covalent?
Covalent is a Pythonic workflow tool for computational scientists, AI/ML software engineers, and anyone who needs to run experiments on limited or expensive computing resources including quantum computers, HPC clusters, GPU arrays, and cloud services.
Covalent enables a researcher to run computation tasks on an advanced hardware platform – such as a quantum computer or serverless HPC cluster – using a single line of code.
💭 Why Covalent?
Covalent overcomes computational and operational challenges inherent in AI/ML experimentation.
| Computational challenges | Operational challenges |
|---|---|
|
|
With Covalent, you:
- Assign functions to appropriate resources: Use advanced hardware (quantum computers, HPC clusters) for the heavy lifting and commodity hardware for bookkeeping.
- Test functions on local servers before shipping them to advanced hardware.
- Let Covalent's services analyze functions for data independence and automatically parallelize them.
- Run experiments from a Jupyter notebook (or whatever your preferred interactive Python environment is).
- Track workflows and examine results in a browser-based GUI.
📦 Installation
Covalent is developed using Python version 3.8 on Linux and macOS. The easiest way to install Covalent is using the PyPI package manager:
pip install covalent
Refer to the Quick Start guide for quick setup instructions, or to the First Experiment guide for a more thorough approach. For a full list of supported platforms, see the Covalent compatibility matrix.
🚀 Getting started
Ready to try it? Go to the First Experiment guide in the documentation.
For a more in-depth description of Covalent's features and how they work, see the Concepts page in the documentation.
| 📚 Know more ! | ✍️ Tutorials and Examples |
How Does It Work?
Covalent has three main components:
- A Python module containing an API that you use to build manageable workflows out of new or existing Python functions.
- A set of services that run locally or on a server to dispatch and execute workflow tasks.
- A browser-based UI from which to manage workflows and view results.
You compose workflows using the Covalent API by simply adding a single line of python decorator and submit them to the Covalent server. The server analyzes the workflow to determine dependencies between tasks, then dispatches each task to its specified execution backend. Independent tasks are executed concurrently if resources are available.
The Covalent UI displays the progress of each workflow at the level of individual tasks.
The Covalent API
The Covalent API is a Python module containing a small collection of classes that implement server-based workflow management. The key elements are two decorators that wrap functions to create managed *tasks* and *workflows*.
The task decorator is called an electron. The electron decorator simply turns the function into a dispatchable task.
The workflow decorator is called a lattice. The lattice decorator turns a function composed of electrons into a manageable workflow.
Covalent Services
The Covalent server is a lightweight service that runs on your local machine or a server. A dispatcher analyzes workflows (lattices) and hands its component functions (electrons) off to executors. Each executor is an adaptor to a backend hardware resource. Covalent has a growing list of turn-key executors for common compute backends. If no executor exists yet for your compute platform, Covalent supports writing your own.
The Covalent GUI
The Covalent user interface runs as a web server on the machine where the Covalent server is running. The GUI dashboard shows a list of dispatched workflows. From there, you can drill down to workflow details or a graphical view of the workflow. You can also view logs, settings, and result sets.
📚 Documentation
The official documentation includes tips on getting started, high-level concepts, tutorials, and the API documentation, and more. To learn more, see the Covalent documentation.
Troubleshooting
Solutions to common issues can be found in the Troubleshooting Guide.
✔️ Contributing
To contribute to Covalent, refer to the Contribution Guidelines. We use GitHub's issue tracking to manage known issues, bugs, and pull requests. Get started by forking the develop branch and submitting a pull request with your contributions. Improvements to the documentation, including tutorials and how-to guides, are also welcome from the community. For more more information on adding tutorials, check the Tutorial Guidelines Participation in the Covalent community is governed by the Code of Conduct.
⚓ Citation
Please use the following citation in any publications:
📃 License
Covalent is licensed under the Apache 2.0 License. See the LICENSE file or contact the support team for more details.
For a detailed history of changes and new features, see the Changelog.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
