Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.4.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e7542ec13865ac71d4e7d254f286ff3b78e65fa55934279469ebce7b939d6452 |
|
MD5 | 25653310696b3ef4bf506fbe7d245238 |
|
BLAKE2b-256 | 99ee8e5474aa1b18e07ea695d6da7f5b48c0c96f08d3d01bc4d0598fef5e6382 |
Hashes for flytekitplugins_dask-1.4.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 10b7fa42276837fa53f7348770e3bff838cd9862051bade7232198e5405bccad |
|
MD5 | aaba54104f71625e27073e7093bebf76 |
|
BLAKE2b-256 | 7d3691cb0769c5b2d75fddee7b97d14acf55658ca88cb80ead31b4b896e3e2a8 |