Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.10.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57711722c99930dd2111b0cbe5252d03bcd744c3bf3bf772d6d04cbce6b5ac4c |
|
MD5 | 250e9aa6990692b63eb2330752aec7df |
|
BLAKE2b-256 | 52930cc9547074ffd59491ed4b4036f7592a090aaf9357f1df798f6cffcdc031 |
Hashes for flytekitplugins_dask-1.10.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9f3fac95f48878c5bcff414448fab73ed5ff7e19a18dec12243ceb22ecc59629 |
|
MD5 | 7e05eef048fff0bbdb03e78862f8ac1d |
|
BLAKE2b-256 | 25698db15b9877f3696702908df5e75ee796bdfa094fff15668b5d2acf9fe081 |