Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.4.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3e1551e84f29f7753c6e6e27f34da57b3b7c3aa296f5ed3f25b9a648a54f707e |
|
MD5 | e877dd7aa1706aa8d3cb88cb6b71f225 |
|
BLAKE2b-256 | 55427888f312184b15e65ac8846a2f51543a8ed1d88d1114aa3661b9f938f602 |
Hashes for flytekitplugins_dask-1.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a83ee3465605836e3aa819d3d450eb211a4ecef00721b79e78cf5cbd2e5db9dd |
|
MD5 | 07a4de2ebe6301c892192f8a0e3d695e |
|
BLAKE2b-256 | df5e39061742524adec242c05294e9ca9caf3ce72301cb93ac60ddb1da214e9d |