Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.10.7.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd9589d3a7c46e53703f68363caf45c4d49df264f767d9e6eb41b67a033e59c2 |
|
MD5 | f7f9feb5b3aafcef03fd145c84250c83 |
|
BLAKE2b-256 | e258158f47a2ef08ee17c463fc6f2a6cfb9524a9fcc1279fa09d22c59c5a242a |
Hashes for flytekitplugins_dask-1.10.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3a134c062fd27ae7435d1d477d6b3d3e5b6c22bbdf9e8c771628ddde51d6bae5 |
|
MD5 | eb6b7c0bab99fcdc9b485e63932065eb |
|
BLAKE2b-256 | 78e8018196e7b1ba0a45281e9715efb5cc099e85d934b9351be337226228a938 |