Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.3.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | fdc3e0b87418cd3b3e54c2921ab5632afb49ce12fe9294503ebdaa2517793bb1 |
|
MD5 | 88afbd54b1ecb2513024d61e80fab0e8 |
|
BLAKE2b-256 | 817b0ad1121d9724e9861c6c28d3d579cd97d9ba987efc86da2d71a497f0fe48 |
Hashes for flytekitplugins_dask-1.3.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 185df2eb5d489040d6c6b874ae1a2e775c309a22167dca53861064659120a4f2 |
|
MD5 | 1223967132001a43f7956bb6e8096789 |
|
BLAKE2b-256 | 352439f9c565e0e9fba678fbd4706c18d60febeebcdbbd89c7e07d4e6ee9bd29 |