Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.5.0b1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | f722d46e9507de7d1d482ca77b4d551ecb6b7381bd4ee4cd86eeb5a129a07c2d |
|
MD5 | 961d7efeb856fbddec092d5c4510452c |
|
BLAKE2b-256 | 97504e04681bca22278fdfcd50b99bb004e01079a552279324dcbaf5b8ca50b9 |
Hashes for flytekitplugins_dask-1.5.0b1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d2bb788b3c7d62e3fc27f95f4cfc870a0bfecca9afc7a183a161ad90e954576 |
|
MD5 | 274858fb1000eb120c963a6375ea6ed8 |
|
BLAKE2b-256 | 04ad79a7d63bb010083e072a373b1bc96949b39f3ae2d7fa51a91a8774bed4dd |