Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.10.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a7122de706ec132070070bd437688a3bd4e044debe1d79a49a2b979d62154f12 |
|
MD5 | b422d467ce80f6c7764fe8e5b53b0cb0 |
|
BLAKE2b-256 | 2230a4378090dca07f2e4da07f711599fcb082219265b52b2ab528ab47e196fa |
Hashes for flytekitplugins_dask-1.10.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c22f8f1d2f18c694dfc9b5bb84a6c6e68433540d164de5786f82f804ffabb1c0 |
|
MD5 | 9f09dcf956ff716be0e6ab75a98367ec |
|
BLAKE2b-256 | 2957270a0245322c8d33c14401a96728579993f7d12b109996067868796ae797 |