Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.5.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c0b779d3c3b82350f982a5ac4cd3b2f2b4219e183bc487221293af6188b390d4 |
|
MD5 | 33d7bb7c3e90fd64a5cd983dcc831ea5 |
|
BLAKE2b-256 | 5e34bb49213c2d42b2c9f64e580c059ed7a2a93b7dc737b63cd6131dc9d450c0 |
Hashes for flytekitplugins_dask-1.5.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce0a42694c7b6c99d71bc26a46f5be3bf35f5822970741a336568f108dc66f89 |
|
MD5 | fe2c9bfcefa0a94d624cfd33ffe89fea |
|
BLAKE2b-256 | 0bce29112c5abf8020ae4924a776d77697961d9a5e033cdbc0558014d0431287 |