Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.6.2b1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5af39097155ac3a3189bf3964ffd6fb9f2b7392bc86c92d7dd3cc597f73c7e1b |
|
MD5 | 1a97f9a043d658b82499ceaf85acc10a |
|
BLAKE2b-256 | 2e16604d0344bd124e81a0cd90efc3677d2e64067394ceebf96f042c455f6a39 |
Hashes for flytekitplugins_dask-1.6.2b1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c9911c2e31b3f13bd417fc139c6d8d9ab46aca63e3300de9eb206961ae958824 |
|
MD5 | d5ef0b247fed2237a0385402ccb8c8cd |
|
BLAKE2b-256 | 5d5b6d09a291c03102605d7b33bbd5fefd06941dece41c33d678409474320eab |