Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.6.0b2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 939de161fdf5304f2c8ac840f959018ec2a14e54d503fc5560cbe11cb51b69bf |
|
MD5 | 6a7af85b1be6c8e274ab586dbfe79b81 |
|
BLAKE2b-256 | 1196b85f8ec5bd174bfe611490ae3c3b4c4608058d23ea7a775a6bb1df6250ac |
Hashes for flytekitplugins_dask-1.6.0b2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dc145919afb7cd65a2ea9dfd84b9fbdbff1304c4a4aea4b6407d60f5eb18646f |
|
MD5 | 6095df8669184011bb3c426e4a66e23e |
|
BLAKE2b-256 | c32641fa04da969e8387a20b27f8d38d20f9ef706f9b66fecfe61b8889548e18 |