Dask Kubernetes

Dask Kubernetes

PyPI Conda Forge Python Support Kubernetes Support

Welcome to the documentation for dask-kubernetes.

Note

If you are looking for general documentation on deploying Dask on Kubernetes new users should head to the Dask documentation page on Kubernetes.

The package dask-kubernetes provides cluster managers for Kubernetes.

KubeCluster

KubeCluster deploys Dask clusters on Kubernetes clusters using native Kubernetes APIs. It is designed to dynamically launch ad-hoc deployments.

from dask_kubernetes import KubeCluster, make_pod_spec

pod_spec = make_pod_spec(image='ghcr.io/dask/dask:latest')
cluster = KubeCluster(pod_spec)
cluster.scale(10)

HelmCluster

HelmCluster is for managing an existing Dask cluster which has been deployed using Helm. You must have already installed the Dask Helm chart and have the cluster running. You can then use it to manage scaling and retrieve logs.

from dask_kubernetes import HelmCluster

cluster = HelmCluster(release_name="myrelease")
cluster.scale(10)