Dask Kubernetes
Contents
Dask Kubernetes¶
Welcome to the documentation for dask-kubernetes.
Note
If you are looking for general documentation on deploying Dask on Kubernetes new users should head to the Dask documentation page on Kubernetes.
The package dask-kubernetes provides cluster managers for Kubernetes. dask-kubernetes is one of many options for deploying Dask clusters, see Deploying Dask in the Dask documentation for an overview of additional options.
KubeCluster¶
KubeCluster deploys Dask clusters on Kubernetes clusters using custom
Kubernetes resources. It is designed to dynamically launch ad-hoc deployments.
$ # Install operator CRDs and controller, needs to be done once on your Kubernetes cluster
$ helm install --repo https://helm.dask.org --create-namespace -n dask-operator --generate-name dask-kubernetes-operator
from dask_kubernetes.operator import KubeCluster
cluster = KubeCluster(name="my-dask-cluster", image='ghcr.io/dask/dask:latest')
cluster.scale(10)
HelmCluster¶
HelmCluster is for managing an existing Dask cluster which has been deployed using
Helm. You must have already installed the Dask Helm chart
and have the cluster running. You can then use it to manage scaling and retrieve logs.
from dask_kubernetes import HelmCluster
cluster = HelmCluster(release_name="myrelease")
cluster.scale(10)