Kubernetes

Kubernetes

This guide will walk you through how to run the Dagster-specific components of a Dagster production deployment on a Kubernetes cluster. This includes the Dagster daemon, a webserver to serve the Dagster UI, a PostgrSQL container, and your Dagster project user code.

Customizing your Kubernetes deployment

This section covers common ways to customize your Dagster Kubernetes deployment.

Using Celery with Kubernetes

In addition to using the k8sjobexecutor to run each op in its own Kubernetes job, Dagster also allows you to use Celery to limit the number of ops that can concurrently connect to a resource across all running Dagster jobs.

Migrating a Dagster instance while upgrading Dagster in a Kubernetes environment

We walk through how to migrate your Dagster instance using a Kubernetes Job from the Helm chart.