Skip to main content

Setting up Branch Deployments

In this guide, we'll walk you through setting up Branch Deployments for a code location. Once you're finished, any time a PR is created or updated in your repository, it will automatically create or update an associated branch deployment in Dagster+.

Prerequisites

To follow the steps in this guide, you'll need:

  • Organization Admin permissions in Dagster+
  • The ability to run a new agent in your infrastructure (only if you are using a Hybrid deployment)

Step 1: Choose a method​

Choose a method for setting up branch deployments:

You can set up GitHub to automatically create branch deployments for new PRs, using GitHub Actions.

Using this approach to branch deployments may be a good fit if:

  • You use GitHub for version control
  • You want Dagster to fully automate Branch Deployments

This approach is simplified if you use the GitHub integration to import your project into Dagster+.

Step 2: Generate a Dagster+ agent token​

In this step, you'll generate a token for the Dagster+ agent. The Dagster+ agent will use this to authenticate to the agent API.

  1. Sign in to your Dagster+ instance.
  2. Click the user menu (your icon) > Organization Settings.
  3. In the Organization Settings page, click the Tokens tab.
  4. Click the Create agent token button.
  5. After the token has been created, click Reveal token.

Keep the token somewhere handy - you'll need it to complete the setup.

Step 3: Create and configure an agent​

note

If using Serverless deployment, this step can be skipped.

While you can use your existing production agent, we recommend creating a dedicated branch deployment agent. This ensures that your production instance isn't negatively impacted by the workload associated with branch deployments.

  1. Set up a new Kubernetes agent. Refer to the Kubernetes agent setup guide for instructions.

  2. After the agent is set up, modify your Helm values file to include the following:

Loading...

Step 4: Set up branch deployments​

Step 4.1: Add GitHub CI/CD script to your project​

note

If you used the GitHub app to configure you're repository, this step isn't required. Skip ahead to Step 4.5

Copy the following files to your project, and replace all references to quickstart-etl with the name of your project:

In the next step, you'll modify these files to work with your Dagster+ setup.

Step 4.2: Add the agent registry to dagster_cloud.yaml​

note

If you used the GitHub app to configure you're repository, this step isn't required. Skip ahead to Step 4.5

In the dagster_cloud.yaml file, replace build.registry with the registry used by the agent you created in Step 2.

For example:

Loading...

Step 4.3: Configure GitHub Action secrets​

note

If you used the GitHub app to configure you're repository, this step isn't required. Skip ahead to Step 4.5

  1. In your GitHub repository, click the Settings tab.
  2. In the Security section of the sidebar, click Secrets > Actions.
  3. Click New repository secret.
  4. In the Name field, enter the name of the secret. For example, DAGSTER_CLOUD_API_TOKEN
  5. In the Value field, paste the value of the secret.
  6. Click Add secret.

Repeat steps 3-6 for each of the secrets required for the registry used by the agent you created in Step 2. See below for more details:

  • DAGSTER_CLOUD_API_TOKEN - The Dagster+ agent token you created in Step 2
  • DAGSTER_CLOUD_URL - Your Dagster+ base URL (https://my_org.dagster.cloud)
  • DOCKERHUB_USERNAME - Your DockerHub username
  • DOCKERHUB_TOKEN - A DockerHub access token

Step 4.4: Configure GitHub Action​

note

If you used the GitHub app to configure you're repository, this step isn't required. Skip ahead to Step 4.5

In this step, you'll update the GitHub workflow files in your repository to set up Docker registry access.

In the .github/workflows/dagster-cloud-deploy.yml file, un-comment the step associated with your registry. For example, for an Amazon ECR registry, you'd un-comment the following portion of the workflow file:

jobs:
  dagster-cloud-deploy:
    steps:
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}

Save and commit the file to your repository.

Step 4.5: Verify GitHub action runs​

The last step is to verify that the GitHub Action runs successfully.

  1. In the repository, click the Actions tab.
  2. In the list of workflows, locate the latest branch deployment run. For example:

Show this in the UI

Next steps​