API reference
asset-checks
Dagster allows you to define and execute checks on your software-defined assets. Each asset check verifies some property of a data asset, e.g. that is has no null values in a particular column.
assets
An asset is an object in persistent storage, such as a table, file, or persisted machine learning model. An asset definition is a description, in code, of an asset that should exist and how to produce and update that asset.
cli
dagster asset
config
Pythonic config system
definitions
class dagster.Definitions
dynamic
These APIs provide the means for a simple kind of dynamic orchestration — where the work to be orchestrated is determined not at job definition time but at runtime, dependent on data that’s observed as part of job execution.
errors
Core Dagster error classes.
execution
Materializing Assets
external-assets
As Dagster doesn’t control scheduling or materializing external assets, it’s up to you to keep their metadata updated. The APIs in this reference can be used to keep external assets updated in Dagster.
graphs
The core of a job is a graph of ops - connected via data dependencies.
hooks
@dagster.success_hook
API reference
internals
Note that APIs imported from Dagster submodules are not considered stable, and are potentially subject to change in the future.
io-managers
IO managers are user-provided objects that store op outputs and load them as inputs to downstream
jobs
A Job binds a Graph and the resources it needs to be executable.
libraries
58 items
loggers
Built-in loggers
metadata
Dagster uses metadata to communicate arbitrary user-specified metadata about structured
ops
The foundational unit of computation in Dagster.
partitions
class dagster.PartitionsDefinition
pipes
Dagster Pipes is a toolkit for building integrations between Dagster and external execution environments. This reference outlines the APIs included with the dagster library, which should be used in the orchestration environment.
repositories
dagster.repository RepositoryDefinition
resources
Pythonic resource system
schedules-sensors
Dagster offers several ways to run data pipelines without manual intervation, including traditional scheduling and event-based triggers. Automating your Dagster pipelines can boost efficiency and ensure that data is produced consistently and reliably.
types
Dagster includes facilities for typing the input and output values of ops (“runtime” types).
utilities
dagster.filerelativepath