Skip to main content

Assets

An asset is an object in persistent storage, such as a table, file, or persisted machine learning model. An asset definition is a description, in code, of an asset that should exist and how to produce and update that asset.

Asset definitions​

Refer to the Asset definitions documentation for more information.

@dagster.asset

Create a definition for how to compute an asset.

A software-defined asset is the combination of:

  1. An asset key, e.g. the name of a table.
  2. A function, which can be run to compute the contents of the asset.
  3. A set of upstream assets that are provided as inputs to the function when computing the asset. Unlike an op, whose dependencies are determined by the graph it lives inside, an asset knows about the upstream assets it depends on. The upstream assets are inferred from the arguments to the decorated function. The name of the argument designates the name of the upstream asset.

An asset has an op inside it to represent the function that computes it. The name of the op will be the segments of the asset key, separated by double-underscores.

Parameters:

  • name (Optional[str]) – The name of the asset. If not provided, defaults to the name of the
  • key_prefix (Optional[Union[str, Sequence[str]]]) – If provided, the asset’s key is the
  • ins (Optional[Mapping[str, AssetInAssetIn]]) – A dictionary that maps input names to information
  • deps (Optional[Sequence[Union[AssetDepAssetDep, AssetsDefinitionAssetsDefinition, SourceAssetSourceAsset, AssetKeyAssetKey, str]]]) – The assets that are upstream dependencies, but do not correspond to a parameter of the
  • config_schema (Optional[ConfigSchemaConfigSchema) – The configuration schema for the asset’s underlying
  • metadata (Optional[Dict[str, Any]]) – A dict of metadata entries for the asset.
  • tags (Optional[Mapping[str, str]]) – Tags for filtering and organizing. These tags are not
  • required_resource_keys (Optional[Set[str]]) – Set of resource handles required by the op.
  • io_manager_key (Optional[str]) – The resource key of the IOManager used
  • io_manager_def (Optional[object]) – experimental
  • dagster_type (Optional[DagsterTypeDagsterType]) – Allows specifying type validation functions that
  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that
  • op_tags (Optional[Dict[str, Any]]) – A dictionary of tags for the op that computes the asset.
  • group_name (Optional[str]) – A string name used to organize multiple assets into groups. If not provided,
  • resource_defs (Optional[Mapping[str, object]]) – experimental
  • output_required (bool) – Whether the decorated function will always materialize an asset.
  • automation_condition (AutomationConditionAutomationCondition) – A condition describing when Dagster should materialize this asset.
  • backfill_policy (BackfillPolicyBackfillPolicy) – experimental
  • retry_policy (Optional[RetryPolicyRetryPolicy]) – The retry policy for the op that computes the asset.
  • code_version (Optional[str]) – Version of the code that generates this asset. In
  • check_specs (Optional[Sequence[AssetCheckSpecAssetCheckSpec]]) – Specs for asset checks that
  • key (Optional[CoeercibleToAssetKey]) – The key for this asset. If provided, cannot specify key_prefix or name.
  • owners (Optional[Sequence[str]]) – experimentalteam:,
  • kinds (Optional[Set[str]]) – A list of strings representing the kinds of the asset. These
  • non_argument_deps (Optional[Union[Set[AssetKeyAssetKey], Set[str]]]) – deprecateddeps instead.) Deprecated, use deps instead.

Examples:

@asset
def my_asset(my_upstream_asset: int) -> int:
    return my_upstream_asset + 1
class dagster.MaterializeResult

An object representing a successful materialization of an asset. These can be returned from @asset and @multi_asset decorated functions to pass metadata or specify specific assets were materialized.

asset_key

Optional in @asset, required in @multi_asset to discern which asset this refers to.

Type: Optional[AssetKey]

metadata

Metadata to record with the corresponding AssetMaterialization event.

Type: Optional[RawMetadataMapping]

check_results

Check results to record with the corresponding AssetMaterialization event.

Type: Optional[Sequence[AssetCheckResult]]

data_version

The data version of the asset that was observed.

Type: Optional[DataVersion]

tags

Tags to record with the corresponding AssetMaterialization event.

Type: Optional[Mapping[str, str]]

class dagster.AssetSpec

Specifies the core attributes of an asset, except for the function that materializes or observes it.

An asset spec plus any materialization or observation function for the asset constitutes an “asset definition”.

key

The unique identifier for this asset.

Type: AssetKey

deps

The asset keys for the upstream assets that materializing this asset depends on.

Type: Optional[AbstractSet[AssetKey]]

description

Human-readable description of this asset.

Type: Optional[str]

metadata

A dict of static metadata for this asset. For example, users can provide information about the database table this asset corresponds to.

Type: Optional[Dict[str, Any]]

skippable

Whether this asset can be omitted during materialization, causing downstream dependencies to skip.

Type: bool

group_name

A string name used to organize multiple assets into groups. If not provided, the name “default” is used.

Type: Optional[str]

code_version

The version of the code for this specific asset, overriding the code version of the materialization function

Type: Optional[str]

backfill_policy

BackfillPolicy to apply to the specified asset.

Type: Optional[BackfillPolicy]

owners

A list of strings representing owners of the asset. Each string can be a user’s email address, or a team name prefixed with team:, e.g. team:finops.

Type: Optional[Sequence[str]]

tags

Tags for filtering and organizing. These tags are not attached to runs of the asset.

Type: Optional[Mapping[str, str]]

kinds

(Optional[Set[str]]): A list of strings representing the kinds of the asset. These will be made visible in the Dagster UI.

partitions_def

Defines the set of partition keys that compose the asset.

Type: Optional[PartitionsDefinition]

with_io_manager_key

Returns a copy of this AssetSpec with an extra metadata value that dictates which I/O manager to use to load the contents of this asset in downstream computations.

Parameters: io_manager_key (str) – The I/O manager key. This will be used as the value for the “dagster/io_manager_key” metadata key.Returns: AssetSpec

class dagster.AssetsDefinition

Defines a set of assets that are produced by the same op or graph.

AssetsDefinitions are typically not instantiated directly, but rather produced using the @asset@asset or @multi_asset@multi_asset decorators.

static from_graph

Constructs an AssetsDefinition from a GraphDefinition.

Parameters:

  • graph_def (GraphDefinitionGraphDefinition) – The GraphDefinition that is an asset.
  • keys_by_input_name (Optional[Mapping[str, AssetKeyAssetKey]]) – A mapping of the input
  • keys_by_output_name (Optional[Mapping[str, AssetKeyAssetKey]]) – A mapping of the output
  • key_prefix (Optional[Union[str, Sequence[str]]]) – If provided, key_prefix will be prepended
  • internal_asset_deps (Optional[Mapping[str, Set[AssetKeyAssetKey]]]) – By default, it is assumed
  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that
  • partition_mappings (Optional[Mapping[str, PartitionMappingPartitionMapping]]) – Defines how to map partition
  • resource_defs (Optional[Mapping[str, ResourceDefinitionResourceDefinition]]) – experimental
  • group_name (Optional[str]) – A group name for the constructed asset. Assets without a
  • group_names_by_output_name (Optional[Mapping[str, Optional[str]]]) – Defines a group name to be
  • descriptions_by_output_name (Optional[Mapping[str, Optional[str]]]) – Defines a description to be
  • metadata_by_output_name (Optional[Mapping[str, Optional[RawMetadataMapping]]]) – Defines metadata to
  • tags_by_output_name (Optional[Mapping[str, Optional[Mapping[str, str]]]]) – Defines
  • freshness_policies_by_output_name (Optional[Mapping[str, Optional[FreshnessPolicy]]]) – Defines a
  • automation_conditions_by_output_name (Optional[Mapping[str, Optional[AutomationConditionAutomationCondition]]]) – Defines an
  • backfill_policy (Optional[BackfillPolicyBackfillPolicy]) – Defines this asset’s BackfillPolicy
  • owners_by_key (Optional[Mapping[AssetKeyAssetKey, Sequence[str]]]) – Defines
static from_op

Constructs an AssetsDefinition from an OpDefinition.

Parameters:

  • op_def (OpDefinitionOpDefinition) – The OpDefinition that is an asset.
  • keys_by_input_name (Optional[Mapping[str, AssetKeyAssetKey]]) – A mapping of the input
  • keys_by_output_name (Optional[Mapping[str, AssetKeyAssetKey]]) – A mapping of the output
  • key_prefix (Optional[Union[str, Sequence[str]]]) – If provided, key_prefix will be prepended
  • internal_asset_deps (Optional[Mapping[str, Set[AssetKeyAssetKey]]]) – By default, it is assumed
  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that
  • partition_mappings (Optional[Mapping[str, PartitionMappingPartitionMapping]]) – Defines how to map partition
  • group_name (Optional[str]) – A group name for the constructed asset. Assets without a
  • group_names_by_output_name (Optional[Mapping[str, Optional[str]]]) – Defines a group name to be
  • descriptions_by_output_name (Optional[Mapping[str, Optional[str]]]) – Defines a description to be
  • metadata_by_output_name (Optional[Mapping[str, Optional[RawMetadataMapping]]]) – Defines metadata to
  • tags_by_output_name (Optional[Mapping[str, Optional[Mapping[str, str]]]]) – Defines
  • freshness_policies_by_output_name (Optional[Mapping[str, Optional[FreshnessPolicy]]]) – Defines a
  • automation_conditions_by_output_name (Optional[Mapping[str, Optional[AutomationConditionAutomationCondition]]]) – Defines an
  • backfill_policy (Optional[BackfillPolicyBackfillPolicy]) – Defines this asset’s BackfillPolicy
get_asset_spec

Returns a representation of this asset as an AssetSpecAssetSpec.

If this is a multi-asset, the “key” argument allows selecting which asset to return the spec for.

Parameters: key (Optional[AssetKeyAssetKey]) – If this is a multi-asset, select which asset to return its AssetSpec. If not a multi-asset, this can be left as None.Returns: AssetSpec

get_partition_mapping

Returns the partition mapping between keys in this AssetsDefinition and a given input asset key (if any).

to_source_asset

Returns a representation of this asset as a SourceAssetSourceAsset.

If this is a multi-asset, the “key” argument allows selecting which asset to return a SourceAsset representation of.

Parameters: key (Optional[Union[str, Sequence[str], AssetKeyAssetKey]]]) – If this is a multi-asset, select which asset to return a SourceAsset representation of. If not a multi-asset, this can be left as None.Returns: SourceAsset

to_source_assets

Returns a SourceAsset for each asset in this definition.

Each produced SourceAsset will have the same key, metadata, io_manager_key, etc. as the corresponding asset

property asset_deps

Maps assets that are produced by this definition to assets that they depend on. The dependencies can be either “internal”, meaning that they refer to other assets that are produced by this definition, or “external”, meaning that they refer to assets that aren’t produced by this definition.

property can_subset

If True, indicates that this AssetsDefinition may materialize any subset of its asset keys in a given computation (as opposed to being required to materialize all asset keys).

Type: bool

property check_specs

Returns the asset check specs defined on this AssetsDefinition, i.e. the checks that can be executed while materializing the assets.

Return type: Iterable[AssetsCheckSpec]

property dependency_keys

The asset keys which are upstream of any asset included in this AssetsDefinition.

Type: Iterable[AssetKey]

property descriptions_by_key

Returns a mapping from the asset keys in this AssetsDefinition to the descriptions assigned to them. If there is no assigned description for a given AssetKey, it will not be present in this dictionary.

Type: Mapping[AssetKey, str]

property group_names_by_key

Returns a mapping from the asset keys in this AssetsDefinition to the group names assigned to them. If there is no assigned group name for a given AssetKey, it will not be present in this dictionary.

Type: Mapping[AssetKey, str]

property key

The asset key associated with this AssetsDefinition. If this AssetsDefinition has more than one asset key, this will produce an error.

Type: AssetKey

property keys

The asset keys associated with this AssetsDefinition.

Type: AbstractSet[AssetKey]

property node_def

Returns the OpDefinition or GraphDefinition that is used to materialize the assets in this AssetsDefinition.

Type: NodeDefinition

property op

Returns the OpDefinition that is used to materialize the assets in this AssetsDefinition.

Type: OpDefinition

property partitions_def

The PartitionsDefinition for this AssetsDefinition (if any).

Type: Optional[PartitionsDefinition]

property required_resource_keys

The set of keys for resources that must be provided to this AssetsDefinition.

Type: Set[str]

property resource_defs

A mapping from resource name to ResourceDefinition for the resources bound to this AssetsDefinition.

Type: Mapping[str, ResourceDefinition]

class dagster.AssetKey

Object representing the structure of an asset key. Takes in a sanitized string, list of strings, or tuple of strings.

Example usage:

from dagster import AssetKey

AssetKey("asset1")
AssetKey(["asset1"]) # same as the above
AssetKey(["prefix", "asset1"])
AssetKey(["prefix", "subprefix", "asset1"])

Parameters: path (Union[str, Sequence[str]]) – String, list of strings, or tuple of strings. A list of strings represent the hierarchical structure of the asset_key.

property path

Graph-backed asset definitions​

Refer to the Graph-backed asset documentation for more information.

@dagster.graph_asset

Creates a software-defined asset that’s computed using a graph of ops.

This decorator is meant to decorate a function that composes a set of ops or graphs to define the dependencies between them.

Parameters:

  • name (Optional[str]) – The name of the asset. If not provided, defaults to the name of the

  • description (Optional[str]) – A human-readable description of the asset.

  • ins (Optional[Mapping[str, AssetInAssetIn]]) – A dictionary that maps input names to information

  • config (Optional[Union[ConfigMappingConfigMapping], Mapping[str, Any]) –

    Describes how the graph underlying the asset is configured at runtime.

    If a ConfigMappingConfigMapping object is provided, then the graph takes on the config schema of this object. The mapping will be applied at runtime to generate the config for the graph’s constituent nodes.

    If a dictionary is provided, then it will be used as the default run config for the graph. This means it must conform to the config schema of the underlying nodes. Note that the values provided will be viewable and editable in the Dagster UI, so be careful with secrets.

  • key_prefix (Optional[Union[str, Sequence[str]]]) – If provided, the asset’s key is the

  • group_name (Optional[str]) – A string name used to organize multiple assets into groups. If

  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that

  • metadata (Optional[RawMetadataMapping]) – Dictionary of metadata to be associated with

  • tags (Optional[Mapping[str, str]]) – (Experimental) Tags for filtering and organizing. These tags are not

  • owners (Optional[Sequence[str]]) – experimentalteam:,

  • kinds (Optional[Set[str]]) – A list of strings representing the kinds of the asset. These

  • automation_condition (Optional[AutomationConditionAutomationCondition]) – The AutomationCondition to use

  • backfill_policy (Optional[BackfillPolicyBackfillPolicy]) – The BackfillPolicy to use for this asset.

  • code_version (Optional[str]) – Version of the code that generates this asset. In

  • key (Optional[CoeercibleToAssetKey]) – The key for this asset. If provided, cannot specify key_prefix or name.

Examples:

@op
def fetch_files_from_slack(context) -> pd.DataFrame:
    ...

@op
def store_files(files) -> None:
    files.to_sql(name="slack_files", con=create_db_connection())

@graph_asset
def slack_files_table():
    return store_files(fetch_files_from_slack())
@dagster.graph_multi_asset

Create a combined definition of multiple assets that are computed using the same graph of ops, and the same upstream assets.

Each argument to the decorated function references an upstream asset that this asset depends on. The name of the argument designates the name of the upstream asset.

Parameters:

  • name (Optional[str]) – The name of the graph.

  • outs – (Optional[Dict[str, AssetOut]]): The AssetOuts representing the produced assets.

  • ins (Optional[Mapping[str, AssetInAssetIn]]) – A dictionary that maps input names to information

  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that

  • backfill_policy (Optional[BackfillPolicyBackfillPolicy]) – The backfill policy for the asset.

  • group_name (Optional[str]) – A string name used to organize multiple assets into groups. This

  • can_subset (bool) – Whether this asset’s computation can emit a subset of the asset

  • config (Optional[Union[ConfigMappingConfigMapping], Mapping[str, Any]) –

    Describes how the graph underlying the asset is configured at runtime.

    If a ConfigMappingConfigMapping object is provided, then the graph takes on the config schema of this object. The mapping will be applied at runtime to generate the config for the graph’s constituent nodes.

    If a dictionary is provided, then it will be used as the default run config for the graph. This means it must conform to the config schema of the underlying nodes. Note that the values provided will be viewable and editable in the Dagster UI, so be careful with secrets.

    If no value is provided, then the config schema for the graph is the default (derived

Multi-asset definitions​

Refer to the Multi-asset documentation for more information.

@dagster.multi_asset

Create a combined definition of multiple assets that are computed using the same op and same upstream assets.

Each argument to the decorated function references an upstream asset that this asset depends on. The name of the argument designates the name of the upstream asset.

You can set I/O managers keys, auto-materialize policies, freshness policies, group names, etc. on an individual asset within the multi-asset by attaching them to the AssetOutAssetOut corresponding to that asset in the outs parameter.

Parameters:

  • name (Optional[str]) – The name of the op.
  • outs – (Optional[Dict[str, AssetOut]]): The AssetOuts representing the assets materialized by
  • ins (Optional[Mapping[str, AssetInAssetIn]]) – A dictionary that maps input names to information
  • deps (Optional[Sequence[Union[AssetsDefinitionAssetsDefinition, SourceAssetSourceAsset, AssetKeyAssetKey, str]]]) – The assets that are upstream dependencies, but do not correspond to a parameter of the
  • config_schema (Optional[ConfigSchemaConfigSchema) – The configuration schema for the asset’s underlying
  • required_resource_keys (Optional[Set[str]]) – Set of resource handles required by the underlying op.
  • internal_asset_deps (Optional[Mapping[str, Set[AssetKeyAssetKey]]]) – By default, it is assumed
  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that
  • backfill_policy (Optional[BackfillPolicyBackfillPolicy]) – The backfill policy for the op that computes the asset.
  • op_tags (Optional[Dict[str, Any]]) – A dictionary of tags for the op that computes the asset.
  • can_subset (bool) – If this asset’s computation can emit a subset of the asset
  • resource_defs (Optional[Mapping[str, object]]) – experimental
  • group_name (Optional[str]) – A string name used to organize multiple assets into groups. This
  • retry_policy (Optional[RetryPolicyRetryPolicy]) – The retry policy for the op that computes the asset.
  • code_version (Optional[str]) – Version of the code encapsulated by the multi-asset. If set,
  • specs (Optional[Sequence[AssetSpecAssetSpec]]) – The specifications for the assets materialized
  • check_specs (Optional[Sequence[AssetCheckSpecAssetCheckSpec]]) – Specs for asset checks that
  • non_argument_deps (Optional[Union[Set[AssetKeyAssetKey], Set[str]]]) – deprecateddeps instead.) Deprecated, use deps instead.

Examples:

@multi_asset(
    specs=[
        AssetSpec("asset1", deps=["asset0"]),
        AssetSpec("asset2", deps=["asset0"]),
    ]
)
def my_function():
    asset0_value = load(path="asset0")
    asset1_result, asset2_result = do_some_transformation(asset0_value)
    write(asset1_result, path="asset1")
    write(asset2_result, path="asset2")

# Or use IO managers to handle I/O:
@multi_asset(
    outs=\{
        "asset1": AssetOut(),
        "asset2": AssetOut(),
    }
)
def my_function(asset0):
    asset1_value = do_some_transformation(asset0)
    asset2_value = do_some_other_transformation(asset0)
    return asset1_value, asset2_value
@dagster.multi_observable_source_asset
experimental

This API may break in future versions, even between dot releases.

Defines a set of assets that can be observed together with the same function.

Parameters:

  • name (Optional[str]) – The name of the op.
  • required_resource_keys (Optional[Set[str]]) – Set of resource handles required by the
  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that
  • can_subset (bool) – If this asset’s computation can emit a subset of the asset
  • resource_defs (Optional[Mapping[str, object]]) – (Experimental) A mapping of resource keys to resources. These resources
  • group_name (Optional[str]) – A string name used to organize multiple assets into groups. This
  • specs (Optional[Sequence[AssetSpecAssetSpec]]) – (Experimental) The specifications for the assets
  • check_specs (Optional[Sequence[AssetCheckSpecAssetCheckSpec]]) – (Experimental) Specs for asset checks that

Examples:

@multi_observable_source_asset(
    specs=[AssetSpec("asset1"), AssetSpec("asset2")],
)
def my_function():
    yield ObserveResult(asset_key="asset1", metadata=\{"foo": "bar"})
    yield ObserveResult(asset_key="asset2", metadata=\{"baz": "qux"})
class dagster.AssetOut

Defines one of the assets produced by a @multi_asset@multi_asset.

key_prefix

If provided, the asset’s key is the concatenation of the key_prefix and the asset’s name. When using @multi_asset, the asset name defaults to the key of the “outs” dictionary Only one of the “key_prefix” and “key” arguments should be provided.

Type: Optional[Union[str, Sequence[str]]]

key

The asset’s key. Only one of the “key_prefix” and “key” arguments should be provided.

Type: Optional[Union[str, Sequence[str], AssetKey]]

dagster_type

The type of this output. Should only be set if the correct type can not be inferred directly from the type signature of the decorated function.

Type: Optional[Union[Type, DagsterType]]]

description

Human-readable description of the output.

Type: Optional[str]

is_required

Whether the presence of this field is required. (default: True)

Type: bool

io_manager_key

The resource key of the IO manager used for this output. (default: “io_manager”).

Type: Optional[str]

metadata

A dict of the metadata for the output. For example, users can provide a file path if the data object will be stored in a filesystem, or provide information of a database table when it is going to load the data into the table.

Type: Optional[Dict[str, Any]]

group_name

A string name used to organize multiple assets into groups. If not provided, the name “default” is used.

Type: Optional[str]

code_version

The version of the code that generates this asset.

Type: Optional[str]

freshness_policy

(Deprecated) A policy which indicates how up to date this asset is intended to be.

Type: Optional[FreshnessPolicy]

automation_condition

AutomationCondition to apply to the specified asset.

Type: Optional[AutomationCondition]

backfill_policy

BackfillPolicy to apply to the specified asset.

Type: Optional[BackfillPolicy]

owners

A list of strings representing owners of the asset. Each string can be a user’s email address, or a team name prefixed with team:, e.g. team:finops.

Type: Optional[Sequence[str]]

tags

Tags for filtering and organizing. These tags are not attached to runs of the asset.

Type: Optional[Mapping[str, str]]

Source assets​

Refer to the External asset dependencies documentation for more information.

class dagster.SourceAsset
deprecated

This API will be removed in version 2.0.0. Use AssetSpec instead. If using the SourceAsset io_manager_key property, use AssetSpec(...).with_io_manager_key(...)..

A SourceAsset represents an asset that will be loaded by (but not updated by) Dagster.

key

The key of the asset.

Type: Union[AssetKey, Sequence[str], str]

metadata

Metadata associated with the asset.

Type: Mapping[str, MetadataValue]

io_manager_key

The key for the IOManager that will be used to load the contents of the asset when it’s used as an input to other assets inside a job.

Type: Optional[str]

io_manager_def

(Experimental) The definition of the IOManager that will be used to load the contents of the asset when it’s used as an input to other assets inside a job.

Type: Optional[IOManagerDefinition]

resource_defs

(Experimental) resource definitions that may be required by the dagster.IOManagerDefinitiondagster.IOManagerDefinition provided in the io_manager_def argument.

Type: Optional[Mapping[str, ResourceDefinition]]

description

The description of the asset.

Type: Optional[str]

partitions_def

Defines the set of partition keys that compose the asset.

Type: Optional[PartitionsDefinition]

observe_fn

Type: Optional[SourceAssetObserveFunction]

op_tags

A dictionary of tags for the op that computes the asset. Frameworks may expect and require certain metadata to be attached to a op. Values that are not strings will be json encoded and must meet the criteria that json.loads(json.dumps(value)) == value.

Type: Optional[Dict[str, Any]]

auto_observe_interval_minutes

While the asset daemon is turned on, a run of the observation function for this asset will be launched at this interval. observe_fn must be provided.

Type: Optional[float]

freshness_policy

A constraint telling Dagster how often this asset is intended to be updated with respect to its root data.

Type: FreshnessPolicy

tags

Tags for filtering and organizing. These tags are not attached to runs of the asset.

Type: Optional[Mapping[str, str]]

property is_observable

Whether the asset is observable.

Type: bool

property op

The OpDefinition associated with the observation function of an observable source asset.

Throws an error if the asset is not observable.

Type: OpDefinition

@dagster.observable_source_asset
experimental

This API may break in future versions, even between dot releases.

Create a SourceAsset with an associated observation function.

The observation function of a source asset is wrapped inside of an op and can be executed as part of a job. Each execution generates an AssetObservation event associated with the source asset. The source asset observation function should return a DataVersion, a ~dagster.DataVersionsByPartition, or an ObserveResultObserveResult.

Parameters:

  • name (Optional[str]) – The name of the source asset. If not provided, defaults to the name of the
  • key_prefix (Optional[Union[str, Sequence[str]]]) – If provided, the source asset’s key is the
  • metadata (Mapping[str, RawMetadataValue]) – Metadata associated with the asset.
  • io_manager_key (Optional[str]) – The key for the IOManager that will be used to load the contents of
  • io_manager_def (Optional[IOManagerDefinitionIOManagerDefinition]) – (Experimental) The definition of the IOManager that will be used to load the contents of
  • description (Optional[str]) – The description of the asset.
  • group_name (Optional[str]) – A string name used to organize multiple assets into groups. If not provided,
  • required_resource_keys (Optional[Set[str]]) – Set of resource keys required by the observe op.
  • resource_defs (Optional[Mapping[str, ResourceDefinitionResourceDefinition]]) – (Experimental) resource
  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – Defines the set of partition keys that
  • op_tags (Optional[Dict[str, Any]]) – A dictionary of tags for the op that computes the asset.
  • tags (Optional[Mapping[str, str]]) – Tags for filtering and organizing. These tags are not
  • observe_fn (Optional[SourceAssetObserveFunction]) – Observation function for the source asset.
  • automation_condition (Optional[AutomationConditionAutomationCondition]) – A condition describing when Dagster
class dagster.ObserveResult
experimental

This API may break in future versions, even between dot releases.

An object representing a successful observation of an asset. These can be returned from an @observable_source_asset decorated function to pass metadata.

asset_key

The asset key. Optional to include.

Type: Optional[AssetKey]

metadata

Metadata to record with the corresponding AssetObservation event.

Type: Optional[RawMetadataMapping]

check_results

Check results to record with the corresponding AssetObservation event.

Type: Optional[Sequence[AssetCheckResult]]

data_version

The data version of the asset that was observed.

Type: Optional[DataVersion]

tags

Tags to record with the corresponding AssetObservation event.

Type: Optional[Mapping[str, str]]

Dependencies​

class dagster.AssetDep

Specifies a dependency on an upstream asset.

asset

The upstream asset to depend on.

Type: Union[AssetKey, str, AssetSpec, AssetsDefinition, SourceAsset]

partition_mapping

Defines what partitions to depend on in the upstream asset. If not provided and the upstream asset is partitioned, defaults to the default partition mapping for the partitions definition, which is typically maps partition keys to the same partition keys in upstream assets.

Type: Optional[PartitionMapping]

Examples:

upstream_asset = AssetSpec("upstream_asset")
downstream_asset = AssetSpec(
    "downstream_asset",
    deps=[
        AssetDep(
            upstream_asset,
            partition_mapping=TimeWindowPartitionMapping(start_offset=-1, end_offset=-1)
        )
    ]
)
class dagster.AssetIn

Defines an asset dependency.

key_prefix

If provided, the asset’s key is the concatenation of the key_prefix and the input name. Only one of the “key_prefix” and “key” arguments should be provided.

Type: Optional[Union[str, Sequence[str]]]

key

The asset’s key. Only one of the “key_prefix” and “key” arguments should be provided.

Type: Optional[Union[str, Sequence[str], AssetKey]]

metadata

A dict of the metadata for the input. For example, if you only need a subset of columns from an upstream table, you could include that in metadata and the IO manager that loads the upstream table could use the metadata to determine which columns to load.

Type: Optional[Dict[str, Any]]

partition_mapping

Defines what partitions to depend on in the upstream asset. If not provided, defaults to the default partition mapping for the partitions definition, which is typically maps partition keys to the same partition keys in upstream assets.

Type: Optional[PartitionMapping]

dagster_type

Allows specifying type validation functions that will be executed on the input of the decorated function before it runs.

Type: DagsterType

Asset jobs​

Asset jobs enable the automation of asset materializations. Dagster’s asset selection syntax can be used to select assets and assign them to a job.

dagster.define_asset_job

Creates a definition of a job which will either materialize a selection of assets or observe a selection of source assets. This will only be resolved to a JobDefinition once placed in a code location.

Parameters:

  • name (str) – The name for the job.

  • selection (Union[str, Sequence[str], Sequence[AssetKeyAssetKey], Sequence[Union[AssetsDefinitionAssetsDefinition, SourceAssetSourceAsset]], AssetSelectionAssetSelection]) –

    The assets that will be materialized or observed when the job is run.

    The selected assets must all be included in the assets that are passed to the assets argument of the Definitions object that this job is included on.

    The string “my_asset*” selects my_asset and all downstream assets within the code location. A list of strings represents the union of all assets selected by strings within the list.

  • config –

    Describes how the Job is parameterized at runtime.

    If no value is provided, then the schema for the job’s run config is a standard format based on its ops and resources.

    If a dictionary is provided, then it must conform to the standard config schema, and it will be used as the job’s run config for the job whenever the job is executed. The values provided will be viewable and editable in the Dagster UI, so be careful with secrets.

  • tags (Optional[Mapping[str, object]]) – A set of key-value tags that annotate the job and can

  • run_tags (Optional[Mapping[str, object]]) – A set of key-value tags that will be automatically attached to runs launched by this

  • metadata (Optional[Mapping[str, RawMetadataValue]]) – Arbitrary metadata about the job.

  • description (Optional[str]) – A description for the Job.

  • executor_def (Optional[ExecutorDefinitionExecutorDefinition]) – How this Job will be executed. Defaults to multi_or_in_process_executormulti_or_in_process_executor,

  • op_retry_policy (Optional[RetryPolicyRetryPolicy]) – The default retry policy for all ops that compute assets in this job.

  • partitions_def (Optional[PartitionsDefinitionPartitionsDefinition]) – deprecated

Returns: The job, which can be placed inside a code location.Return type: UnresolvedAssetJobDefinition Examples:

# A job that targets all assets in the code location:
@asset
def asset1():
    ...

defs = Definitions(
    assets=[asset1],
    jobs=[define_asset_job("all_assets")],
)

# A job that targets a single asset
@asset
def asset1():
    ...

defs = Definitions(
    assets=[asset1],
    jobs=[define_asset_job("all_assets", selection=[asset1])],
)

# A job that targets all the assets in a group:
defs = Definitions(
    assets=assets,
    jobs=[define_asset_job("marketing_job", selection=AssetSelection.groups("marketing"))],
)

@observable_source_asset
def source_asset():
    ...

# A job that observes a source asset:
defs = Definitions(
    assets=assets,
    jobs=[define_asset_job("observation_job", selection=[source_asset])],
)

# Resources are supplied to the assets, not the job:
@asset(required_resource_keys=\{"slack_client"})
def asset1():
    ...

defs = Definitions(
    assets=[asset1],
    jobs=[define_asset_job("all_assets")],
    resources=\{"slack_client": prod_slack_client},
)
class dagster.AssetSelection

An AssetSelection defines a query over a set of assets and asset checks, normally all that are defined in a code location.

You can use the “|”, “&”, and “-” operators to create unions, intersections, and differences of selections, respectively.

AssetSelections are typically used with define_asset_job()define_asset_job().

By default, selecting assets will also select all of the asset checks that target those assets.

Examples:

# Select all assets in group "marketing":
AssetSelection.groups("marketing")

# Select all assets in group "marketing", as well as the asset with key "promotion":
AssetSelection.groups("marketing") | AssetSelection.assets("promotion")

# Select all assets in group "marketing" that are downstream of asset "leads":
AssetSelection.groups("marketing") & AssetSelection.assets("leads").downstream()

# Select a list of assets:
AssetSelection.assets(*my_assets_list)

# Select all assets except for those in group "marketing"
AssetSelection.all() - AssetSelection.groups("marketing")

# Select all assets which are materialized by the same op as "projections":
AssetSelection.assets("projections").required_multi_asset_neighbors()

# Select all assets in group "marketing" and exclude their asset checks:
AssetSelection.groups("marketing") - AssetSelection.all_asset_checks()

# Select all asset checks that target a list of assets:
AssetSelection.checks_for_assets(*my_assets_list)

# Select a specific asset check:
AssetSelection.checks(my_asset_check)
static all

Returns a selection that includes all assets and their asset checks.

Parameters: include_sources (bool) – experimental

static all_asset_checks

Returns a selection that includes all asset checks.

static assets

Returns a selection that includes all of the provided assets and asset checks that target them.

Parameters: *assets_defs (Union[AssetsDefinitionAssetsDefinition, str, Sequence[str], AssetKeyAssetKey]) – The assets to select. Examples:

AssetSelection.assets(AssetKey(["a"]))

AssetSelection.assets("a")

AssetSelection.assets(AssetKey(["a"]), AssetKey(["b"]))

AssetSelection.assets("a", "b")

@asset
def asset1():
    ...

AssetSelection.assets(asset1)

asset_key_list = [AssetKey(["a"]), AssetKey(["b"])]
AssetSelection.assets(*asset_key_list)
static checks

Returns a selection that includes all of the provided asset checks or check keys.

static checks_for_assets

Returns a selection with the asset checks that target the provided assets.

Parameters: *assets_defs (Union[AssetsDefinitionAssetsDefinition, str, Sequence[str], AssetKeyAssetKey]) – The assets to select checks for.

downstream

Returns a selection that includes all assets that are downstream of any of the assets in this selection, selecting the assets in this selection by default. Includes the asset checks targeting the returned assets. Iterates through each asset in this selection and returns the union of all downstream assets.

depth (Optional[int]): If provided, then only include assets to the given depth. A depth of 2 means all assets that are children or grandchildren of the assets in this selection.

include_self (bool): If True, then include the assets in this selection in the result. If the include_self flag is False, return each downstream asset that is not part of the original selection. By default, set to True.

static groups

Returns a selection that includes materializable assets that belong to any of the provided groups and all the asset checks that target them.

Parameters: include_sources (bool) – If True, then include source assets matching the group in the selection.

static key_prefixes

Returns a selection that includes assets that match any of the provided key prefixes and all the asset checks that target them.

Parameters: include_sources (bool) – If True, then include source assets matching the key prefix(es) in the selection. Examples:

# match any asset key where the first segment is equal to "a" or "b"
# e.g. AssetKey(["a", "b", "c"]) would match, but AssetKey(["abc"]) would not.
AssetSelection.key_prefixes("a", "b")

# match any asset key where the first two segments are ["a", "b"] or ["a", "c"]
AssetSelection.key_prefixes(["a", "b"], ["a", "c"])
static keys
deprecated

This API will be removed in version 2.0. Use AssetSelection.assets instead..

Returns a selection that includes assets with any of the provided keys and all asset checks that target them.

Deprecated: use AssetSelection.assets instead.

Examples:

AssetSelection.keys(AssetKey(["a"]))

AssetSelection.keys("a")

AssetSelection.keys(AssetKey(["a"]), AssetKey(["b"]))

AssetSelection.keys("a", "b")

asset_key_list = [AssetKey(["a"]), AssetKey(["b"])]
AssetSelection.keys(*asset_key_list)
materializable

Given an asset selection, returns a new asset selection that contains all of the assets that are materializable. Removes any assets which are not materializable.

required_multi_asset_neighbors

Given an asset selection in which some assets are output from a multi-asset compute op which cannot be subset, returns a new asset selection that contains all of the assets required to execute the original asset selection. Includes the asset checks targeting the returned assets.

roots

Given an asset selection, returns a new asset selection that contains all of the root assets within the original asset selection. Includes the asset checks targeting the returned assets.

A root asset is an asset that has no upstream dependencies within the asset selection. The root asset can have downstream dependencies outside of the asset selection.

Because mixed selections of source and materializable assets are currently not supported, keys corresponding to SourceAssets will not be included as roots. To select source assets, use the upstream_source_assets method.

sinks

Given an asset selection, returns a new asset selection that contains all of the sink assets within the original asset selection. Includes the asset checks targeting the returned assets.

A sink asset is an asset that has no downstream dependencies within the asset selection. The sink asset can have downstream dependencies outside of the asset selection.

sources
deprecated

This API will be removed in version 2.0. Use AssetSelection.roots instead..

Given an asset selection, returns a new asset selection that contains all of the root assets within the original asset selection. Includes the asset checks targeting the returned assets.

A root asset is a materializable asset that has no upstream dependencies within the asset selection. The root asset can have downstream dependencies outside of the asset selection.

Because mixed selections of source and materializable assets are currently not supported, keys corresponding to SourceAssets will not be included as roots. To select source assets, use the upstream_source_assets method.

static tag
experimental

This API may break in future versions, even between dot releases.

Returns a selection that includes materializable assets that have the provided tag, and all the asset checks that target them.

Parameters: include_sources (bool) – If True, then include source assets matching the group in the selection.

upstream

Returns a selection that includes all materializable assets that are upstream of any of the assets in this selection, selecting the assets in this selection by default. Includes the asset checks targeting the returned assets. Iterates through each asset in this selection and returns the union of all upstream assets.

Because mixed selections of source and materializable assets are currently not supported, keys corresponding to SourceAssets will not be included as upstream of regular assets.

Parameters:

  • depth (Optional[int]) – If provided, then only include assets to the given depth. A depth
  • include_self (bool) – If True, then include the assets in this selection in the result.
upstream_source_assets

Given an asset selection, returns a new asset selection that contains all of the source assets that are parents of assets in the original selection. Includes the asset checks targeting the returned assets.

without_checks

Removes all asset checks in the selection.

Code locations​

Loading assets and asset jobs into a code location makes them available to Dagster tools like the UI, CLI, and GraphQL API.

dagster.load_assets_from_modules

Constructs a list of assets and source assets from the given modules.

Parameters:

  • modules (Iterable[ModuleType]) – The Python modules to look for assets inside.
  • group_name (Optional[str]) – Group name to apply to the loaded assets. The returned assets will be copies of the
  • key_prefix (Optional[Union[str, Sequence[str]]]) – Prefix to prepend to the keys of the loaded assets. The returned assets will be copies
  • freshness_policy (Optional[FreshnessPolicy]) – FreshnessPolicy to apply to all the loaded
  • automation_condition (Optional[AutomationConditionAutomationCondition]) – AutomationCondition to apply
  • backfill_policy (Optional[AutoMaterializePolicy]) – BackfillPolicy to apply to all the loaded assets.
  • source_key_prefix (bool) – Prefix to prepend to the keys of loaded SourceAssets. The returned

Returns: A list containing assets and source assets defined in the given modules.Return type: Sequence[Union[AssetsDefinition, SourceAsset]]

dagster.load_assets_from_current_module

Constructs a list of assets, source assets, and cacheable assets from the module where this function is called.

Parameters:

  • group_name (Optional[str]) – Group name to apply to the loaded assets. The returned assets will be copies of the
  • key_prefix (Optional[Union[str, Sequence[str]]]) – Prefix to prepend to the keys of the loaded assets. The returned assets will be copies
  • freshness_policy (Optional[FreshnessPolicy]) – FreshnessPolicy to apply to all the loaded
  • automation_condition (Optional[AutomationConditionAutomationCondition]) – AutomationCondition to apply
  • backfill_policy (Optional[AutoMaterializePolicy]) – BackfillPolicy to apply to all the loaded assets.
  • source_key_prefix (bool) – Prefix to prepend to the keys of loaded SourceAssets. The returned

Returns: A list containing assets, source assets, and cacheable assets defined in the module.Return type: Sequence[Union[AssetsDefinition, SourceAsset, CachableAssetsDefinition]]

dagster.load_assets_from_package_module

Constructs a list of assets and source assets that includes all asset definitions, source assets, and cacheable assets in all sub-modules of the given package module.

A package module is the result of importing a package.

Parameters:

  • package_module (ModuleType) – The package module to looks for assets inside.
  • group_name (Optional[str]) – Group name to apply to the loaded assets. The returned assets will be copies of the
  • key_prefix (Optional[Union[str, Sequence[str]]]) – Prefix to prepend to the keys of the loaded assets. The returned assets will be copies
  • freshness_policy (Optional[FreshnessPolicy]) – FreshnessPolicy to apply to all the loaded
  • automation_condition (Optional[AutomationConditionAutomationCondition]) – AutomationCondition to apply
  • backfill_policy (Optional[AutoMaterializePolicy]) – BackfillPolicy to apply to all the loaded assets.
  • source_key_prefix (bool) – Prefix to prepend to the keys of loaded SourceAssets. The returned

Returns: A list containing assets, source assets, and cacheable assets defined in the module.Return type: Sequence[Union[AssetsDefinition, SourceAsset, CacheableAssetsDefinition]]

dagster.load_assets_from_package_name

Constructs a list of assets, source assets, and cacheable assets that includes all asset definitions and source assets in all sub-modules of the given package.

Parameters:

  • package_name (str) – The name of a Python package to look for assets inside.
  • group_name (Optional[str]) – Group name to apply to the loaded assets. The returned assets will be copies of the
  • key_prefix (Optional[Union[str, Sequence[str]]]) – Prefix to prepend to the keys of the loaded assets. The returned assets will be copies
  • freshness_policy (Optional[FreshnessPolicy]) – FreshnessPolicy to apply to all the loaded
  • auto_materialize_policy (Optional[AutoMaterializePolicy]) – AutoMaterializePolicy to apply
  • backfill_policy (Optional[AutoMaterializePolicy]) – BackfillPolicy to apply to all the loaded assets.
  • source_key_prefix (bool) – Prefix to prepend to the keys of loaded SourceAssets. The returned

Returns: A list containing assets, source assets, and cacheable assets defined in the module.Return type: Sequence[Union[AssetsDefinition, SourceAsset, CacheableAssetsDefinition]]

Observations​

Refer to the Asset observation documentation for more information.

class dagster.AssetObservation

Event that captures metadata about an asset at a point in time.

Parameters:

  • asset_key (Union[str, List[str], AssetKeyAssetKey]) – A key to identify the asset.
  • partition (Optional[str]) – The name of a partition of the asset that the metadata
  • tags (Optional[Mapping[str, str]]) – A mapping containing tags for the observation.
  • metadata (Optional[Dict[str, Union[str, float, int, MetadataValueMetadataValue]]]) – Arbitrary metadata about the asset. Keys are displayed string labels, and values are

Declarative Automation​

Refer to the Declarative Automation documentation for more information.

class dagster.AutomationCondition

An AutomationCondition represents a condition of an asset that impacts whether it should be automatically executed. For example, you can have a condition which becomes true whenever the code version of the asset is changed, or whenever an upstream dependency is updated.

from dagster import AutomationCondition, asset

@asset(automation_condition=AutomationCondition.on_cron("0 0 * * *"))
def my_asset(): ...

AutomationConditions may be combined together into expressions using a variety of operators.

from dagster import AssetSelection, AutomationCondition, asset

# any dependencies from the "important" group are missing
any_important_deps_missing = AutomationCondition.any_deps_match(
    AutomationCondition.missing(),
).allow(AssetSelection.groups("important"))

# there is a new code version for this asset since the last time it was requested
new_code_version = AutomationCondition.code_version_changed().since(
    AutomationCondition.newly_requested()
)

# there is a new code version and no important dependencies are missing
my_condition = new_code_version & ~any_important_deps_missing

@asset(automation_condition=my_condition)
def my_asset(): ...
static all_checks_match

Returns an AutomationCondition that is true for an asset partition if all of its checks evaluate to True for the given condition.

Parameters:

  • condition (AutomationConditionAutomationCondition) – The AutomationCondition that will be evaluated against
  • blocking_only (bool) – Determines if this condition will only be evaluated against blocking
static all_deps_blocking_checks_passed

Returns an AutomationCondition that is true for any partition where all upstream blocking checks have passed, or will be requested on this tick.

In-tick requests are allowed to enable creating runs that target both a parent with blocking checks and a child. Even though the checks have not currently passed, if they fail within the run, the run machinery will prevent the child from being materialized.

static all_deps_match

Returns an AutomationCondition that is true for a if at least one partition of the all of the target’s dependencies evaluate to True for the given condition.

Parameters: condition (AutomationConditionAutomationCondition) – The AutomationCondition that will be evaluated against this target’s dependencies.

static all_deps_updated_since_cron

Returns an AutomatonCondition that is true if all of the target’s dependencies have updated since the latest tick of the provided cron schedule.

static any_checks_match

Returns an AutomationCondition that is true for if at least one of the target’s checks evaluate to True for the given condition.

Parameters:

  • condition (AutomationConditionAutomationCondition) – The AutomationCondition that will be evaluated against
  • blocking_only (bool) – Determines if this condition will only be evaluated against blocking
static any_deps_in_progress

Returns an AutomationCondition that is true if the target has at least one dependency that is in progress.

static any_deps_match

Returns an AutomationCondition that is true for a if at least one partition of the any of the target’s dependencies evaluate to True for the given condition.

Parameters: condition (AutomationConditionAutomationCondition) – The AutomationCondition that will be evaluated against this target’s dependencies.

static any_deps_missing

Returns an AutomationCondition that is true if the target has at least one dependency that is missing, and will not be requested on this tick.

static any_deps_updated

Returns an AutomationCondition that is true if the target has at least one dependency that has updated since the previous tick, or will be requested on this tick.

Will ignore parent updates if the run that updated the parent also plans to update the asset or check that this condition is applied to.

static any_downstream_conditions
experimental

This API may break in future versions, even between dot releases.

Returns an AutomationCondition which represents the union of all distinct downstream conditions.

static asset_matches

Returns an AutomationCondition that is true if this condition is true for the given key.

static backfill_in_progress

Returns an AutomationCondition that is true if the target is part of an in-progress backfill.

static check_failed

Returns an AutomationCondition that is true for an asset check if it has evaluated against the latest materialization of an asset and failed.

static check_passed

Returns an AutomationCondition that is true for an asset check if it has evaluated against the latest materialization of an asset and passed.

static code_version_changed

Returns an AutomationCondition that is true if the target’s code version has been changed since the previous tick.

static cron_tick_passed

Returns an AutomationCondition that is whenever a cron tick of the provided schedule is passed.

static eager

Returns an AutomationCondition which will cause a target to be executed if any of its dependencies update, and will execute missing partitions if they become missing after this condition is applied to the target.

This will not execute targets that have any missing or in progress dependencies, or are currently in progress.

For time partitioned assets, only the latest time partition will be considered.

static execution_failed

Returns an AutomationCondition that is true if the latest execution of the target failed.

static in_latest_time_window

Returns an AutomationCondition that is true when the target it is within the latest time window.

Parameters: lookback_delta (Optional, datetime.timedelta) – If provided, the condition will return all partitions within the provided delta of the end of the latest time window. For example, if this is used on a daily-partitioned asset with a lookback_delta of 48 hours, this will return the latest two partitions.

static in_progress

Returns an AutomationCondition that is true for an asset partition if it is part of an in-progress run or backfill.

static initial_evaluation

Returns an AutomationCondition that is true on the first evaluation of the expression.

static missing

Returns an AutomationCondition that is true if the target has not been executed.

static newly_missing

Returns an AutomationCondition that is true on the tick that the target becomes missing.

static newly_requested

Returns an AutomationCondition that is true if the target was requested on the previous tick.

static newly_updated

Returns an AutomationCondition that is true if the target has been updated since the previous tick.

static on_cron

Returns an AutomationCondition which will cause a target to be executed on a given cron schedule, after all of its dependencies have been updated since the latest tick of that cron schedule.

For time partitioned assets, only the latest time partition will be considered.

static on_missing

Returns an AutomationCondition which will execute partitions of the target that are added after this condition is applied to the asset.

This will not execute targets that have any missing dependencies.

For time partitioned assets, only the latest time partition will be considered.

static run_in_progress

Returns an AutomationCondition that is true if the target is part of an in-progress run.

static will_be_requested

Returns an AutomationCondition that is true if the target will be requested this tick.

class dagster.AutomationConditionSensorDefinition
experimental

This API may break in future versions, even between dot releases.

Targets a set of assets and repeatedly evaluates all the AutomationConditions on all of those assets to determine which to request runs for.

Parameters:

  • name – The name of the sensor.
  • target (Union[str, Sequence[str], Sequence[AssetKeyAssetKey], Sequence[Union[AssetsDefinitionAssetsDefinition, SourceAssetSourceAsset]], AssetSelectionAssetSelection]) – A selection of assets to evaluate AutomationConditions of and request runs for.
  • tags (Optional[Mapping[str, str]]) – A set of key-value tags that annotate the sensor and can
  • run_tags (Optional[Mapping[str, Any]]) – Tags that will be automatically attached to runs launched by this sensor.
  • metadata (Optional[Mapping[str, object]]) – A set of metadata entries that annotate the
  • default_status (DefaultSensorStatus) – Whether the sensor starts as running or not. The default
  • minimum_interval_seconds (Optional[int]) – The frequency at which to try to evaluate the
  • description (Optional[str]) – A human-readable description of the sensor.
  • emit_backfills (bool) – If set to True, will emit a backfill on any tick where more than one partition
  • use_user_code_server (bool) – (experimental) If set to True, this sensor will be evaluated in the user
  • default_condition (Optional[AutomationConditionAutomationCondition]) – (experimental) If provided, this condition will

Examples:

import dagster as dg

# automation condition sensor that defaults to running
defs1 = dg.Definitions(
    assets=...,
    sensors=[
        dg.AutomationConditionSensorDefinition(
            name="automation_condition_sensor",
            default_status=dg.DefaultSensorStatus.RUNNING,
        ),
    ]
)

# one automation condition sensor per group
defs2 = dg.Definitions(
    assets=...,
    sensors=[
        dg.AutomationConditionSensorDefinition(
            name="raw_data_automation_condition_sensor",
            target=dg.AssetSelection.groups("raw_data"),
        ),
        dg.AutomationConditionSensorDefinition(
            name="ml_automation_condition_sensor",
            target=dg.AssetSelection.groups("machine_learning"),
        ),
    ]
)

Asset values​

class dagster.AssetValueLoader

Caches resource definitions that are used to load asset values across multiple load invocations.

Should not be instantiated directly. Instead, use get_asset_value_loader()get_asset_value_loader().

load_asset_value

Loads the contents of an asset as a Python object.

Invokes load_input on the IOManagerIOManager associated with the asset.

Parameters:

  • asset_key (Union[AssetKeyAssetKey, Sequence[str], str]) – The key of the asset to load.
  • python_type (Optional[Type]) – The python type to load the asset as. This is what will
  • partition_key (Optional[str]) – The partition of the asset to load.
  • input_definition_metadata (Optional[Dict[str, Any]]) – Input metadata to pass to the IOManagerIOManager
  • resource_config (Optional[Any]) – A dictionary of resource configurations to be passed

Returns: The contents of an asset as a Python object.