Dagster & Airbyte
Using this integration, you can trigger Airbyte syncs and orchestrate your Airbyte connections from within Dagster, making it easy to chain an Airbyte sync with upstream or downstream steps in your workflow.
Installation
pip install dagster-airbyte
Example
from dagster_airbyte import AirbyteResource, load_assets_from_airbyte_instance
import dagster as dg
# Load all assets from your Airbyte instance
airbyte_assets = load_assets_from_airbyte_instance(
# Connect to your OSS Airbyte instance
AirbyteResource(
host="localhost",
port="8000",
# If using basic auth, include username and password:
username="airbyte",
password=dg.EnvVar("AIRBYTE_PASSWORD"),
)
)
defs = dg.Definitions(
assets=[airbyte_assets],
)
About Airbyte
Airbyte is an open source data integration engine that helps you consolidate your SaaS application and database data into your data warehouses, lakes and databases.