Skip to main content

Dagster & SSH/SFTP

This integration provides a resource for SSH remote execution using Paramiko. It allows you to establish secure connections to networked resources and execute commands remotely. The integration also provides an SFTP client for secure file transfers between the local and remote systems.

Installation

pip install dagster-ssh

Example

from dagster_ssh import SSHResource

import dagster as dg


@dg.asset
def ssh_asset(ssh: SSHResource):
ssh.sftp_get("/path/to/remote.csv", "path/to/local.csv")


defs = dg.Definitions(
assets=[ssh_asset],
resources={"ssh": SSHResource(remote_host="foo.com", key_file="path/to/id_rsa")},
)

About SSH SFTP

The SSH protocol allows for secure remote login with strong authentication to networked resources. It protects network connections with strong encryption. The Dagster library provides direct SSH and SFTP calls from within the execution of your pipelines.