Spark (dagster-spark)

class dagster_spark.SparkOpError

dagster_spark.define_spark_config

Spark configuration.

See the Spark documentation for reference: https://spark.apache.org/docs/latest/submitting-applications.html

dagster_spark.create_spark_op

dagster_spark.construct_spark_shell_command

Constructs the spark-submit command for a Spark job.

Legacy

dagster_spark.spark_resource ResourceDefinition