dependencies

etl_craft.engine.repository.dependencies ¶

Dependencies: a pipeline's graph, and the edges the cross-pipeline gates check.

PipelineGraphData `dataclass` ¶

A pipeline's active tasks and same-pipeline edges, as build_graph takes them.

Edges on tasks in other pipelines are not part of the graph; each task counts them, and cross_pipeline_task_ids names the tasks that have any.

PipelineDependencyEdge `dataclass` ¶

One active dependency of a pipeline on another pipeline.

CrossPipelineTaskEdge `dataclass` ¶

One active dependency of a task on a task in another pipeline.

depends_on_label names the upstream as PIPELINE_CODE.TASK_CODE.

fetch_pipeline_graph ¶

fetch_pipeline_graph(conn: Connection, pipeline_id: int) -> PipelineGraphData

Return the tasks and dependencies of pipeline_id for its dependency graph.

The dependencies of an inactive task are left out: it does not run. An active task that depends on an inactive one in the same pipeline could never run, so it is a MetadataError.

fetch_pipeline_dependency_edges ¶

fetch_pipeline_dependency_edges(conn: Connection, pipeline_id: int) -> list[PipelineDependencyEdge]

Return the active dependencies of pipeline_id on other pipelines.

fetch_cross_pipeline_task_edges ¶

fetch_cross_pipeline_task_edges(conn: Connection, task_id: int) -> list[CrossPipelineTaskEdge]

Return the active dependencies of task_id on tasks in other pipelines.