validation

etl_craft.engine.repository.validation

What validate reads: every active task, and every active dependency with its upstream.

ActiveTask dataclass

An active task of an active pipeline.

label property

label: str

PIPELINE_CODE.TASK_CODE.

DependencyEdge dataclass

An active dependency of an active task, with what is known of its upstream task.

written_pipeline_id is the row's DEPENDS_ON_PIPELINE_ID; depends_on_pipeline_id is the pipeline the upstream task belongs to.

label property

label: str

The dependent task, PIPELINE_CODE.TASK_CODE.

depends_on_label property

depends_on_label: str

The upstream task, PIPELINE_CODE.TASK_CODE.

PipelineEdge dataclass

An active dependency of an active pipeline on another.

fetch_active_tasks

fetch_active_tasks(conn: Connection) -> list[ActiveTask]

Return every active task of an active pipeline, by pipeline and task code.

fetch_pipeline_parameters

fetch_pipeline_parameters(conn: Connection) -> list[tuple[str, object]]

Return every active pipeline's code and PIPELINE_PARAMETERS as stored.

fetch_task_dependency_edges

fetch_task_dependency_edges(conn: Connection) -> list[DependencyEdge]

Return every active dependency of an active task of an active pipeline.

fetch_pipeline_edges

fetch_pipeline_edges(conn: Connection) -> list[PipelineEdge]

Return every active dependency of an active pipeline, upstream active or not.