trackers

etl_craft.engine.repository.trackers ¶

Cross-pipeline dependency trackers: upstream runs, and the run each dependency last consumed.

LatestRun `dataclass` ¶

An upstream's most recent run, in any status, and when it started.

FinishedRun `dataclass` ¶

An upstream's latest finished run: its status, and whether it wrote any rows.

ConsumedRun `dataclass` ¶

The upstream identity and published revision last consumed by an edge.

GateDecision `dataclass` ¶

An immutable judgement, including the exact upstream snapshot it inspected.

fetch_latest_pipeline_run ¶

fetch_latest_pipeline_run(conn: Connection, pipeline_id: int) -> LatestRun | None

Return the active upstream or its latest started run; exclude queued and backfill runs.

fetch_latest_task_run ¶

fetch_latest_task_run(conn: Connection, task_id: int) -> LatestRun | None

Return the latest row of task_id outside backfill runs, or None.

fetch_latest_finished_pipeline_run ¶

fetch_latest_finished_pipeline_run(conn: Connection, pipeline_id: int, ended_by: datetime) -> FinishedRun | None

Return the latest run of pipeline_id that finished by ended_by, or None.

A pipeline run has data when any of its tasks reported a positive target count.

fetch_latest_finished_task_run ¶

fetch_latest_finished_task_run(conn: Connection, task_id: int) -> FinishedRun | None

Return the latest finished row of task_id, or None.

fetch_average_pipeline_seconds ¶

fetch_average_pipeline_seconds(conn: Connection, pipeline_id: int) -> float | None

Return the average length of the finished runs of pipeline_id, or None.

fetch_average_task_seconds ¶

fetch_average_task_seconds(conn: Connection, task_id: int) -> float | None

Return the average length of the finished runs of task_id, or None.

fetch_pipeline_last_consumed ¶

fetch_pipeline_last_consumed(conn: Connection, pipeline_dependency_id: int) -> ConsumedRun | None

Return the upstream pipeline identity and revision last consumed.

fetch_task_last_consumed ¶

fetch_task_last_consumed(conn: Connection, task_dependency_id: int) -> ConsumedRun | None

Return the upstream task identity and its published pipeline revision last consumed.

record_decisions ¶

record_decisions(conn: Connection, run_id: int, decisions: tuple[GateDecision, ...], *, attempt_id: int | None = None) -> None

Record the admission snapshot in the transaction that creates its run or attempt.

consume_pipeline_decisions ¶

consume_pipeline_decisions(conn: Connection, run_id: int) -> None

Consume only the successful run's recorded satisfied admission snapshots.

consume_task_decisions ¶

consume_task_decisions(conn: Connection, task_id: int, run_id: int, *, attempt_id: int | None = None) -> None

Consume snapshots of successful attempts without rereading upstream history or edges.