Skip to content

email_alert

etl_craft.handlers.email_alert

HANDLER=EMAIL_ALERT: one email about how a pipeline run went.

An alert task sits at the end of its pipeline, after the tasks it reports on. It works out the run's outcome from every other task's status under the run:

  • FAILED when any task failed;
  • COMPLETED_WITH_ERRORS when none failed but the run was not clean: a task was skipped, needed a retry, succeeded with an error message, or has not finished; or when Enforce_sla is on and the run is already past its pipeline's SLA;
  • SUCCESS otherwise.

Other alert tasks are left out, so two alerts (say, one to operations on FAILED and one to stakeholders on SUCCESS) agree. The task's parameters:

  • EMAIL_TO: the recipients, separated by |; required.
  • EMAIL_SUBJECT and EMAIL_BODY: the subject and an opening paragraph, with EMAIL_SUBJECT_<OUTCOME> / EMAIL_BODY_<OUTCOME> for one outcome only. A subject is required; a body is required unless EMAIL_PIPELINES is set.
  • EMAIL_ON_STATUS: the outcomes to send on, separated by |; all when unset. On another outcome the task succeeds without sending, and its task log says why.
  • EMAIL_PIPELINES: ALL, or pipeline codes separated by |, adds a table of each one's latest run with a section per pipeline listing its tasks.

Subjects and bodies may use $$status (the outcome), $$pipeline_id (the pipeline definition), $$pipeline_run_id (the pipeline execution) and $$task_run_id (this task execution), $$pipeline_code, $$task_code and $$error_message (the error messages of the tasks this one watches through a FAILURE dependency). Any other $$ token fails the task.

PARAMETERS module-attribute 

PARAMETERS = frozenset({'EMAIL_TO', 'EMAIL_SUBJECT', 'EMAIL_BODY', 'EMAIL_ON_STATUS', 'EMAIL_PIPELINES', *(f'EMAIL_{part}_{outcome}' for part in ('SUBJECT', 'BODY') for outcome in OUTCOMES)})

The task parameters an alert task reads.

PipelineDigest dataclass

A pipeline's latest run, and its tasks, for the status table.

run 

run(context: TaskContext, engine_db: Engine) -> HandlerResult

Work out the run's outcome and send the alert, or record why none was sent.

alert_parameter_problems 

alert_parameter_problems(params: Mapping[str, str]) -> list[str]

Return what is wrong with an alert task's parameters, for every outcome it can send on.

A run sends on one outcome, so it notices a missing subject or body only when that outcome happens; this checks each outcome EMAIL_ON_STATUS allows.

run_outcome 

run_outcome(statuses: Sequence[TaskStatus], *, exclude_task_id: int, sla_missed: bool) -> str

Return the run's outcome from its tasks' statuses; alert tasks are left out.

substitute 

substitute(text: str, values: Mapping[str, str], setting: str) -> str

Replace the $$ tokens in text; HandlerError naming an unknown one.

sla_so_far 

sla_so_far(conn: Connection, context: TaskContext) -> SlaResult | None

Return the SLA the run has already missed when Enforce_sla is on, else None.

latest_run_of 

latest_run_of(conn: Connection, pipeline_code: str) -> PipelineDigest

Return a pipeline's latest run; an unknown code is UNKNOWN, one never run NEVER_RUN.

One wrong code in EMAIL_PIPELINES shows as its own row rather than losing the report on every other pipeline.

digest_html 

digest_html(entries: Sequence[PipelineDigest]) -> str

Render the pipelines' latest runs: a summary table, then a section per pipeline.