registry
etl_craft.handlers.registry ¶
What a task handler receives and returns, and which handler runs each HANDLER value.
A handler is a function (context, engine_db) -> HandlerResult. It runs inside the task's own
process; a problem it can explain is a HandlerError, which records the task FAILED with
that message. Handlers are registered by import path, so a task process imports only the
handler it runs.
HANDLERS
module-attribute
¶
HANDLERS: dict[str, str] = {'SQL': 'etl_craft.handlers.sql:run', 'BUSINESS_RULES': 'etl_craft.handlers.business_rules:run', 'PYTHON': 'etl_craft.handlers.python_scripts:run', 'EMAIL_ALERT': 'etl_craft.handlers.email_alert:run'}
Each HANDLER value and the module:function that runs it.
COMMON_PARAMETERS
module-attribute
¶
COMMON_PARAMETERS = frozenset({'TASK_TIMEOUT_SECONDS', 'DOCUMENTATION', 'RETRIES', 'RETRY_DELAY_SECONDS', 'RETRY_BACKOFF'})
The task parameters every handler's tasks may set.
TaskContext
dataclass
¶
Everything a handler needs about the task it runs, resolved before it starts.
force is set by run --force. rerun is set when the task already ended SUCCESS
or SKIPPED under the run and is run again, so nothing it did before is skipped.
run_date is the date the run runs as of (SQL's $$run_date), and backfill says the
run is part of a backfill.
HandlerResult
dataclass
¶
Bases: Counts
The counts a handler reports for AUD_TASK_RUN_LOG, and any named values.
variables are the values a task reports by name (an ingestion script's declared return
values); they are listed in TASK_LOG as NAME = value lines. offset is stored
together with the attempt's SUCCESS outcome, rather than by the handler.
format_task_log ¶
Render result as NAME = value lines: its counts, then its variables.
resolve_handler ¶
Import and return the handler for name; HandlerError when none is installed.