log

etl_craft.core.log

Logging output for the etl_craft logger tree.

Modules log through logging.getLogger(__name__). Importing etl-craft configures nothing; the command line calls configure once to choose the level and the format, text or JSON lines.

log_context attaches fields to every record written inside it, such as the pipeline, task and run a task process is working on, so each line of a failure's log says where it came from.

ContextFilter

Bases: Filter

Copy the current log_context fields onto each record.

Each field becomes a record attribute (a JSON field), and context holds them as [name=value ...] for the text format, or an empty string outside any context.

filter

filter(record: LogRecord) -> bool

Add the context to record; never drop it.

LogFormat

Bases: StrEnum

How each log record is written.

JsonFormatter

Bases: Formatter

Write each record as one JSON object per line.

The object has time (UTC, ISO 8601), level, logger and message, then every field passed through extra=, then exception when the record carries one.

format

format(record: LogRecord) -> str

Return the record as a single-line JSON object.

log_context

log_context(**fields: object) -> Iterator[None]

Attach fields to every record logged inside the with body, nested contexts merged.

parse_level

parse_level(level: str | int) -> int

Return the logging level for a name such as info, or a numeric level unchanged.

Raises UsageError for a name outside LEVELS.

parse_format

parse_format(fmt: str | LogFormat) -> LogFormat

Return the LogFormat for text or json; raises UsageError otherwise.

configure

configure(level: str | int = 'INFO', fmt: str | LogFormat = LogFormat.TEXT, stream: IO[str] | None = None) -> logging.Handler

Send etl_craft records at level and above to stream (stderr by default).

Returns the installed handler. Calling it again replaces the handler it installed before.

capture_all_loggers

capture_all_loggers() -> Iterator[None]

Write records of loggers outside etl_craft (a script's own) like ours, in the body.

They go to the stream configure set up, with the same format and context fields, at the same level. HTTP and SDK loggers stay at WARNING or above unless our level is DEBUG. Without a configured handler this does nothing.