supervisor
etl_craft.execution.supervisor ¶
Run child processes with a time limit, captured output and bounded parallelism.
Every child is a freshly started interpreter or program, never a fork of the engine, so it inherits no open connections, locks or threads. Each child leads its own process group; on timeout the whole group is stopped, including anything the child started.
CANCEL_POLL_SECONDS
module-attribute
¶
How often a child that can be cancelled checks whether it has been.
TAIL_BYTES
module-attribute
¶
How much of the end of a child's output ChildResult.output_tail keeps.
KILL_GRACE_SECONDS
module-attribute
¶
How long a timed-out child's process group has to exit after SIGTERM before SIGKILL.
ChildSpec
dataclass
¶
One child process to run.
timeout_seconds of None or 0 means no limit. The child's standard output and
standard error are appended to log_path when given, and to a temporary file otherwise.
ChildResult
dataclass
¶
How a child process ended.
returncode is negative when a signal ended the child. output_tail is the end of
what it wrote during this run, decoded as UTF-8 with invalid bytes replaced. cancelled
is true when the caller stopped it.
run_child ¶
run_child(spec: ChildSpec, *, tail_bytes: int = TAIL_BYTES, kill_grace_seconds: float = KILL_GRACE_SECONDS, cancel: Event | None = None, on_started: Callable[[int], None] | None = None) -> ChildResult
Run spec to completion and return how it ended.
When the time limit passes, or cancel is set, the child's process group gets SIGTERM,
then SIGKILL after kill_grace_seconds. If waiting is interrupted, for example by Ctrl-C,
the group is stopped the same way before the exception propagates.