Skip to content

auth

etl_craft.config.auth

What each Engine DB, warehouse and mail relay accepts, as data.

The loader checks every connection profile against these tables before anything connects: an auth mode a target does not offer, or a field that mode needs and the profile lacks, is a configuration error. The dialects read the same tables, so the loader and the connection code cannot disagree.

Each auth_fields table maps an auth mode to the profile fields it needs beyond the connection itself; its keys are the modes the target accepts. verified_auth_modes are the modes run against a live service in this project; the others follow the vendor's documentation and are untested, which doctor reports.

AUTH_EXTRA_FIELDS module-attribute 

AUTH_EXTRA_FIELDS = ('key_file', 'cert_file', 'client_id', 'token_url', 'scope', 'issuer', 'region', 'role_arn')

Profile fields that configure an auth mode rather than the connection.

SECRET_AUTH_MODES module-attribute 

SECRET_AUTH_MODES = frozenset({AuthMode.PASSWORD, AuthMode.TOKEN, AuthMode.OAUTH})

Auth modes that always present a secret; the others need one only when a profile names one.

ENGINE_NAMES module-attribute 

ENGINE_NAMES = {'postgres': 'postgresql', 'postgresql': 'postgresql', 'sqlite': 'sqlite'}

Engine.Name spellings, lower-cased, and the Engine DB each selects.

WAREHOUSE_NAMES module-attribute 

WAREHOUSE_NAMES = {'postgres': 'postgresql', 'duckdb': 'duckdb', 'trino': 'trino', 'databricks': 'databricks', 'snowflake': 'snowflake'}

Warehouse.Name spellings, lower-cased, and the SQLAlchemy dialect each selects.

EngineSpec dataclass

An Engine DB the engine can keep its metadata in.

auth_modes property 

auth_modes: frozenset[str]

The auth modes this Engine DB accepts.

WarehouseSpec dataclass

One warehouse dialect: a database and the table format it writes.

preferred_fields are the separate connection fields a profile may give instead of a whole JDBC URL; profile_fields are further settings the dialect reads from the profile. per_task_format says whether a task may choose another table format than the warehouse's own.

auth_modes property 

auth_modes: frozenset[str]

The auth modes this warehouse accepts.

engine_for_jdbc_url 

engine_for_jdbc_url(jdbc_url: str) -> EngineSpec

Return the Engine DB a JDBC URL names; raises ConfigurationError for any other.

warehouse_spec 

warehouse_spec(sqlalchemy_name: str, table_format: str) -> WarehouseSpec

Choose the warehouse dialect for a connection's SQLAlchemy name and a table format.

Trino is Iceberg whichever format is asked for, because its catalog decides. PostgreSQL has no Iceberg tables, so asking for them is a ConfigurationError, as is a database with no dialect of its own.

warehouse_by_key 

warehouse_by_key(key: str) -> WarehouseSpec

Return the warehouse dialect registered under key, such as trino_iceberg.