databricks

etl_craft.dialects.warehouse.databricks ¶

Databricks warehouse, Delta tables.

DatabricksWarehouse ¶

Bases: WarehouseDialect

Databricks, Delta tables. A session has no default schema, so scratch tables are named.

identity_table_ddl ¶

identity_table_ddl(target: str, columns: str, params: Mapping[str, str]) -> str

Declare Delta's identity before any rows are inserted.

identity_replacement_ddl ¶

identity_replacement_ddl(conn: Connection, target: str, candidate: str, columns: str, params: Mapping[str, str], *, existing: bool) -> tuple[str, str]

Prepare independent identity rows, then deep-clone them into the target atomically.

existing_table_format ¶

existing_table_format(conn: Connection, target: str) -> TableFormat

Distinguish ordinary Delta from Delta exposing Iceberg metadata through UniForm.

row_id_generated ¶

row_id_generated(conn: Connection, target: str) -> bool

Keep older computed ROW_ID tables writable until explicitly replaced.

oauth_token ¶

oauth_token(profile: ConnectionProfile, secret: str, url: WarehouseUrl) -> str

Mint a workspace access token for a service principal (OAuth machine-to-machine).

The token URL defaults to the workspace's own endpoint and the scope to all-apis.

present ¶

present(profile: ConnectionProfile, secret: str, url: WarehouseUrl) -> Presented

Browser SSO is the connector's own flow; the other modes are a bearer token.

create_table_clause ¶

create_table_clause() -> str

Name Delta explicitly rather than relying on the workspace default.

delta_clause ¶

delta_clause(location: str = '') -> str

Return USING DELTA, with LOCATION for an external table, and table properties.

Databricks expects LOCATION before TBLPROPERTIES.

table_properties ¶

table_properties() -> str

Return the TBLPROPERTIES clause this table format needs; none for plain Delta.

create_table_as ¶

create_table_as(conn: Connection, qualified_name: str, select_sql: str, params: Mapping[str, str]) -> None

Run CREATE TABLE ... AS SELECT; an EXTERNAL_LOCATION parameter makes it external.

EXTERNAL_LOCATION is the table's own path in cloud storage (s3://, abfss://, gs://), which Unity Catalog must cover with an external location. Without it, the table is managed and stored where its catalog or schema says.

task_storage_problem ¶

task_storage_problem(params: Mapping[str, str]) -> str | None

Refuse an EXTERNAL_LOCATION containing a quote; it is written into the DDL.

mirror_table_ddl ¶

mirror_table_ddl(name: str, column_ddl: str, cloning: CloningConfig) -> str | None

Create a cloning mirror; it is external under the Cloning Base_location if set.

cloning_storage_problem ¶

cloning_storage_problem(cloning: CloningConfig) -> str | None

Refuse a Base_location containing a quote; it is written into the DDL.

audit_column_type ¶

audit_column_type(column: str) -> str

Use Databricks' TIMESTAMP, which is always UTC, for audit instants.

timestamp_text ¶

timestamp_text(value: str, kind: str) -> str

Convert the session's local representation of instants to a naive UTC value.

replacement_ddl ¶

replacement_ddl(conn: Connection, target: str, select_sql: str, params: Mapping[str, str], *, existing: bool) -> str

Preserve the existing table's declared properties in one CTAS replacement.

overwrite_statement ¶

overwrite_statement(target: str, columns: str, select_sql: str) -> str

Commit a complete Delta overwrite with one statement.