Synced Database Tables

Package: databricks.bundles.synced_database_tables

Classes

class Lifecycle
prevent_destroy: bool | None = None

Lifecycle setting to prevent the resource from being destroyed.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class NewPipelineSpec

Custom fields that user can set for pipeline while creating SyncedDatabaseTable. Note that other fields of pipeline are still inferred by table def internally

storage_catalog: str | None = None

[Public Preview] This field needs to be specified if the destination catalog is a managed postgres catalog.

UC catalog for the pipeline to store intermediate files (checkpoints, event logs etc). This needs to be a standard catalog where the user has permissions to create Delta tables.

storage_schema: str | None = None

[Public Preview] This field needs to be specified if the destination catalog is a managed postgres catalog.

UC schema for the pipeline to store intermediate files (checkpoints, event logs etc). This needs to be in the standard catalog where the user has permissions to create Delta tables.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class SyncedDatabaseTable
name: str

[Public Preview] Full three-part (catalog, schema, table) name of the table.

database_instance_name: str | None = None

[Public Preview] Name of the target database instance. This is required when creating synced database tables in standard catalogs. This is optional when creating synced database tables in registered catalogs. If this field is specified when creating synced database tables in registered catalogs, the database instance name MUST match that of the registered catalog (or the request will be rejected).

lifecycle: Lifecycle | None = None

Settings that control the deployment lifecycle of the resource, such as preventing it from being destroyed.

logical_database_name: str | None = None

[Public Preview] Target Postgres database object (logical database) name for this table.

When creating a synced table in a registered Postgres catalog, the target Postgres database name is inferred to be that of the registered catalog. If this field is specified in this scenario, the Postgres database name MUST match that of the registered catalog (or the request will be rejected).

When creating a synced table in a standard catalog, this field is required. In this scenario, specifying this field will allow targeting an arbitrary postgres database. Note that this has implications for the create_database_objects_is_missing field in spec.

spec: SyncedTableSpec | None = None

[Public Preview] Specification of a synced database table.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class SyncedTableSchedulingPolicy
CONTINUOUS = 'CONTINUOUS'
TRIGGERED = 'TRIGGERED'
SNAPSHOT = 'SNAPSHOT'
class SyncedTableSpec

Specification of a synced database table.

create_database_objects_if_missing: bool | None = None

[Public Preview] If true, the synced table’s logical database and schema resources in PG will be created if they do not already exist.

existing_pipeline_id: str | None = None

[Public Preview] At most one of existing_pipeline_id and new_pipeline_spec should be defined.

If existing_pipeline_id is defined, the synced table will be bin packed into the existing pipeline referenced. This avoids creating a new pipeline and allows sharing existing compute. In this case, the scheduling_policy of this synced table must match the scheduling policy of the existing pipeline.

new_pipeline_spec: NewPipelineSpec | None = None

[Public Preview] At most one of existing_pipeline_id and new_pipeline_spec should be defined.

If new_pipeline_spec is defined, a new pipeline is created for this synced table. The location pointed to is used to store intermediate files (checkpoints, event logs etc). The caller must have write permissions to create Delta tables in the specified catalog and schema. Again, note this requires write permissions, whereas the source table only requires read permissions.

primary_key_columns: list[str]

[Public Preview] Primary Key columns to be used for data insert/update in the destination.

scheduling_policy: SyncedTableSchedulingPolicy | None = None

[Public Preview] Scheduling policy of the underlying pipeline.

source_table_full_name: str | None = None

[Public Preview] Three-part (catalog, schema, table) name of the source Delta table.

timeseries_key: str | None = None

[Public Preview] Time series key to deduplicate (tie-break) rows with the same primary key.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict