OptionalannotationsFree-form key-value annotations on the datasource, managed by the client.
OptionalcatalogName of the Unity Catalog catalog that contains the datasource. Required on create and
cannot be changed afterward. Reads return the value as supplied; the catalog's current name
is reported in effective_catalog_name.
OptionalcomputeCompute mode for the datasource's ingestion job. If unset, the default_compute_mode of
the catalog's datasource configuration applies, or DS_COMPUTE_PERFORMANCE_OPTIMIZED if
the catalog has none.
OptionalcomputeID of the workspace whose compute runs this datasource's ingestion job and owns its associated resources, such as the storage the ingested data lands in. Server-assigned.
OptionalcreatedOptionalcreateOptionaldatasourceUnity Catalog securable ID (a UUID) of the datasource. It never changes, including when the
datasource is renamed in Unity Catalog, and is the final segment of name.
OptionaldatasourceName of the datasource in Unity Catalog, unique within its schema. Required on create and
can be updated within the same catalog and schema. Get synchronizes this field with the
current Unity Catalog name, also reported in effective_datasource_name.
OptionaldescriptionFree-form description of the datasource.
OptionaleffectiveCurrent name of the Unity Catalog catalog that contains the datasource, which reflects
catalog renames without a change to catalog_name. Populated on get, create, and update
responses; list responses omit it.
OptionaleffectiveCurrent UC leaf name of the datasource (server-owned). Tracks UC renames so
callers can observe the live name without having to update the client-owned
datasource_name; unset until resolved.
OptionaleffectiveThe identity the datasource runs as: the resolved run_as_principal, or the requester's
identity if run_as_principal was omitted on create or cleared on update. Given as a user
email or service principal application ID, or as the numeric principal ID if the identity
can no longer be resolved. Populated on get, create, and update responses; list responses
omit it.
OptionaleffectiveCurrent name of the Unity Catalog schema that contains the datasource, which reflects
schema renames without a change to schema_name. Populated on get, create, and update
responses; list responses omit it.
OptionaletagOpaque entity tag for optimistic concurrency control. Returned by get, create, and update
responses and by list responses in the DATASOURCE_VIEW_FULL view. To make an update
conditional on the datasource not having changed since it was read, pass this value as the
update request's etag. Changes to status do not change the etag.
OptionalingestionHow the datasource ingests data. Exactly one ingestion method is required, and it cannot be changed after creation.
OptionalnameResource name of the datasource, in the format datasources/{datasource_id}. Unlike the
datasource's Unity Catalog location (catalog_name, schema_name, and datasource_name),
the name does not change when the datasource or its catalog or schema is renamed in Unity
Catalog.
OptionalownerUnity Catalog owner of the datasource, assigned by the server. Create responses omit it.
OptionalparsersParsers that schematize the ingested data. Each record is matched against the parsers in order and written to the parsed table of the first parser that it matches. Existing-table and Unity Catalog connection datasources require at least one parser; an Auto Loader datasource with no parsers writes only its raw table.
OptionalrunThe user or service principal that the datasource runs as, given as a user email or
service principal application ID. When it is supplied on create or update, the requester
must be authorized to run the datasource as that principal, or the request is rejected.
When it is omitted on create, the datasource runs as the requester. On update, a mask that
includes run_as_principal without a value clears it, so the datasource runs as the
requester; a mask that omits it keeps the current identity. Reads return the value as
supplied; the identity the datasource runs as is reported in effective_run_as_principal.
OptionalscheduleSchedule on which the datasource's ingestion job runs. A datasource cannot be paused, so a
request that sets schedule.enabled to false is rejected. Continuous schedules are not
supported for Unity Catalog connection datasources.
OptionalschemaName of the Unity Catalog schema that contains the datasource. Required on create and
cannot be changed afterward. Reads return the value as supplied; the schema's current name
is reported in effective_schema_name.
OptionalstatusRuntime status of the jobs and pipelines that run this datasource.
OptionalupdatedOptionalupdateOptionalversionVersion number of the datasource, incremented on each update. Informational only; use
etag for optimistic concurrency control.
OptionalwarehouseID of the SQL warehouse that runs the checks Lakewatch performs to validate a create or update request, such as source and destination access checks. Required and non-blank on every create and update request except a rename-only PATCH; there is no default warehouse. The requester must be able to use the warehouse. It is not stored, is not returned on reads, and does not affect the compute that runs the ingestion job.
A Lakewatch datasource: a pipeline that ingests security data and writes it to Unity Catalog tables.
What a datasource ingests is defined by its ingestion method (
autoloader,unity_catalog_connection, orexisting_table), which sets how the data arrives, together with itsparsers, which define the schemas that the arriving data is matched against and written as.