Features

Package: databricks.bundles.features

Classes

class AggregationFunction

An aggregation function applied over a time window.

approx_count_distinct: ApproxCountDistinctFunction | None = None

Computes the approximate count of distinct values.

approx_percentile: ApproxPercentileFunction | None = None

Computes the approximate percentile of values.

avg: AvgFunction | None = None

Computes the average of values.

count_function: CountFunction | None = None

Computes the count of values.

first: FirstFunction | None = None

Returns the first value.

last: LastFunction | None = None

Returns the last value.

max: MaxFunction | None = None

Computes the maximum value.

min: MinFunction | None = None

Computes the minimum value.

stddev_pop: StddevPopFunction | None = None

Computes the population standard deviation.

stddev_samp: StddevSampFunction | None = None

Computes the sample standard deviation.

sum: SumFunction | None = None

Computes the sum of values.

time_window: TimeWindow | None = None

The time window over which the aggregation is computed.

var_pop: VarPopFunction | None = None

Computes the population variance.

var_samp: VarSampFunction | None = None

Computes the sample variance.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class ApproxCountDistinctFunction

Computes the approximate count of distinct values.

input: str

The input column from which the approximate count of distinct values is computed.

relative_sd: float | None = None

The maximum relative standard deviation allowed (default defined by Spark). Deprecated: use relative_error. When relative_error is set, reads return the same value here for clients that don’t have relative_error, and a request may only send that value.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class ApproxPercentileFunction

Computes the approximate percentile of values.

input: str

The input column from which the approximate percentile is computed.

percentile: float

The percentile value to compute (between 0 and 1).

accuracy: int | None = None

The accuracy parameter (higher is more accurate but slower). Deprecated: use relative_error. When relative_error is set, reads return ceil(1 / relative_error) here for clients that don’t have relative_error, and a request may only send that value.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class AvgFunction

Computes the average of values.

input: str

The input column from which the average is computed. For Kafka sources, use dot-prefixed path notation (e.g., “value.amount”). For nested fields, the leaf node name is used. Colon-prefixed notation (e.g., “value:amount”) is supported for backwards compatibility but is deprecated; migrate to dot notation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class ColumnSelection

A ColumnSelection function, equivalent to the LAST() record of an entity over a lifetime window

column: str

Column name from source to select as the feature value.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class CountFunction

Computes the count of values.

input: str

The input column from which the count is computed. For Kafka sources, use dot-prefixed path notation (e.g., “value.amount”). For nested fields, the leaf node name is used. Colon-prefixed notation (e.g., “value:amount”) is supported for backwards compatibility but is deprecated; migrate to dot notation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class DataSource

Specifies the data source backing a feature. Exactly one source type must be set.

delta_table_source: DeltaTableSource | None = None

A Delta table data source.

kafka_source: KafkaSource | None = None

A Kafka stream data source.

request_source: RequestSource | None = None

A request-time data source.

stream_source: StreamSource | None = None

A Stream data source.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class DeltaTableSource
full_name: str

The full three-part (catalog, schema, table) name of the Delta table.

dataframe_schema: str | None = None

Schema of the resulting dataframe after transformations, in Spark StructType JSON format (from df.schema.json()). Required if transformation_sql is specified. Example: {“type”:”struct”,”fields”:[{“name”:”col_a”,”type”:”integer”,”nullable”:true,”metadata”:{}},{“name”:”col_c”,”type”:”integer”,”nullable”:true,”metadata”:{}}]}

filter_condition: str | None = None

Single WHERE clause to filter delta table before applying transformations. Will be row-wise evaluated, so should only include conditionals and projections.

transformation_sql: str | None = None

A single SQL SELECT expression applied after filter_condition. Should contains all the columns needed (eg. “SELECT , col_a + col_b AS col_c FROM x.y.z WHERE col_a > 0” would have `transformation_sql` “, col_a + col_b AS col_c”) If transformation_sql is not provided, all columns of the delta table are present in the DataSource dataframe.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class DependsOn
resource: str

Key of the resource to depend on, as foo or jobs.foo.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class EntityColumn
name: str

The name of the entity column. For Kafka sources, use dot-prefixed path notation to reference fields within the key or value schema (e.g., “value.user_id”, “key.partition_key”). For nested fields, the leaf node name (e.g., “user_id” from “value.trip_details.user_id”) is what will be present in materialized tables and expected to match at query time. Colon-prefixed notation (e.g., “value:user_id”) is supported for backwards compatibility but is deprecated; migrate to dot notation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class Feature
full_name: str

The full three-part name (catalog, schema, name) of the feature.

function: Function

The function by which the feature is computed.

source: DataSource

The data source of the feature.

description: str | None = None

The description of the feature.

entities: list[EntityColumn]

The entity columns for the feature, used as aggregation keys and for query-time lookup. Optional since entities are not set for RequestSource features or on-demand calculated features.

lifecycle: Lifecycle | None = None
timeseries_column: TimeseriesColumn | None = None

Column recording time, used for point-in-time joins, backfills, and aggregations. Optional since a timeseries column is not set for RequestSource features or on-demand calculated features.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class FieldDefinition

A single field definition within a FlatSchema, specifying the field name and its scalar data type. Does not support nested or complex types (arrays, maps, structs).

data_type: ScalarDataType

The scalar data type of the field.

name: str

The name of the field.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class FirstFunction

Returns the first value.

input: str

The input column from which the first value is returned.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class FlatSchema

A flat (non-nested) schema for request-time fields, defined as an ordered list of field definitions. This schema only supports scalar types.

fields: list[FieldDefinition]

The list of fields in this schema.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class Function
aggregation_function: AggregationFunction | None = None

An aggregation function applied over a time window.

column_selection: ColumnSelection | None = None

Selects the latest value of a single column in a data source

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class KafkaSource
name: str

Name of the Kafka source, used to identify it. This is used to look up the corresponding KafkaConfig object. Can be distinct from topic name.

filter_condition: str | None = None

The filter condition applied to the source data before aggregation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class LastFunction

Returns the last value.

input: str

The input column from which the last value is returned.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class Lifecycle
depends_on: list[DependsOn]

Resources that must be deployed before this resource and deleted after it. Unlike variable references, this only affects ordering.

prevent_destroy: bool | None = None

Lifecycle setting to prevent the resource from being destroyed.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class MaxFunction

Computes the maximum value.

input: str

The input column from which the maximum is computed.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class MinFunction

Computes the minimum value.

input: str

The input column from which the minimum is computed.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class RequestSource

A request-time data source whose value is provided at inference time: offline batch scoring or online serving endpoint

flat_schema: FlatSchema | None = None

A flat schema with scalar-typed fields only.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class RollingWindow

A rolling time window with an optional non-negative delay.

delay: str | None = None

Non-negative analytic lag that evaluates the window this far in the past. Use this for timing variations unrelated to source lateness, such as a 30-day count as of one week ago. If unset, the analytic lag is zero. It composes with source.lateness when both are set.

window_duration: str | None = None

The duration of the rolling window. Must be positive when set; absent means lifetime (aggregate over the entity’s entire history).

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class ScalarDataType

Scalar data types for request-time field definitions. Only flat (non-nested) types are supported.

INTEGER = 'INTEGER'
FLOAT = 'FLOAT'
BOOLEAN = 'BOOLEAN'
STRING = 'STRING'
DOUBLE = 'DOUBLE'
LONG = 'LONG'
TIMESTAMP = 'TIMESTAMP'
DATE = 'DATE'
SHORT = 'SHORT'
BINARY = 'BINARY'
DECIMAL = 'DECIMAL'
class SlidingWindow
slide_duration: str

The slide duration (interval by which windows advance, must be positive and less than duration).

window_duration: str | None = None

The duration of the sliding window. Must be positive when set; absent means lifetime (aggregate over the entity’s entire history).

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class StddevPopFunction

Computes the population standard deviation.

input: str

The input column from which the population standard deviation is computed. For Kafka sources, use dot-prefixed path notation (e.g., “value.amount”). For nested fields, the leaf node name is used. Colon-prefixed notation (e.g., “value:amount”) is supported for backwards compatibility but is deprecated; migrate to dot notation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class StddevSampFunction

Computes the sample standard deviation.

input: str

The input column from which the sample standard deviation is computed.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class StreamSource

A Stream entity used as a data source for a feature.

full_name: str

Three-part full name of the Stream (catalog.schema.stream).

filter_condition: str | None = None

The filter condition applied to the source data before aggregation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class SumFunction

Computes the sum of values.

input: str

The input column from which the sum is computed. For Kafka sources, use dot-prefixed path notation (e.g., “value.amount”). For nested fields, the leaf node name is used. Colon-prefixed notation (e.g., “value:amount”) is supported for backwards compatibility but is deprecated; migrate to dot notation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class TimeWindow
rolling: RollingWindow | None = None

A rolling time window with an optional non-negative delay.

sliding: SlidingWindow | None = None
tumbling: TumblingWindow | None = None
classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class TimeseriesColumn
name: str

The name of the timeseries column. For Kafka sources, use dot-prefixed path notation to reference fields within the key or value schema (e.g., “value.event_timestamp”). For nested fields, the leaf node name (e.g., “event_timestamp” from “value.event_details.event_timestamp”) is what will be present in materialized tables and expected to match at query time. Colon-prefixed notation (e.g., “value:event_timestamp”) is supported for backwards compatibility but is deprecated; migrate to dot notation.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class TumblingWindow
window_duration: str

The duration of each tumbling window (non-overlapping, fixed-duration windows).

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class VarPopFunction

Computes the population variance.

input: str

The input column from which the population variance is computed.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict
class VarSampFunction

Computes the sample variance.

input: str

The input column from which the sample variance is computed.

classmethod from_dict(
value: dict,
) → Self
as_dict(
self,
) → dict