Databricks SDK for JavaScript
    Preparing search index...

    Interface ModelServiceConfig_RoutingConfig

    Routing configuration for a model service, nesting destinations, routing strategy, and fallback under a single sub-message.

    interface ModelServiceConfig_RoutingConfig {
        destinations?: ModelServiceConfig_DestinationConfig[];
        fallback?: ModelServiceConfig_FallbackConfig;
        firstTokenTimeout?: Duration;
        routingStrategy?: {
            $case: "trafficSplitting";
            trafficSplitting: ModelServiceConfig_RoutingConfig_TrafficSplitting;
        };
    }
    Index

    Properties

    destinations? fallback? firstTokenTimeout? routingStrategy?

    Properties

    destinations?: ModelServiceConfig_DestinationConfig[]

    Primary routing destinations. At most 10 are allowed. At least one is required on CreateModelService; on UpdateModelService it is required only when config.routing (or a config.routing.* subpath) appears in update_mask.

    fallback?: ModelServiceConfig_FallbackConfig

    Fallback routing config, applied after primary destinations fail.

    firstTokenTimeout?: Duration

    Timeout for the first token of a streaming response. If a destination does not return its first token within this duration, AI Gateway aborts the attempt and fails over to the next destination. Applies to streaming requests only. Leave unset for no first-token timeout.

    routingStrategy?: {
        $case: "trafficSplitting";
        trafficSplitting: ModelServiceConfig_RoutingConfig_TrafficSplitting;
    }

    Selects how requests are distributed across destinations.

    Type Declaration

    • $case: "trafficSplitting"
    • trafficSplitting: ModelServiceConfig_RoutingConfig_TrafficSplitting

      Marker message selecting request-based traffic splitting. Traffic is distributed according to each destination's traffic_percentage value; no configuration lives on this message itself.

    Settings

    Member Visibility

    On This Page

    Properties