Databricks SDK for JavaScript
    Preparing search index...

    Interface AwsAttributes

    Attributes set during cluster creation which are related to Amazon Web Services.

    interface AwsAttributes {
        availability?: AwsAvailability;
        ebsVolumeCount?: number;
        ebsVolumeIops?: number;
        ebsVolumeSize?: number;
        ebsVolumeThroughput?: number;
        ebsVolumeType?: EbsVolumeType;
        firstOnDemand?: number;
        instanceProfileArn?: string;
        spotBidPricePercent?: number;
        zoneId?: string;
    }
    Index

    Properties

    availability? ebsVolumeCount? ebsVolumeIops? ebsVolumeSize? ebsVolumeThroughput? ebsVolumeType? firstOnDemand? instanceProfileArn? spotBidPricePercent? zoneId?

    Properties

    availability?: AwsAvailability
    ebsVolumeCount?: number

    The number of volumes launched for each instance. Users can choose up to 10 volumes. This feature is only enabled for supported node types. Legacy node types cannot specify custom EBS volumes. For node types with no instance store, at least one EBS volume needs to be specified; otherwise, cluster creation will fail.

    These EBS volumes will be mounted at /ebs0, /ebs1, and etc. Instance store volumes will be mounted at /local_disk0, /local_disk1, and etc.

    If EBS volumes are attached, will configure Spark to use only the EBS volumes for scratch storage because heterogenously sized scratch devices can lead to inefficient disk utilization. If no EBS volumes are attached, will configure Spark to use instance store volumes.

    Please note that if EBS volumes are specified, then the Spark configuration spark.local.dir will be overridden.

    ebsVolumeIops?: number

    If using gp3 volumes, what IOPS to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used.

    ebsVolumeSize?: number

    The size of each EBS volume (in GiB) launched for each instance. For general purpose SSD, this value must be within the range 100 - 4096. For throughput optimized HDD, this value must be within the range 500 - 4096.

    ebsVolumeThroughput?: number

    If using gp3 volumes, what throughput to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used.

    ebsVolumeType?: EbsVolumeType

    The type of EBS volumes that will be launched with this cluster.

    firstOnDemand?: number

    The first first_on_demand nodes of the cluster will be placed on on-demand instances. If this value is greater than 0, the cluster driver node in particular will be placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, first_on_demand nodes will be placed on on-demand instances and the remainder will be placed on availability instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster.

    instanceProfileArn?: string

    Nodes for this cluster will only be placed on AWS instances with this instance profile. If ommitted, nodes will be placed on instances without an IAM instance profile. The instance profile must have previously been added to the environment by an account administrator.

    This feature may only be available to certain customer plans.

    spotBidPricePercent?: number

    The bid price for AWS spot instances, as a percentage of the corresponding instance type's on-demand price. For example, if this field is set to 50, and the cluster needs a new r3.xlarge spot instance, then the bid price is half of the price of on-demand r3.xlarge instances. Similarly, if this field is set to 200, the bid price is twice the price of on-demand r3.xlarge instances. If not specified, the default value is 100. When spot instances are requested for this cluster, only spot instances whose bid price percentage matches this field will be considered. Note that, for safety, we enforce this field to be no more than 10000.

    zoneId?: string

    Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like "us-west-2a". The provided availability zone must be in the same region as the deployment. For example, "us-west-2a" is not a valid zone id if the deployment resides in the "us-east-1" region. This is an optional field at cluster creation, and if not specified, the zone "auto" will be used. If the zone specified is "auto", will try to place cluster in a zone with high availability, and will retry placement in a different AZ if there is not enough capacity.

    The list of available zones as well as the default value can be found by using the List Zones method.

    Settings

    Member Visibility

    On This Page

    Properties