Databricks SDK for JavaScript
    Preparing search index...

    Interface DiskSpec

    Describes the disks that are launched for each instance in the spark cluster. For example, if the cluster has 3 instances, each instance is configured to launch 2 disks, 100 GiB each, then will launch a total of 6 disks, 100 GiB each, for this cluster.

    interface DiskSpec {
        diskCount?: number;
        diskIops?: number;
        diskSize?: number;
        diskThroughput?: number;
        diskType?: DiskType;
    }
    Index

    Properties

    diskCount? diskIops? diskSize? diskThroughput? diskType?

    Properties

    diskCount?: number

    The number of disks launched for each instance:

    • This feature is only enabled for supported node types.
    • Users can choose up to the limit of the disks supported by the node type.
    • For node types with no OS disk, at least one disk must be specified; otherwise, cluster creation will fail.

    If disks are attached, will configure Spark to use only the disks for scratch storage, because heterogenously sized scratch devices can lead to inefficient disk utilization. If no disks are attached, will configure Spark to use instance store disks.

    Note: If disks are specified, then the Spark configuration spark.local.dir will be overridden.

    Disks will be mounted at:

    • For AWS: /ebs0, /ebs1, and etc.
    • For Azure: /remote_volume0, /remote_volume1, and etc.
    diskIops?: number
    diskSize?: number

    The size of each disk (in GiB) launched for each instance. Values must fall into the supported range for a particular instance type.

    For AWS:

    • General Purpose SSD: 100 - 4096 GiB
    • Throughput Optimized HDD: 500 - 4096 GiB

    For Azure:

    • Premium LRS (SSD): 1 - 1023 GiB
    • Standard LRS (HDD): 1- 1023 GiB
    diskThroughput?: number
    diskType?: DiskType

    The type of disks that will be launched with this cluster.

    Settings

    Member Visibility

    On This Page

    Properties