Interface InitScriptInfo

Type Declaration

• { $case: "dbfs"; dbfs: DbfsStorageInfo }

  • $case: "dbfs"

  • dbfs: DbfsStorageInfo

    destination needs to be provided. e.g. { "dbfs": { "destination" : "dbfs:/home/cluster_log" } }

• { $case: "s3"; s3: S3StorageInfo }

  • $case: "s3"

  • s3: S3StorageInfo

    destination and either the region or endpoint need to be provided. e.g. { \"s3\": { \"destination\": \"s3://cluster_log_bucket/prefix\", \"region\": \"us-west-2\" } } Cluster iam role is used to access s3, please make sure the cluster iam role in instance_profile_arn has permission to write data to the s3 destination.

• { $case: "file"; file: LocalFileInfo }

  • $case: "file"

  • file: LocalFileInfo

    destination needs to be provided, e.g. { "file": { "destination": "file:/my/local/file.sh" } }

• { $case: "gcs"; gcs: GcsStorageInfo }

  • $case: "gcs"

  • gcs: GcsStorageInfo

    destination needs to be provided, e.g. { "gcs": { "destination": "gs://my-bucket/file.sh" } }

• { $case: "abfss"; abfss: Adlsgen2Info }

  • $case: "abfss"

  • abfss: Adlsgen2Info

    destination needs to be provided, e.g. abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/<directory-name>

• { $case: "workspace"; workspace: WorkspaceStorageInfo }

  • $case: "workspace"

  • workspace: WorkspaceStorageInfo

    destination needs to be provided, e.g. { "workspace": { "destination": "/cluster-init-scripts/setup-datadog.sh" } }

• { $case: "volumes"; volumes: VolumesStorageInfo }

  • $case: "volumes"

  • volumes: VolumesStorageInfo

    destination needs to be provided. e.g. { \"volumes\" : { \"destination\" : \"/Volumes/my-init.sh\" } }