Interface ClusterLogConf

Type Declaration

• { $case: "dbfs"; dbfs: DbfsStorageInfo }

  • $case: "dbfs"

  • dbfs: DbfsStorageInfo

    destination needs to be provided. e.g. { "dbfs" : { "destination" : "dbfs:/home/cluster_log" } }

• { $case: "s3"; s3: S3StorageInfo }

  • $case: "s3"

  • s3: S3StorageInfo

    destination and either the region or endpoint need to be provided. e.g. { "s3": { "destination" : "s3://cluster_log_bucket/prefix", "region" : "us-west-2" } } Cluster iam role is used to access s3, please make sure the cluster iam role in instance_profile_arn has permission to write data to the s3 destination.

• { $case: "volumes"; volumes: VolumesStorageInfo }

  • $case: "volumes"

  • volumes: VolumesStorageInfo

    destination needs to be provided, e.g. { "volumes": { "destination": "/Volumes/catalog/schema/volume/cluster_log" } }