API docs by Redocly

Starburst Manager (v2)

Download OpenAPI specification:

About

APIs to manage your system

⬇️ Download OpenAPI specification

Roles

All APIs support the standard Trino role header of X-Trino-Role. Set a role for the API request by setting the header to system=ROLE{role-name}.

Uploads

The combined size of an uploaded set of text files, binary files, or secrets cannot exceed 10MB and each individual file or secret cannot exceed 1MB.

Manager Service

System management

resumeInstance

Resume an instance

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

batch

Shutdown and remove all batch jobs

Authorizations:
accessToken

Responses

Response samples

Content type
application/json
{
  • "reason": "string",
  • "metadata": {
    }
}

allInstanceLogs

Get all the logs of an instance as a single Zip file

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

userInstance

List instances available to the current user

Authorizations:
accessToken
query Parameters
showAll
boolean

If true, show terminated instances as well as active instances

Responses

Response samples

Content type
application/json
[
  • {
    }
]

connect

Shutdown and remove all connect servers

Authorizations:
accessToken

Responses

Response samples

Content type
application/json
{
  • "reason": "string",
  • "metadata": {
    }
}

shutdownInstance

Shutdown an instance

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

connectInstance

Start a new Spark Connect driver

Authorizations:
accessToken
query Parameters
doNotSaveConfiguration
boolean

Use this to disable saving the configuration for this instance. Normally, the configuration used to start instances is saved so that it can be examined and/or duplicated later. If for some reason you do not want this (due to sensitive information, for example), then add this query parameter.

Request Body schema: application/json
required
applicationName
string

The application name. If not provided a name will be generated.

object

Any Spark configuration/properties to set arguments

jars
Array of strings

Any jars to pass in the --jars Spark argument

pythonFiles
Array of strings

Any python files to pass in the --py-files Spark argument

files
Array of strings

Any files to pass in the --files Spark argument

archives
Array of strings

Any archives to pass in the --archives Spark argument

object

Any environment variables to set

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

secretUploads
Array of strings

Optional - secret uploads

Secrets will be set as environment variables in the Spark driver and executors.

Array of objects

Optional - file uploads (read only)

Array of objects

Optional - inline file uploads. See Uploads for more details and limits. (read only)

options
Array of strings
Items Value: "EncryptCommunication"

Details:

  • "EncryptCommunication": Enable encryption for communication between the driver and executors

Responses

Request samples

Content type
application/json
{
  • "applicationName": "string",
  • "sparkProperties": {
    },
  • "jars": [
    ],
  • "pythonFiles": [
    ],
  • "files": [
    ],
  • "archives": [
    ],
  • "environmentVariables": {
    },
  • "resourcePool": "string",
  • "secretUploads": [
    ],
  • "fileUploads": [
    ],
  • "inlineFileUploads": [
    ],
  • "options": [
    ]
}

Response samples

Content type
application/json
{
  • "instanceId": "string"
}

createPresignedUrl

Create pre-signed URLs for the given bucket, key and credentials

Authorizations:
accessToken
Request Body schema: application/json
required
accessKey
required
string

The proxy AccessKey provided by your administrator

secretKey
required
string

The proxy SecretKey provided by your administrator

region
required
string

The S3 region of the bucket

bucket
required
string

The bucket for creating the pre-signed URLs

key
required
string

The key for creating the pre-signed URLs

Responses

Request samples

Content type
application/json
{
  • "accessKey": "string",
  • "secretKey": "string",
  • "region": "string",
  • "bucket": "string",
  • "key": "string"
}

Response samples

Content type
application/json
{
  • "presignedUrls": {
    }
}

batchInstance

Submit and run a batch job

Authorizations:
accessToken
query Parameters
doNotSaveConfiguration
boolean

Use this to disable saving the configuration for this instance. Normally, the configuration used to start instances is saved so that it can be examined and/or duplicated later. If for some reason you do not want this (due to sensitive information, for example), then add this query parameter.

Request Body schema: application/json
required
applicationResource
required
string

The application resource to run - must be on an accessible object store

mainClass
string

The main class of the batch job

cronTimeZone
string

Optional CRON timezone. If provided, specify a time zone for the CRON job. Default is UTC.

See List of Time Zones.

cronSchedule
string

Optional CRON schedule. If provided, runs the job on the given schedule.

See Wikipedia's CRON article for details on CRON schedules.

ttlSecondsAfterFinished
integer <int32>

Optional.

ttlSecondsAfterFinished specifies the number of seconds after completion that the driver is eligible to be deleted/cleaned. Default is 1 seconds.

applicationArguments
Array of strings

Any application arguments

applicationName
string

The application name. If not provided a name will be generated.

object

Any Spark configuration/properties to set arguments

jars
Array of strings

Any jars to pass in the --jars Spark argument

pythonFiles
Array of strings

Any python files to pass in the --py-files Spark argument

files
Array of strings

Any files to pass in the --files Spark argument

archives
Array of strings

Any archives to pass in the --archives Spark argument

object

Any environment variables to set

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

secretUploads
Array of strings

Optional - secret uploads

Secrets will be set as environment variables in the Spark driver and executors.

Array of objects

Optional - file uploads (read only)

Array of objects

Optional - inline file uploads. See Uploads for more details and limits. (read only)

options
Array of strings
Items Value: "EncryptCommunication"

Details:

  • "EncryptCommunication": Enable encryption for communication between the driver and executors

Responses

Request samples

Content type
application/json
{
  • "applicationResource": "string",
  • "mainClass": "string",
  • "cronTimeZone": "string",
  • "cronSchedule": "string",
  • "ttlSecondsAfterFinished": 0,
  • "applicationArguments": [
    ],
  • "applicationName": "string",
  • "sparkProperties": {
    },
  • "jars": [
    ],
  • "pythonFiles": [
    ],
  • "files": [
    ],
  • "archives": [
    ],
  • "environmentVariables": {
    },
  • "resourcePool": "string",
  • "secretUploads": [
    ],
  • "fileUploads": [
    ],
  • "inlineFileUploads": [
    ],
  • "options": [
    ]
}

Response samples

Content type
application/json
{
  • "instanceId": "string"
}

restartInstance

Restart an instance

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

listEvent

List system events

Authorizations:
accessToken

Responses

Response samples

Content type
application/json
[
  • {
    }
]

duplicateInstance

Create/start a duplicate instance of the any instance

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)
query Parameters
doNotSaveConfiguration
boolean

Use this to disable saving the configuration for this instance. Normally, the configuration used to start instances is saved so that it can be examined and/or duplicated later. If for some reason you do not want this (due to sensitive information, for example), then add this query parameter.

Request Body schema: application/json
required
newName
string

The application name. If not provided a name will be generated.

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

Responses

Request samples

Content type
application/json
{
  • "newName": "string",
  • "resourcePool": "string"
}

Response samples

Content type
application/json
{
  • "instanceId": "string"
}

updateRole

Change the role associated with the token used in this API call

Authorizations:
accessToken
path Parameters
roleId
required
string

A role

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

getInstanceStatus

Get the status of an instance

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)
query Parameters
showAll
boolean

If true, show terminated instances as well as active instances

Responses

Response samples

Content type
application/json
{
  • "status": {
    }
}

getInstanceLogs

Get the log output of a batch job, a connect driver or a notebook server

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

notebookInstance

Start a new notebook server

Authorizations:
accessToken
query Parameters
doNotSaveConfiguration
boolean

Use this to disable saving the configuration for this instance. Normally, the configuration used to start instances is saved so that it can be examined and/or duplicated later. If for some reason you do not want this (due to sensitive information, for example), then add this query parameter.

Request Body schema: application/json
required
notebookName
string

The notebook name. If not provided a name will be generated.

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

description
string

Optional - the description of the notebook instance

Responses

Request samples

Content type
application/json
{
  • "notebookName": "string",
  • "resourcePool": "string",
  • "description": "string"
}

Response samples

Content type
application/json
{
  • "instanceId": "string"
}

getLogs

Get the system logs

Authorizations:
accessToken
path Parameters
logsId
required
string

Logs from an instance

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

listInstance

List all known instances

Authorizations:
accessToken
query Parameters
showAll
boolean

If true, show terminated instances as well as active instances

Responses

Response samples

Content type
application/json
[
  • {
    }
]

suspendInstance

Suspend an instance

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

getLogsWithIndex

Get the log output of an instance. logsId is the log index to return: Some cluster types have multiple nodes/workers. Pass 0 to get the main logs and then increase the index to get other logs.

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)
logsId
required
string

Logs from an instance

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

notebook

Shutdown and remove all notebook servers

Authorizations:
accessToken
query Parameters
preserveUserData
boolean

Preserve user data when deleting the notebook servers

Responses

Response samples

Content type
application/json
{
  • "reason": "string",
  • "metadata": {
    }
}

deleteInstance

Shutdown and remove an instance

Authorizations:
accessToken
path Parameters
instanceId
required
string
  • An instance in the system
  • This parameter can be looked up using name instead of its Id. Use name=value instead of an Id to lookup/search using the value. value must be encoded (see RFC including =)
query Parameters
preserveUserData
boolean

Preserve user data when deleting the notebook server

force
boolean

Use with caution! Immediately deletes instance without waiting for normal shutdown. This will likely result in loss of final logs, events, etc.

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

Resources Service

Resource pool management

getResourcePool

Return the current resource pool set including the total available memory and cores

Authorizations:
accessToken

Responses

Response samples

Content type
application/json
{
  • "totalMemory": "string",
  • "totalCores": "string",
  • "totalGpus": "string",
  • "resourcePools": [
    ]
}

updateResourcePool

Update the set of available resource pools (replace the entire set)

Authorizations:
accessToken
Request Body schema: application/json
required
required
Array of objects (ResourcePools)

The set of resource pools

Responses

Request samples

Content type
application/json
{
  • "resourcePools": [
    ]
}

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

createResourcePool

Create new resource pool

Authorizations:
accessToken
Request Body schema: application/json
required
resourcePoolId
required
string

The name of this resource pool (must be unique)

priority
integer <int32>

The priority of this pool. Pools with larger/higher priority numbers have priority over pools with smaller/lower priority numbers. If not specified, the priority is "0".

minMemory
required
string

Minimum memory (as a Spark quantity string)

minCores
required
string

Minimum virtual cores (as a CPU unit)

maxMemory
required
string

Maximum memory (as a Spark quantity string)

maxCores
required
string

Maximum virtual cores (as a CPU unit)

maxGpus
string

Maximum gpus (as a Spark quantity string)

applicationMaxGpus
string

Maximum gpus per application (as a Spark quantity string)

Responses

Request samples

Content type
application/json
{
  • "resourcePoolId": "string",
  • "priority": 0,
  • "minMemory": "string",
  • "minCores": "string",
  • "maxMemory": "string",
  • "maxCores": "string",
  • "maxGpus": "string",
  • "applicationMaxGpus": "string"
}

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

resources.updateResourcePool

Edit resource pool (update all parameters for a single resource pool)

Authorizations:
accessToken
path Parameters
resourcePoolId
required
string

A resource pool that describes a minimum and maximum resource set of memory and cores

Request Body schema: application/json
required
resourcePoolId
required
string

The name of this resource pool (must be unique)

priority
integer <int32>

The priority of this pool. Pools with larger/higher priority numbers have priority over pools with smaller/lower priority numbers. If not specified, the priority is "0".

minMemory
required
string

Minimum memory (as a Spark quantity string)

minCores
required
string

Minimum virtual cores (as a CPU unit)

maxMemory
required
string

Maximum memory (as a Spark quantity string)

maxCores
required
string

Maximum virtual cores (as a CPU unit)

maxGpus
string

Maximum gpus (as a Spark quantity string)

applicationMaxGpus
string

Maximum gpus per application (as a Spark quantity string)

Responses

Request samples

Content type
application/json
{
  • "resourcePoolId": "string",
  • "priority": 0,
  • "minMemory": "string",
  • "minCores": "string",
  • "maxMemory": "string",
  • "maxCores": "string",
  • "maxGpus": "string",
  • "applicationMaxGpus": "string"
}

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

deleteResourcePool

Delete resource pool

Authorizations:
accessToken
path Parameters
resourcePoolId
required
string

A resource pool that describes a minimum and maximum resource set of memory and cores

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

Uploads Service

File and secret upload management

getFileUpload

Get a file upload

Authorizations:
accessToken
path Parameters
uploadId
required
string

A text or binary file. See Uploads for more details and limits.

Responses

Response samples

Content type
application/json
{
  • "comment": "string",
  • "textData": {
    },
  • "binaryData": {
    }
}

updateFileUpload

Update a file upload

Authorizations:
accessToken
path Parameters
uploadId
required
string

A text or binary file. See Uploads for more details and limits.

Request Body schema: application/json
required
comment
required
string

Comment or description. Used only for your own reference purposes.

required
object

Map of name-to-text files/data

required
object

Map of name-to-binary files/data. Data must be Base64 encoded.

Responses

Request samples

Content type
application/json
{
  • "comment": "string",
  • "textData": {
    },
  • "binaryData": {
    }
}

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

deleteFileUpload

Delete a file upload

Authorizations:
accessToken
path Parameters
uploadId
required
string

A text or binary file. See Uploads for more details and limits.

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

listFileUploads

List file uploads

Authorizations:
accessToken

Responses

Response samples

Content type
application/json
[
  • {
    }
]

createFileUpload

Create a new file upload

Authorizations:
accessToken
Request Body schema: application/json
required
comment
required
string

Comment or description. Used only for your own reference purposes.

required
object

Map of name-to-text files/data

required
object

Map of name-to-binary files/data. Data must be Base64 encoded.

Responses

Request samples

Content type
application/json
{
  • "comment": "string",
  • "textData": {
    },
  • "binaryData": {
    }
}

Response samples

Content type
application/json
{
  • "uploadId": "string",
  • "comment": "string",
  • "textNames": [
    ],
  • "binaryNames": [
    ]
}

listSecretUploads

List file uploads

Authorizations:
accessToken

Responses

Response samples

Content type
application/json
[
  • {
    }
]

createSecretUpload

Create a new secret upload

Authorizations:
accessToken
Request Body schema: application/json
required
comment
required
string

Comment or description. Used only for your own reference purposes.

required
object

Map of name-to-binary secrets. Data must be Base64 encoded.

When the uploaded secret is used in an instance this map of secrets/values are set as environment variables. Thus, the secret name must be valid environment variable identifier.

See Uploads for more details and limits.

Responses

Request samples

Content type
application/json
{
  • "comment": "string",
  • "secrets": {
    }
}

Response samples

Content type
application/json
{
  • "uploadId": "string",
  • "comment": "string",
  • "secretNames": [
    ]
}

getSecretUpload

Get a secret upload

Authorizations:
accessToken
path Parameters
uploadId
required
string

A text or binary file. See Uploads for more details and limits.

Responses

Response samples

Content type
application/json
{
  • "comment": "string",
  • "secretNames": [
    ]
}

updateSecretUpload

Update a secret upload

Authorizations:
accessToken
path Parameters
uploadId
required
string

A text or binary file. See Uploads for more details and limits.

Request Body schema: application/json
required
comment
required
string

Comment or description. Used only for your own reference purposes.

required
object

Map of name-to-binary secrets. Data must be Base64 encoded.

When the uploaded secret is used in an instance this map of secrets/values are set as environment variables. Thus, the secret name must be valid environment variable identifier.

See Uploads for more details and limits.

Responses

Request samples

Content type
application/json
{
  • "comment": "string",
  • "secrets": {
    }
}

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

deleteSecretUpload

Delete a secret upload

Authorizations:
accessToken
path Parameters
uploadId
required
string

A text or binary file. See Uploads for more details and limits.

Responses

Response samples

Content type
application/json
{
  • "fieldViolations": [
    ]
}

Model Definitions

ResourcePoolsInfo

totalMemory
required
string

Total available memory (as a Spark quantity string)

totalCores
required
string

Total available virtual cores

totalGpus
required
string

Total available gpus (as quantity string)

required
Array of objects

The set of resource pools

{
  • "totalMemory": "string",
  • "totalCores": "string",
  • "totalGpus": "string",
  • "resourcePools": [
    ]
}

UploadInfo

uploadId
required
string

The ID of this upload

comment
required
string

Comment or description. Used only for your own reference purposes.

textNames
required
Array of strings

Names of text data in the upload

binaryNames
required
Array of strings

Names of binary data in the upload

{
  • "uploadId": "string",
  • "comment": "string",
  • "textNames": [
    ],
  • "binaryNames": [
    ]
}

SecretUploadInfo

uploadId
required
string

The ID of this upload

comment
required
string

Comment or description. Used only for your own reference purposes.

secretNames
required
Array of strings

Secret names

{
  • "uploadId": "string",
  • "comment": "string",
  • "secretNames": [
    ]
}

InstanceInfo

instanceId
required
string

The instance Id

name
required
string

The instance name

type
required
string

The instance type

createdBy
required
string

User that created the instance

required
object

Any additional details about the instance

{
  • "instanceId": "string",
  • "name": "string",
  • "type": "string",
  • "createdBy": "string",
  • "details": {
    }
}

SparkBatchJob

applicationResource
required
string

The application resource to run - must be on an accessible object store

mainClass
string

The main class of the batch job

cronTimeZone
string

Optional CRON timezone. If provided, specify a time zone for the CRON job. Default is UTC.

See List of Time Zones.

cronSchedule
string

Optional CRON schedule. If provided, runs the job on the given schedule.

See Wikipedia's CRON article for details on CRON schedules.

ttlSecondsAfterFinished
integer <int32>

Optional.

ttlSecondsAfterFinished specifies the number of seconds after completion that the driver is eligible to be deleted/cleaned. Default is 1 seconds.

applicationArguments
Array of strings

Any application arguments

applicationName
string

The application name. If not provided a name will be generated.

object

Any Spark configuration/properties to set arguments

jars
Array of strings

Any jars to pass in the --jars Spark argument

pythonFiles
Array of strings

Any python files to pass in the --py-files Spark argument

files
Array of strings

Any files to pass in the --files Spark argument

archives
Array of strings

Any archives to pass in the --archives Spark argument

object

Any environment variables to set

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

secretUploads
Array of strings

Optional - secret uploads

Secrets will be set as environment variables in the Spark driver and executors.

Array of objects

Optional - file uploads (read only)

Array of objects

Optional - inline file uploads. See Uploads for more details and limits. (read only)

options
Array of strings
Items Value: "EncryptCommunication"

Details:

  • "EncryptCommunication": Enable encryption for communication between the driver and executors
{
  • "applicationResource": "string",
  • "mainClass": "string",
  • "cronTimeZone": "string",
  • "cronSchedule": "string",
  • "ttlSecondsAfterFinished": 0,
  • "applicationArguments": [
    ],
  • "applicationName": "string",
  • "sparkProperties": {
    },
  • "jars": [
    ],
  • "pythonFiles": [
    ],
  • "files": [
    ],
  • "archives": [
    ],
  • "environmentVariables": {
    },
  • "resourcePool": "string",
  • "secretUploads": [
    ],
  • "fileUploads": [
    ],
  • "inlineFileUploads": [
    ],
  • "options": [
    ]
}

Upload

comment
required
string

Comment or description. Used only for your own reference purposes.

required
object

Map of name-to-text files/data

required
object

Map of name-to-binary files/data. Data must be Base64 encoded.

{
  • "comment": "string",
  • "textData": {
    },
  • "binaryData": {
    }
}

Instance

instanceId
required
string

The instance Id

{
  • "instanceId": "string"
}

ResourcePoolSet

required
Array of objects (ResourcePools)

The set of resource pools

{
  • "resourcePools": [
    ]
}

SecretUpload

comment
required
string

Comment or description. Used only for your own reference purposes.

required
object

Map of name-to-binary secrets. Data must be Base64 encoded.

When the uploaded secret is used in an instance this map of secrets/values are set as environment variables. Thus, the secret name must be valid environment variable identifier.

See Uploads for more details and limits.

{
  • "comment": "string",
  • "secrets": {
    }
}

RedactedSecretUpload

comment
required
string

Comment or description. Used only for your own reference purposes.

secretNames
required
Array of strings

Secret names

{
  • "comment": "string",
  • "secretNames": [
    ]
}

StatusV2

required
object

The status

{
  • "status": {
    }
}

NotebookInstance

notebookName
string

The notebook name. If not provided a name will be generated.

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

description
string

Optional - the description of the notebook instance

{
  • "notebookName": "string",
  • "resourcePool": "string",
  • "description": "string"
}

SparkConnectDriver

applicationName
string

The application name. If not provided a name will be generated.

object

Any Spark configuration/properties to set arguments

jars
Array of strings

Any jars to pass in the --jars Spark argument

pythonFiles
Array of strings

Any python files to pass in the --py-files Spark argument

files
Array of strings

Any files to pass in the --files Spark argument

archives
Array of strings

Any archives to pass in the --archives Spark argument

object

Any environment variables to set

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

secretUploads
Array of strings

Optional - secret uploads

Secrets will be set as environment variables in the Spark driver and executors.

Array of objects

Optional - file uploads (read only)

Array of objects

Optional - inline file uploads. See Uploads for more details and limits. (read only)

options
Array of strings
Items Value: "EncryptCommunication"

Details:

  • "EncryptCommunication": Enable encryption for communication between the driver and executors
{
  • "applicationName": "string",
  • "sparkProperties": {
    },
  • "jars": [
    ],
  • "pythonFiles": [
    ],
  • "files": [
    ],
  • "archives": [
    ],
  • "environmentVariables": {
    },
  • "resourcePool": "string",
  • "secretUploads": [
    ],
  • "fileUploads": [
    ],
  • "inlineFileUploads": [
    ],
  • "options": [
    ]
}

EventV2

time
required
string

Time of the event

type
required
string

Event type

reason
required
string

Event reason

name
required
string

Event name

action
required
string

Event action

{
  • "time": "string",
  • "type": "string",
  • "reason": "string",
  • "name": "string",
  • "action": "string"
}

ResourcePools

resourcePoolId
required
string

The name of this resource pool (must be unique)

priority
integer <int32>

The priority of this pool. Pools with larger/higher priority numbers have priority over pools with smaller/lower priority numbers. If not specified, the priority is "0".

minMemory
required
string

Minimum memory (as a Spark quantity string)

minCores
required
string

Minimum virtual cores (as a CPU unit)

maxMemory
required
string

Maximum memory (as a Spark quantity string)

maxCores
required
string

Maximum virtual cores (as a CPU unit)

maxGpus
string

Maximum gpus (as a Spark quantity string)

applicationMaxGpus
string

Maximum gpus per application (as a Spark quantity string)

{
  • "resourcePoolId": "string",
  • "priority": 0,
  • "minMemory": "string",
  • "minCores": "string",
  • "maxMemory": "string",
  • "maxCores": "string",
  • "maxGpus": "string",
  • "applicationMaxGpus": "string"
}

PresignedUrlRequest

accessKey
required
string

The proxy AccessKey provided by your administrator

secretKey
required
string

The proxy SecretKey provided by your administrator

region
required
string

The S3 region of the bucket

bucket
required
string

The bucket for creating the pre-signed URLs

key
required
string

The key for creating the pre-signed URLs

{
  • "accessKey": "string",
  • "secretKey": "string",
  • "region": "string",
  • "bucket": "string",
  • "key": "string"
}

DuplicateInstance

newName
string

The application name. If not provided a name will be generated.

resourcePool
string

Optional - the resource pool to use (you must have permission to use it)

{
  • "newName": "string",
  • "resourcePool": "string"
}

PresignedUrl

required
object

The pre-signed URLs. The key is an HTTP verb (GET, PUT, POST, DELETE). The value is the pre-signed URL

{
  • "presignedUrls": {
    }
}

Responses

Resourceinfo

resourceName
required
string

Name of the resource (read only)

description
required
string

Violation description (read only)

{
  • "resourceName": "string",
  • "description": "string"
}

Quotafailure

required
Array of objects (FieldViolations)

Field violations (read only)

{
  • "fieldViolations": [
    ]
}

Badrequest

required
Array of objects (FieldViolations)

Field violations (read only)

{
  • "fieldViolations": [
    ]
}

Errorinfo

reason
required
string

Error reason/detail (read only)

required
object

Any additional details (read only)

{
  • "reason": "string",
  • "metadata": {
    }
}

FieldViolations

field
required
string

Field name

description
required
string

Description of the violation

{
  • "field": "string",
  • "description": "string"
}

Empty

type
required
string

Always the value "empty" (read only)

{
  • "type": "string"
}