PluginsDatabricksTasksDbfsUpload

Upload​Upload

Upload Upload

yaml
type: "io.kestra.plugin.databricks.dbfs.Upload"

Upload a file to the Databricks File System.

The file can be of any size. The task will upload the file in chunks of 1MB.

Examples

Upload a file to the Databricks File System.

yaml
id: databricks_dbfs_upload
namespace: company.team

inputs:
  - id: file
    type: FILE
    description: File to be uploaded to DBFS

tasks:
  - id: upload_file
    type: io.kestra.plugin.databricks.dbfs.Upload
    authentication:
      token: <your-token>
    host: <your-host>
    from: "{{ inputs.file }}"
    to: /Share/myFile.txt

Properties

from

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The file to upload.

Must be a file from Kestra internal storage.

to

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The destination path.

accountId

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Databricks account identifier.

authentication

Databricks authentication configuration.

This property allows to configure the authentication to Databricks, different properties should be set depending on the type of authentication and the cloud provider. All configuration options can also be set using the standard Databricks environment variables. Check the Databricks authentication guide for more information.

configFile

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Databricks configuration file, use this if you don't want to configure each Databricks account properties one by one.

host

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Databricks host.

Outputs

Definitions

io.kestra.plugin.databricks.AbstractTask-AuthenticationConfig

Properties

authType
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
azureClientId
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
azureClientSecret
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
azureTenantId
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
clientId
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
clientSecret
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
googleCredentials
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
googleServiceAccount
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
password
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
token
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
username
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Metrics

file.size

  • Type: counter

Was this page helpful?