PluginsMinIOTasksDownloads

Downloads​Downloads

Downloads Downloads

yaml
type: "io.kestra.plugin.minio.Downloads"

Downloads multiple files from a bucket.

Examples

yaml
id: minio_downloads
namespace: company.team

tasks:
  - id: downloads
    type: io.kestra.plugin.minio.Downloads
    accessKeyId: "<access-key>"
    secretKeyId: "<secret-key>"
    region: "eu-central-1"
    bucket: "my-bucket"
    prefix: "sub-dir"
    action: "DELETE"

Download files from an S3-compatible storage — here, Spaces Object Storage from Digital Ocean.

yaml
id: s3_compatible_downloads
namespace: company.team

tasks:
  - id: downloads
    type: io.kestra.plugin.minio.Downloads
    accessKeyId: "<access-key>"
    secretKeyId: "<secret-key>"
    endpoint: https://<region>.digitaloceanspaces.com
    bucket: "kestra-test-bucket"
    prefix: "data/orders"
    action: "DELETE"

Properties

action

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️
  • Possible Values:
    • MOVE
    • DELETE
    • NONE

The action to perform on the retrieved files. If using 'NONE' make sure to handle the files inside your flow to avoid infinite triggering.

accessKeyId

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Access Key Id for authentication.

bucket

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The bucket name.

delimiter

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

A delimiter is a character you use to group keys.

endpoint

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

URL to the MinIO endpoint.

filter

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: BOTH
  • Possible Values:
    • FILES
    • DIRECTORY
    • BOTH

The type of objects to filter: files, directory, or both.

marker

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Marker is where you want to start listing from.

Start listing after this specified key. Marker can be any key in the bucket.

maxKeys

  • Type: integer
  • Dynamic: ✔️
  • Required: ❌
  • Default: 1000

Sets the maximum number of keys returned in the response.

By default, the action returns up to 1,000 key names. The response might contain fewer keys but will never contain more.

moveTo

The destination bucket and key for MOVE action.

prefix

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Limits the response to keys that begin with the specified prefix.

regexp

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

A regexp to filter on full key.

ex: regExp: .* to match all files regExp: .*2020-01-0.\\.csv to match files between 01 and 09 of january ending with .csv

region

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

MinIO region with which the SDK should communicate.

secretKeyId

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Secret Key Id for authentication.

Outputs

objects

The list of objects.

outputFiles

  • Type: object
  • SubType: string
  • Required: ❌

The downloaded files as a map of from/to URIs.

Definitions

io.kestra.plugin.minio.model.MinioObject

Properties

etag
  • Type: string
  • Dynamic: ❓
  • Required: ❓
key
  • Type: string
  • Dynamic: ❓
  • Required: ❓
lastModified
  • Type: string
  • Dynamic: ❓
  • Required: ❓
  • Format: date-time
owner
  • Type: Owner
  • Dynamic: ❓
  • Required: ❓
size
  • Type: integer
  • Dynamic: ❓
  • Required: ❓
uri
  • Type: string
  • Dynamic: ❓
  • Required: ❓
  • Format: uri

io.kestra.plugin.minio.Copy-CopyObject

Properties

bucket
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The bucket name

key
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The bucket key

io.kestra.plugin.minio.model.Owner

Properties

displayName
  • Type: string
  • Dynamic: ❓
  • Required: ❓
id
  • Type: string
  • Dynamic: ❓
  • Required: ❓

Was this page helpful?