# Plugins setup
> This bundle contains all pages in the Plugins setup section.
> Source: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins ===

# Plugins

This section includes the steps to configure integrations for the Flyte platform.

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/kubernetes-plugins ===

# Kubernetes Plugins

This guide will help you configure the Flyte plugins that provision resources on Kubernetes.
The steps are defined in terms of the Helm chart you used to install Flyte.

## Install the Kubernetes operator

Select the integration you need and follow the steps to install the corresponding Kubernetes operator:

<details>
<summary>PyTorch/TensorFlow/MPI</summary>

1. Install the [Kubeflow training-operator](https://github.com/kubeflow/training-operator?tab=readme-ov-file#stable-release) (stable release):

```bash

  kubectl apply -k "github.com/kubeflow/training-operator/manifests/overlays/standalone?ref=v1.7.0"
```
**Optional: Using a gang scheduler**

To address potential issues with worker pods of distributed training jobs being scheduled at different times due to resource constraints, you can opt for a gang scheduler. This ensures that all worker pods are scheduled simultaneously, reducing the likelihood of job failures caused by timeout errors.

To enable gang scheduling for the ``training-operator``:

a. Select a second scheduler from
[Kubernetes scheduler plugins with co-scheduling](https://www.kubeflow.org/docs/components/training/user-guides/job-scheduling/#running-jobs-with-gang-scheduling)
or [Apache YuniKorn](https://yunikorn.apache.org/docs/next/user_guide/workloads/run_tf/).

b. Configure a Flyte ``PodTemplate`` to use the gang scheduler for your Tasks:

**K8s scheduler plugins with co-scheduling**

```yaml

template:
  spec:
    schedulerName: "scheduler-plugins-scheduler"
```
**Apache Yunikorn**

```yaml

template:
  metadata:
    annotations:
      yunikorn.apache.org/task-group-name: ""
      yunikorn.apache.org/task-groups: ""
      yunikorn.apache.org/schedulingPolicyParameters: ""
```
You can set the scheduler name in the Pod template passed to the ``@task`` decorator. However, to prevent the two different schedulers from competing for resources, we recommend setting the scheduler name in the pod template in the ``flyte`` namespace which is applied to all tasks. Non distributed training tasks can be scheduled by the
gang scheduler as well.

</details>

<details>
<summary>Ray</summary>

To add the Kuberay Helm repo, run the following command:

```bash

helm repo add kuberay https://ray-project.github.io/kuberay-helm/
```
To install the Kuberay operator, run the following command:

```bash

helm install kuberay-operator kuberay/kuberay-operator --namespace ray-system --version 1.1.0 --create-namespace
```

</details>

<details>
<summary>Spark</summary>

To add the Spark Helm repository, run the following commands:

```bash

helm repo add spark-operator https://kubeflow.github.io/spark-operator
```
    To install the Spark operator, run the following command:

```bash

helm install spark-operator spark-operator/spark-operator --namespace spark-operator --create-namespace
```

</details>

<details>
<summary>Dask</summary>

To add the Dask Helm repository, run the following command:

```bash

helm repo add dask https://helm.dask.org
```
To install the Dask operator, run the following command:

```bash

helm install dask-operator dask/dask-kubernetes-operator --namespace dask-operator --create-namespace
```

</details>

<details>
<summary>Volcano-Scheduled PodTask</summary>

PodTasks in Flyte can be scheduled using the Volcano scheduler, which offers queue-based features such as multi-tenant resource management, task prioritization, and resource preemption. These features enhance scheduling efficiency and improve overall cluster utilization in shared environments.

To enable Volcano-scheduled PodTasks in Flyte, follow these steps:

1. Install Volcano in your cluster by following the [instructions in the volcano repository](https://github.com/volcano-sh/volcano).

2. Configure a Flyte PodTemplate to use the Volcano scheduler for your PodTasks:

```yaml

template:
  spec:
    schedulerName: "volcano"
```

</details>

## Specify plugin configuration

<details>
<summary>Pytorch</summary>

### flyte-binary

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configuration:
  inline:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - pytorch
        default-for-task-types:
          - container: container
          - container_array: k8s-array
          - pytorch: pytorch
```
### flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml
configmap:
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - pytorch
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          pytorch: pytorch
```

</details>

<details>
<summary>Tensorflow</summary>

### flyte-binary

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configuration:
  inline:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - tensorflow
        default-for-task-types:
          - container: container
          - container_array: k8s-array
          - tensorflow: tensorflow
```
### flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configmap:
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - tensorflow
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          tensorflow: tensorflow
```

</details>

<details>
<summary>MPI</summary>

### flyte-binary

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configuration:
  inline:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - mpi
        default-for-task-types:
          - container: container
          - container_array: k8s-array
          - mpi: mpi
```
#### flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configmap:
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - mpi
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          mpi: mpi
```

</details>

<details>
<summary>Ray</summary>

### flyte-binary

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configuration:
  inline:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - ray
        default-for-task-types:
          - container: container
          - container_array: k8s-array
          - ray: ray
    plugins:
      ray:
      // Shutdown Ray cluster after 1 hour of inactivity
        ttlSecondsAfterFinished: 3600
```
### flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configmap:
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - ray
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          ray: ray
    plugins:
      ray:
        // Shutdown Ray cluster after 1 hour of inactivity
        ttlSecondsAfterFinished: 3600
```

</details>

<details>
<summary>Spark</summary>

## flyte-binary on AWS

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configuration:
  inline:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - spark
        default-for-task-types:
          - container: container
          - container_array: k8s-array
          - spark: spark
    cluster_resources:
      - production:
        - defaultIamRole:
            value: <FLYTE_IAM_USER_ARN>
      - staging:
        - defaultIamRole:
            value: <FLYTE_IAM_USER_ARN>
      - development:
        - defaultIamRole:
            value: <FLYTE_IAM_USER_ARN>
    plugins:
      spark:
      # Edit the Spark configuration as you see fit
        spark-config-default:
          - spark.driver.cores: "1"
          - spark.hadoop.fs.s3a.aws.credentials.provider: "com.amazonaws.auth.DefaultAWSCredentialsProviderChain"
          - spark.kubernetes.allocation.batch.size: "50"
          - spark.hadoop.fs.s3a.acl.default: "BucketOwnerFullControl"
          - spark.hadoop.fs.s3n.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
          - spark.hadoop.fs.AbstractFileSystem.s3n.impl: "org.apache.hadoop.fs.s3a.S3A"
          - spark.hadoop.fs.s3.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
          - spark.hadoop.fs.AbstractFileSystem.s3.impl: "org.apache.hadoop.fs.s3a.S3A"
          - spark.hadoop.fs.s3a.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
          - spark.hadoop.fs.AbstractFileSystem.s3a.impl: "org.apache.hadoop.fs.s3a.S3A"
          - spark.network.timeout: 600s
          - spark.executorEnv.KUBERNETES_REQUEST_TIMEOUT: 100000
          - spark.executor.heartbeatInterval: 60s
  clusterResourceTemplates:
  inline:
    #This section automates the creation of the project-domain namespaces
    - key: aa_namespace
      value: |
        apiVersion: v1
        kind: Namespace
        metadata:
          name: {{ namespace }}
        spec:
          finalizers:
          - kubernetes
    # This block performs the automated annotation of KSAs across all project-domain namespaces
    - key: ab_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: default
          namespace: '{{ namespace }}'
          annotations:
            eks.amazonaws.com/role-arn: '{{ defaultIamRole }}'
    - key: ac_spark_role
      value: |
        apiVersion: rbac.authorization.k8s.io/v1
        kind: Role
        metadata:
          name: spark-role
          namespace: "{{ namespace }}"
        rules:
        - apiGroups: ["*"]
          resources:
          - pods
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - services
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - configmaps
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - persistentvolumeclaims
          verbs:
          - "*"
    - key: ad_spark_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: spark
          namespace: "{{ namespace }}"
          annotations:
            eks.amazonaws.com/role-arn: '{{ defaultIamRole }}'
    - key: ae_spark_role_binding
      value: |
        apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: spark-role-binding
          namespace: "{{ namespace }}"
        roleRef:
          apiGroup: rbac.authorization.k8s.io
          kind: Role
          name: spark-role
        subjects:
          - kind: ServiceAccount
            name: spark
            namespace: "{{ namespace }}"
```
2. (Optional) The Spark operator supports Kubernetes ResourceQuota enforcement. If you plan to use it, set [per-Task resource requests](https://docs.flyte.org/en/latest/user_guide/productionizing/customizing_task_resources.html#customizing-task-resources)that fit into the quota for each project-namespace. A Task without resource requests or limits will be rejected by the K8s scheduler as described [in the Kubernetes docs](https://kubernetes.io/docs/concepts/policy/resource-quotas/).
The following is a sample configuration you can add to your Helm chart values, adjusting the resources to match your needs:

```yaml

customData:
  - production:
      - projectQuotaCpu:
          value: "5"
      - projectQuotaMemory:
          value: "4000Mi"
  - staging:
      - projectQuotaCpu:
          value: "2"
      - projectQuotaMemory:
          value: "3000Mi"
  - development:
      - projectQuotaCpu:
          value: "4"
      - projectQuotaMemory:
          value: "3000Mi"
```
Plus an additional Cluster Resource template to automate the creation of the ``ResourceQuota``:

```yaml

templates:
  - key: ab_project_resource_quota
    value: |
      apiVersion: v1
      kind: ResourceQuota
      metadata:
        name: project-quota
        namespace: {{ namespace }}
      spec:
        hard:
          limits.cpu: {{ projectQuotaCpu }}
          limits.memory: {{ projectQuotaMemory }}
```
## flyte-binary on GCP

> Check out the [reference implementation for GCP](https://github.com/unionai-oss/deploy-flyte/blob/main/environments/gcp/flyte-core/README.md)for information on how all the Flyte prerequisites are configured.

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configuration:
  inline:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - spark
        default-for-task-types:
          - container: container
          - container_array: k8s-array
          - spark: spark
    cluster_resources:
      - production:
        - gsa:
            value: <GoogleServiceAccount-EMAIL>
      - staging:
        - gsa:
            value: <GoogleServiceAccount-EMAIL>
      - development:
        - gsa:
            value: <GoogleServiceAccount-EMAIL>
    plugins:
      spark:
      # Edit the Spark configuration as you see fit
        spark-config-default:
          - spark.eventLog.enabled: "true"
          - spark.eventLog.dir: "{{ .Values.userSettings.bucketName }}/spark-events"
          - spark.driver.cores: "1"
          - spark.executorEnv.HTTP2_DISABLE: "true"
          - spark.hadoop.fs.AbstractFileSystem.gs.impl: com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS
          - spark.kubernetes.allocation.batch.size: "50"
          - spark.kubernetes.driverEnv.HTTP2_DISABLE: "true"
          - spark.network.timeout: 600s
          - spark.executorEnv.KUBERNETES_REQUEST_TIMEOUT: 100000
          - spark.executor.heartbeatInterval: 60s

clusterResourceTemplates:
  inline:
    #This section automates the creation of the project-domain namespaces
    - key: aa_namespace
      value: |
        apiVersion: v1
        kind: Namespace
        metadata:
          name: {{ namespace }}
        spec:
          finalizers:
          - kubernetes
    # This block performs the automated annotation of KSAs across all project-domain namespaces
    - key: ab_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: default
          namespace: '{{ namespace }}'
          annotations:
            iam.gke.io/gcp-service-account: {{ gsa }}
    - key: ac_spark_role
      value: |
        apiVersion: rbac.authorization.k8s.io/v1
        kind: Role
        metadata:
          name: spark-role
          namespace: "{{ namespace }}"
        rules:
        - apiGroups: ["*"]
          resources:
          - pods
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - services
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - configmaps
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - persistentvolumeclaims
          verbs:
          - "*"
    - key: ad_spark_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: spark
          namespace: "{{ namespace }}"
          annotations:
            iam.gke.io/gcp-service-account: {{ gsa }}
    - key: ae_spark_role_binding
      value: |
        apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: spark-role-binding
          namespace: "{{ namespace }}"
        roleRef:
          apiGroup: rbac.authorization.k8s.io
          kind: Role
          name: spark-role
        subjects:
          - kind: ServiceAccount
            name: spark
            namespace: "{{ namespace }}"
```

## flyte-core on AWS

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configmap:
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - spark
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          spark: spark
cluster_resource_manager:
  enabled: true
  standalone_deploy: false
  # -- Resource templates that should be applied
  templates:
    # -- Template for namespaces resources
    - key: aa_namespace
      value: |
        apiVersion: v1
        kind: Namespace
        metadata:
          name: {{ namespace }}
        spec:
          finalizers:
          - kubernetes
    - key: ac_spark_role
      value: |
        apiVersion: rbac.authorization.k8s.io/v1beta1
        kind: Role
        metadata:
          name: spark-role
          namespace: {{ namespace }}
        rules:
        - apiGroups: ["*"]
          resources:
          - pods
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - services
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - configmaps
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - persistentvolumeclaims
          verbs:
          - "*"

    - key: ad_spark_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: spark
          namespace: {{ namespace }}

    - key: ae_spark_role_binding
      value: |
        apiVersion: rbac.authorization.k8s.io/v1beta1
        kind: RoleBinding
        metadata:
          name: spark-role-binding
          namespace: {{ namespace }}
        roleRef:
          apiGroup: rbac.authorization.k8s.io
          kind: Role
          name: spark-role
        subjects:
        - kind: ServiceAccount
          name: spark
          namespace: {{ namespace }}

sparkoperator:
  enabled: true
  plugin_config:
    plugins:
      spark:
        # Edit the Spark configuration as you see fit
        spark-config-default:
          - spark.driver.cores: "1"
          - spark.hadoop.fs.s3a.aws.credentials.provider: "com.amazonaws.auth.DefaultAWSCredentialsProviderChain"
          - spark.kubernetes.allocation.batch.size: "50"
          - spark.hadoop.fs.s3a.acl.default: "BucketOwnerFullControl"
          - spark.hadoop.fs.s3n.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
          - spark.hadoop.fs.AbstractFileSystem.s3n.impl: "org.apache.hadoop.fs.s3a.S3A"
          - spark.hadoop.fs.s3.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
          - spark.hadoop.fs.AbstractFileSystem.s3.impl: "org.apache.hadoop.fs.s3a.S3A"
          - spark.hadoop.fs.s3a.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
          - spark.hadoop.fs.AbstractFileSystem.s3a.impl: "org.apache.hadoop.fs.s3a.S3A"
          - spark.network.timeout: 600s
          - spark.executorEnv.KUBERNETES_REQUEST_TIMEOUT: 100000
          - spark.executor.heartbeatInterval: 60s
```
## flyte-core on GCP

>  Check out the [reference implementation for GCP](https://github.com/unionai-oss/deploy-flyte/blob/main/environments/gcp/flyte-core/README.md) for guidance on how all the Flyte prerequisites are configured.

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

enabled_plugins:
  tasks:
    task-plugins:
      enabled-plugins:
        - container
        - sidecar
        - k8s-array
        - spark
      default-for-task-types:
        container: container
        sidecar: sidecar
        container_array: k8s-array
        spark: spark
cluster_resource_manager:
  enabled: true
  standalone_deploy: false
  config:
    cluster_resources:
      customData:
      - production:
          - gsa:
          #This is the GSA that the Task Pods will use to access GCP resources.
              value: "<GoogleServiceAccount-email>"
      - staging:
          - gsa:
              value: "<GoogleServiceAccount-email>"
      - development:
          - gsa:
              value: "<GoogleServiceAccount-email>"
  templates:
    # -- Template for namespaces resources
    - key: aa_namespace
      value: |
        apiVersion: v1
        kind: Namespace
        metadata:
          name: {{ namespace }}
        spec:
          finalizers:
          - kubernetes
    # -- Patch default service account
    - key: aab_default_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: default
          namespace: {{ namespace }}
          annotations:
            # Annotation needed for GCP Workload Identity to function
            # https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity
            iam.gke.io/gcp-service-account: {{ gsa }}
    - key: ac_spark_role
      value: |
        apiVersion: rbac.authorization.k8s.io/v1
        kind: Role
        metadata:
          name: spark-role
          namespace: "{{ namespace }}"
        rules:
        - apiGroups: ["*"]
          resources:
          - pods
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - services
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - configmaps
          verbs:
          - '*'
        - apiGroups: ["*"]
          resources:
          - persistentvolumeclaims
          verbs:
          - "*"
    #While the Spark Helm chart creates a spark ServiceAccount, this template creates one
    # on each project-domain namespace and annotates it with the GSA
    #You should always run workflows with the Spark service account (eg pyflyte run --remote --service-account=spark ...)
    - key: ad_spark_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: spark
          namespace: "{{ namespace }}"
          annotations:
            iam.gke.io/gcp-service-account: {{ gsa }}
    - key: ae_spark_role_binding
      value: |
        apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: spark-role-binding
          namespace: "{{ namespace }}"
        roleRef:
          apiGroup: rbac.authorization.k8s.io
          kind: Role
          name: spark-role
        subjects:
          - kind: ServiceAccount
            name: spark
            namespace: "{{ namespace }}"
sparkoperator:
enabled: true
plugins:
  spark:
    spark-config-default:
      - spark.eventLog.enabled: "true"
      - spark.eventLog.dir: "{{ .Values.userSettings.bucketName }}/spark-events"
      - spark.driver.cores: "1"
      - spark.executorEnv.HTTP2_DISABLE: "true"
      - spark.hadoop.fs.AbstractFileSystem.gs.impl: com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS
      - spark.kubernetes.allocation.batch.size: "50"
      - spark.kubernetes.driverEnv.HTTP2_DISABLE: "true"
      - spark.network.timeout: 600s
      - spark.executorEnv.KUBERNETES_REQUEST_TIMEOUT: 100000
      - spark.executor.heartbeatInterval: 60s
```
## flyte-sandbox

If you installed the [flyte-sandbox](https://github.com/flyteorg/flyte/tree/master/charts/flyte-sandbox) Helm chart to a K8s cluster, follow this section to configure the Spark plugin.
Note that none of this configuration applies to the demo cluster that you spin up with ``flytectl demo start``.

1. Create a file named ``values-override.yaml`` and add the following config to it:

> Within the flyte-binary block, the value of ``inline.storage.signedURL.stowConfigOverride.endpoint`` should be set to the corresponding node Hostname/IP on the MinIO pod if you are deploying on a Kubernetes cluster.

```yaml

flyte-binary:
  nameOverride: flyte-sandbox
  enabled: true
  configuration:
    database:
      host: '{{ printf "%s-postgresql" .Release.Name | trunc 63 | trimSuffix "-" }}'
      password: postgres
    storage:
      metadataContainer: my-s3-bucket
      userDataContainer: my-s3-bucket
      provider: s3
      providerConfig:
        s3:
          disableSSL: true
          v2Signing: true
          endpoint: http://{{ printf "%s-minio" .Release.Name | trunc 63 | trimSuffix "-" }}.{{ .Release.Namespace }}:9000
          authType: accesskey
          accessKey: minio
          secretKey: miniostorage
    logging:
      level: 5
      plugins:
        kubernetes:
          enabled: true
          templateUri: |-
            http://localhost:30080/kubernetes-dashboard/#/log/{{.namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}
    inline:
      task_resources:
        defaults:
          cpu: 500m
          ephemeralStorage: 0
          gpu: 0
          memory: 1Gi
        limits:
          cpu: 0
          ephemeralStorage: 0
          gpu: 0
          memory: 0
      storage:
        signedURL:
          stowConfigOverride:
            endpoint: http://localhost:30002
      plugins:
        k8s:
          default-env-vars:
            - FLYTE_AWS_ENDPOINT: http://{{ printf "%s-minio" .Release.Name | trunc 63 | trimSuffix "-" }}.{{ .Release.Namespace }}:9000
            - FLYTE_AWS_ACCESS_KEY_ID: minio
            - FLYTE_AWS_SECRET_ACCESS_KEY: miniostorage
        spark:
          spark-config-default:
            - spark.driver.cores: "1"
            - spark.hadoop.fs.s3a.aws.credentials.provider: "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider"
            - spark.hadoop.fs.s3a.endpoint: http://{{ printf "%s-minio" .Release.Name | trunc 63 | trimSuffix "-" }}.{{ .Release.Namespace }}:9000
            - spark.hadoop.fs.s3a.access.key: "minio"
            - spark.hadoop.fs.s3a.secret.key: "miniostorage"
            - spark.hadoop.fs.s3a.path.style.access: "true"
            - spark.kubernetes.allocation.batch.size: "50"
            - spark.hadoop.fs.s3a.acl.default: "BucketOwnerFullControl"
            - spark.hadoop.fs.s3n.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
            - spark.hadoop.fs.AbstractFileSystem.s3n.impl: "org.apache.hadoop.fs.s3a.S3A"
            - spark.hadoop.fs.s3.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
            - spark.hadoop.fs.AbstractFileSystem.s3.impl: "org.apache.hadoop.fs.s3a.S3A"
            - spark.hadoop.fs.s3a.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
            - spark.hadoop.fs.AbstractFileSystem.s3a.impl: "org.apache.hadoop.fs.s3a.S3A"
    inlineConfigMap: '{{ include "flyte-sandbox.configuration.inlineConfigMap" . }}'
  clusterResourceTemplates:
    inlineConfigMap: '{{ include "flyte-sandbox.clusterResourceTemplates.inlineConfigMap" . }}'
  deployment:
    image:
      repository: flyte-binary
      tag: sandbox
      pullPolicy: Never
    waitForDB:
      image:
        repository: bitnami/postgresql
        tag: sandbox
        pullPolicy: Never
  rbac:
    # This is strictly NOT RECOMMENDED in production clusters, and is only for use
    # within local Flyte sandboxes.
    # When using cluster resource templates to create additional namespaced roles,
    # Flyte is required to have a superset of those permissions. To simplify
    # experimenting with new backend plugins that require additional roles be created
    # with cluster resource templates (e.g. Spark), we add the following:
    extraRules:
      - apiGroups:
        - '*'
        resources:
        - '*'
        verbs:
        - '*'
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - connector-service
          - spark
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          spark: spark
```

</details>

<details>
<summary>Dask</summary>

## flyte-binary

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

tasks:
task-plugins:
  enabled-plugins:
    - container
    - sidecar
    - dask
  default-for-task-types:
    - container: container
    - dask: dask
```
## flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configmap:
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - dask
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          dask: dask
```

</details>

<details>
<summary>Volcano-Scheduled PodTask</summary>

### flyte-binary

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configuration:
  inline:
    plugins:
      k8s:
        enable-create-pod-group-for-pod: true
```
### flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml

configmap:
  k8s:
    plugins:
      k8s:
        enable-create-pod-group-for-pod: true
```

</details>

## Upgrade the deployment

```bash
helm upgrade <RELEASE_NAME> flyteorg/<HELM_CHART> -n <YOUR_NAMESPACE> --values values-override.yaml

```

Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``),
``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``) and `<HELM_CHART>` with `flyte-binary`, `flyte-core ` or `flyte-sandbox`.

Wait for the upgrade to complete. You can check the status of the deployment pods by running the following command:

```bash
kubectl get pods -n flyte
```

Once all the components are up and running, go to the examples section to learn more about how to use Flyte backend plugins.

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/athena ===

# Athena Plugin

This guide provides an overview of setting up Athena in your Flyte deployment.

> Please note that the Athena plugin requires a Flyte deployment in the AWS cloud; it won't work with demo/GCP/Azure.

## Set up the AWS Flyte cluster

1. Ensure you have a functional Flyte cluster up and running in `AWS <https://docs.flyte.org/en/latest/deployment/aws/index.html#deployment-aws>`__
2. Verify that you have the correct ``kubeconfig`` and have selected the appropriate Kubernetes context
3. Double-check that your ``~/.flyte/config.yaml`` file contains the correct Flytectl configuration

## Specify plugin configuration

### flyte-binary

Edit the relevant YAML file to specify the plugin.

```yaml
tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - k8s-array
      - athena
    default-for-task-types:
      - container: container
      - container_array: k8s-array
     - athena: athena
```

## flyte-core

Create a file named ``values-override.yaml`` and include the following configuration:

```yaml
        configmap:
          enabled_plugins:
            tasks:
              task-plugins:
                enabled-plugins:
                  - container
                  - sidecar
                  - k8s-array
                  - athena
                default-for-task-types:
                  container: container
                  sidecar: sidecar
                  container_array: k8s-array
                  athena: athena
```
Ensure that the propeller has the correct service account for Athena.

## Upgrade the Flyte Helm release

```bash
      helm upgrade <RELEASE_NAME> flyteorg/<HELM_CHART> -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
```
Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``), ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``), `<HELM_CHART>` with either `flyte-binary` or `flyte-core` and ``<YOUR_YAML_FILE>`` with the name of your YAML file.

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/batch ===

# AWS Batch

This setup document applies to both MapTasks and regular tasks running on AWS Batch.

> For single [non-map] task use, please take note of the additional code when updating the flytepropeller config.

AWS Batch simplifies the process for developers, scientists and engineers to run
hundreds of thousands of batch computing jobs on AWS.

Flyte abstracts away the complexity of integrating AWS Batch into users' workflows,
taking care of packaging inputs, reading outputs, scheduling map tasks and
optimizing AWS Batch job queues for load distribution and priority coordination.

## Set up AWS Batch

Follow the guide [Running batch jobs at scale for less](https://aws.amazon.com/getting-started/hands-on/run-batch-jobs-at-scale-with-ec2-spot/).

By the end of this step, your AWS Account should have a configured compute environment
and one or more AWS Batch Job Queues.

### Modify AWS IAM role trust policy document

Follow the guide [AWS Batch Execution IAM role](https://docs.aws.amazon.com/batch/latest/userguide/execution-IAM-role.html).

When running workflows in Flyte, users can specify a Kubernetes service account and/or an IAM Role to run as.
For AWS Batch, an IAM Role must be specified. For each of these IAM Roles, modify the trust policy
to allow elastic container service (ECS) to assume the role.

### Modify system's AWS IAM role policies

Follow the guide [Granting a user permissions to pass a role to an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_passrole.html).

The best practice for granting permissions to Flyte components is by utilizing OIDC,
as described in the
[OIDC documentation](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
This approach entails assigning an IAM Role to each service account being used.
To proceed, identify the IAM Role associated with the flytepropeller's Kubernetes service account,
and subsequently, modify the policy document to enable the role to pass other roles to AWS Batch.

### Update FlyteAdmin configuration

FlyteAdmin must be informed of all the AWS Batch job queues
and how the system should distribute the load among them.
The simplest setup is as follows:

```yaml
  flyteadmin:
    roleNameKey: "eks.amazonaws.com/role-arn"
  queues:
    # A list of items, one per AWS Batch Job Queue.
    executionQueues:
      # The name of the job queue from AWS Batch
      - dynamic: "tutorial"
        # A list of tags/attributes that can be used to match workflows to this queue.
        attributes:
          - default
    # A list of configs to match project and/or domain and/or workflows to job queues using tags.
    workflowConfigs:
      # An empty rule to match any workflow to the queue tagged as "default"
      - tags:
          - default
```
If you are using Helm, you can add this block under the ``configMaps.adminServer`` section,
as shown [here](https://github.com/flyteorg/flyte/blob/95baed556f5844e6a494507c3aa5a03fe6d42fbb/charts/flyte-core/values.yaml#L12).

For a more complex matching configuration, the example below defines three different queues
with distinct attributes and matching logic based on project/domain/workflowName.

```yaml
   queues:
     executionQueues:
       - dynamic: "gpu_dynamic"
         attributes:
         - gpu
       - dynamic: "critical"
         attributes:
         - critical
       - dynamic: "default"
         attributes:
         - default
     workflowConfigs:
       - project: "my_queue_1"
         domain: "production"
         workflowName: "my_workflow_1"
         tags:
         - critical
       - project: "production"
         workflowName: "my_workflow_2"
         tags:
         - gpu
       - project: "my_queue_3"
         domain: "production"
         workflowName: "my_workflow_3"
         tags:
         - critical
       - tags:
         - default
```
These settings can also be dynamically altered through ``flytectl`` (or FlyteAdmin API).
[Learn about how to customize resources](https://www.union.ai/docs/v1/flyte/deployment/flyte-configuration/customizable_resources).

### Update FlytePropeller's configuration

The AWS Array Plugin requires specific configurations to ensure proper communication with the AWS Batch Service.

These configurations reside within FlytePropeller's configMap. Modify the config in the relevant YAML file to set the following keys:

```yaml

  plugins:
    aws:
      batch:
        # Must match that set in flyteAdmin's configMap flyteadmin.roleNameKey
        roleAnnotationKey: eks.amazonaws.com/role-arn
      # Must match the desired region to launch these tasks.
      region: us-east-2
  tasks:
    task-plugins:
      enabled-plugins:
        # Enable aws_array task plugin.
        - aws_array
      default-for-task-types:
        # Set it as the default handler for array/map tasks.
        container_array: aws_array
        # Make sure to add this line to enable single (non-map) AWS Batch tasks
        aws-batch: aws_array
```
> To register the map task on Flyte, use the command ``pyflyte register <name-of-the-python-file>``. Launch the execution through the FlyteConsole by selecting the appropriate ``IAM Role`` and entering the full ``AWS ARN`` of an IAM Role configured according to the above guide.

Once the task starts executing, you'll find a link for the AWS Array Job in the log links section of the Flyte Console.
As individual jobs start getting scheduled, links to their respective CloudWatch log streams will also appear in the UI.

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/sagemaker ===

# Sagemaker Plugin Setup

This guide gives an overview of how to set up Sagemaker in your Flyte deployment.

> The Sagemaker plugin needs Flyte deployment in AWS cloud; sandbox/GCP/Azure won't work.

## Prerequisites

* Flyte cluster in [AWS](https://docs.flyte.org/en/latest/deployment/aws/index.html#deployment-aws)
* [AWS role set up for SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html)
* [AWS SageMaker K8s operator](https://github.com/aws/amazon-sagemaker-operator-for-k8s) is installed in your k8s cluster
* Correct kubeconfig and Kubernetes context
* Correct FlyteCTL config at `~/.flyte/config.yaml`

## Specify Plugin Configuration

Create a file named ``values-override.yaml`` and add the following config to it.
Please make sure that the propeller has the correct service account for Sagemaker.

```yaml
    configmap:
      enabled_plugins:
        # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig)
        tasks:
          # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig)
          task-plugins:
            # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config).
            # plugins
            enabled-plugins:
              - container
              - sidecar
              - k8s-array
              - sagemaker_training
              - sagemaker_hyperparameter_tuning
            default-for-task-types:
              container: container
              sidecar: sidecar
              container_array: k8s-array
```
## Upgrade the Flyte Helm release

```bash
helm upgrade -n flyte -f values-override.yaml flyteorg/flyte-core
```

## Register the Sagemaker plugin example

```bash
flytectl register files https://github.com/flyteorg/flytesnacks/releases/download/v0.3.0/snacks-cookbook-integrations-aws-sagemaker_training.tar.gz --archive -p flytesnacks -d development
```

### Launch an execution

#### Flyte console UI

* Navigate to Flyte Console's UI (e.g. `sandbox <http://localhost:30081/console>`_) and find the workflow.
* Click on `Launch` to open up the launch form.
* Submit the form.

#### flytectl

Retrieve an execution form in the form of a YAML file:

```bash
flytectl get launchplan --config ~/.flyte/flytectl.yaml \
    --project flytesnacks \
    --domain development \
    sagemaker_training.sagemaker_custom_training.mnist_trainer \
    --latest \
    --execFile exec_spec.yaml
 ```
Launch! 🚀

```bash
flytectl --config ~/.flyte/flytectl.yaml create execution \
    -p <project> -d <domain> --execFile ~/exec_spec.yaml
```

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/bigquery ===

# Google BigQuery Plugin

This guide provides an overview of setting up BigQuery in your Flyte deployment.
Please note that the BigQuery plugin requires Flyte deployment in the GCP cloud;
it is not compatible with demo/AWS/Azure.

## Set up the GCP Flyte cluster

* Ensure you have a functional Flyte cluster running in `GCP <https://docs.flyte.org/en/latest/deployment/gcp/index.html#deployment-gcp>`__.
* Create a service account for BigQuery. For more details, refer to [GCP docs](https://cloud.google.com/bigquery/docs/quickstarts/quickstart-client-libraries).
* Verify that you have the correct kubeconfig and have selected the appropriate Kubernetes context.
* Confirm that you have the correct Flytectl configuration at ``~/.flyte/config.yaml``.

## Specify plugin configuration

### flyte-binary

Edit the relevant YAML file to specify the plugin.

```yaml

      tasks:
        task-plugins:
          enabled-plugins:
            - container
            - sidecar
            - bigquery
          default-for-task-types:
            - container: container
            - bigquery_query_job_task: bigquery
```

### flyte-core

Create a file named ``values-override.yaml`` and add the following configuration to it.

```yaml
        configmap:
          enabled_plugins:
            # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig)
            tasks:
              # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig)
              task-plugins:
                # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend
                enabled-plugins:
                  - container
                  - sidecar
                  - k8s-array
                  - bigquery
                default-for-task-types:
                  container: container
                  sidecar: sidecar
                  container_array: k8s-array
                  bigquery_query_job_task: bigquery
```

Ensure that flytepropeller has the correct service account for BigQuery.

## Upgrade the Flyte Helm release

```bash
helm upgrade <RELEASE_NAME> flyteorg/<HELM_CHART> -n <YOUR_NAMESPACE> --values values-override.yaml

```

Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``),
``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``) and `<HELM_CHART>` with `flyte-binary`, `flyte-core ` or `flyte-sandbox`.

Wait for the upgrade to complete. You can check the status of the deployment pods by running the following command:

```bash
kubectl get pods -n flyte
```

Once all the components are up and running, go to the examples section to learn more about how to use Flyte plugins.

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/databricks ===

# Databricks Plugin

This guide provides an overview of how to set up Databricks in your Flyte deployment.

## Databricks workspace

To set up your Databricks account, follow these steps:

1. Create a [Databricks account](https://www.databricks.com/).
2. Ensure that you have a Databricks workspace up and running.
3. Generate a [personal access token](https://docs.databricks.com/dev-tools/auth.html#databricks-personal-ACCESS_TOKEN-authentication) to be used in the Flyte configuration. You can find the personal access token in the user settings within the workspace.
4. When testing the Databricks plugin on the demo cluster, create an S3 bucket because the local demo cluster utilizes MinIO. Follow the [AWS instructions](https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html) to generate access and secret keys, which can be used to access your preferred S3 bucket.
5. Create an [instance profile](https://docs.databricks.com/administration-guide/cloud-configurations/aws/instance-profiles.html) for the Spark cluster. This profile enables the Spark job to access your data in the S3 bucket.
Please follow all four steps specified in the documentation.

Upload the following entrypoint.py file to either
[DBFS](https://docs.databricks.com/archive/legacy/data-tab.html)
(the final path can be ``dbfs:///FileStore/tables/entrypoint.py``) or S3.
This file will be executed by the Spark driver node, overriding the default command in the
[dbx](https://docs.databricks.com/dev-tools/dbx.html) job.

```python

  import os
  import sys
  from typing import List

  import click
  import pandas
  from flytekit.bin.entrypoint import fast_execute_task_cmd as _fast_execute_task_cmd
  from flytekit.bin.entrypoint import execute_task_cmd as _execute_task_cmd
  from flytekit.exceptions.user import FlyteUserException
  from flytekit.tools.fast_registration import download_distribution

  def fast_execute_task_cmd(additional_distribution: str, dest_dir: str, task_execute_cmd: List[str]):
      if additional_distribution is not None:
          if not dest_dir:
              dest_dir = os.getcwd()
          download_distribution(additional_distribution, dest_dir)

      # Insert the call to fast before the unbounded resolver args
      cmd = []
      for arg in task_execute_cmd:
          if arg == "--resolver":
              cmd.extend(["--dynamic-addl-distro", additional_distribution, "--dynamic-dest-dir", dest_dir])
          cmd.append(arg)

      click_ctx = click.Context(click.Command("dummy"))
      parser = _execute_task_cmd.make_parser(click_ctx)
      args, _, _ = parser.parse_args(cmd[1:])
      _execute_task_cmd.callback(test=False, **args)

  def main():

      args = sys.argv

      click_ctx = click.Context(click.Command("dummy"))
      if args[1] == "pyflyte-fast-execute":
          parser = _fast_execute_task_cmd.make_parser(click_ctx)
          args, _, _ = parser.parse_args(args[2:])
          fast_execute_task_cmd(**args)
      elif args[1] == "pyflyte-execute":
          parser = _execute_task_cmd.make_parser(click_ctx)
          args, _, _ = parser.parse_args(args[2:])
          _execute_task_cmd.callback(test=False, dynamic_addl_distro=None, dynamic_dest_dir=None, **args)
      else:
          raise FlyteUserException(f"Unrecognized command: {args[1:]}")

  if __name__ == '__main__':
      main()
```

## Specify plugin configuration

### flyte-binary

Edit the relevant YAML file to specify the plugin.

```yaml
          tasks:
            task-plugins:
              enabled-plugins:
                - container
                - sidecar
                - k8s-array
                - databricks
              default-for-task-types:
                - container: container
                - container_array: k8s-array
                - spark: databricks

          inline:
            plugins:
              databricks:
                entrypointFile: dbfs:///FileStore/tables/entrypoint.py
                databricksInstance: <DATABRICKS_ACCOUNT>.cloud.databricks.com
```

Substitute ``<DATABRICKS_ACCOUNT>`` with the name of your Databricks account.

### flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml
      configmap:
        enabled_plugins:
          tasks:
            task-plugins:
              enabled-plugins:
                - container
                - sidecar
                - k8s-array
                - databricks
              default-for-task-types:
                container: container
                sidecar: sidecar
                container_array: k8s-array
                spark: databricks
      databricks:
        enabled: True
        plugin_config:
          plugins:
            databricks:
              entrypointFile: dbfs:///FileStore/tables/entrypoint.py
              databricksInstance: <DATABRICKS_ACCOUNT>.cloud.databricks.com
```
Substitute ``<DATABRICKS_ACCOUNT>`` with the name of your Databricks account.

## Add the Databricks access token

Add the Databricks access token to FlytePropeller:

### flyte-binary

Create a secret as follows (or add to it if it already exists from other plugins):

```bash
          cat <<EOF | kubectl apply -f -
          apiVersion: v1
          kind: Secret
          metadata:
            name: flyte-binary-external-services
            namespace: flyte
          type: Opaque
          stringData:
            FLYTE_DATABRICKS_API_TOKEN: <ACCESS_TOKEN>
          EOF
```
Reference the newly created secret in  ``.Values.configuration.inlineSecretRef`` in your YAML file as follows:

```yaml
          configuration:
            inlineSecretRef: flyte-binary-external-services
```
Replace ``<ACCESS_TOKEN>`` with your access token.

### flyte-core

Add the access token as a secret to ``flyte-secret-auth``.

```bash
kubectl edit secret -n flyte flyte-secret-auth
```

```yaml
      apiVersion: v1
      data:
        FLYTE_DATABRICKS_API_TOKEN: <ACCESS_TOKEN>
        client_secret: Zm9vYmFy
      kind: Secret
      ...
```
Replace ``<ACCESS_TOKEN>`` with your access token.

## Upgrade the deployment

```bash
helm upgrade <RELEASE_NAME> flyteorg/<HELM_CHART> -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
```
Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``), ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``), `<HELM_CHART>` with either `flyte-binary` or `flyte-core` and ``<YOUR_YAML_FILE>`` with the name of your YAML file.

> Make sure you enable [custom containers](https://docs.databricks.com/administration-guide/clusters/container-services.html) on your Databricks cluster before you trigger the workflow.

=== PAGE: https://www.union.ai/docs/v1/flyte/deployment/flyte-plugins/snowflake ===

# Snowflake Plugin

This guide provides an overview of how to set up Snowflake in your Flyte deployment.

## Specify plugin configuration

### flyte-binary

Edit the relevant YAML file to specify the plugin.

```yaml

          tasks:
            task-plugins:
              enabled-plugins:
                - container
                - sidecar
                - k8s-array
                - snowflake
              default-for-task-types:
                - container: container
                - container_array: k8s-array
                - snowflake: snowflake
```
### flyte-core

Create a file named ``values-override.yaml`` and add the following config to it:

```yaml
        configmap:
          enabled_plugins:
            # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig)
            tasks:
              # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig)
              task-plugins:
                # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend
                # plugins
                enabled-plugins:
                  - container
                  - sidecar
                  - k8s-array
                  - snowflake
                default-for-task-types:
                  container: container
                  sidecar: sidecar
                  container_array: k8s-array
                  snowflake: snowflake
```

## Obtain and add the Snowflake JWT token

Create a Snowflake account, and follow the [Snowflake docs](https://docs.snowflake.com/en/developer-guide/sql-api/authenticating#using-key-pair-authentication)
to generate a JWT token.
Then, add the Snowflake JWT token to FlytePropeller.

### flyte-binary

Create a secret as follows (or add to it if it already exists from other plugins):

```bash
          cat <<EOF | kubectl apply -f -
          apiVersion: v1
          kind: Secret
          metadata:
            name: flyte-binary-external-services
            namespace: flyte
          type: Opaque
          stringData:
            FLYTE_SNOWFLAKE_CLIENT_TOKEN: <JWT_TOKEN>
          EOF
  ```
Replace ``<JWT_TOKEN>`` with your JWT token.

Reference the newly created secret in ``.Values.configuration.inlineSecretRef`` in your YAML file as follows:

```yaml
configuration:
  inlineSecretRef: flyte-binary-external-services
```

### flyte-core

Add the JWT token as a secret to ``flyte-secret-auth``.

```bash
kubectl edit secret -n flyte flyte-secret-auth
```
```yaml
      apiVersion: v1
      data:
        FLYTE_SNOWFLAKE_CLIENT_TOKEN: <JWT_TOKEN>
        client_secret: Zm9vYmFy
      kind: Secret
      ...
```
Replace ``<JWT_TOKEN>`` with your JWT token.

### Upgrade the deployment

```bash
helm upgrade <RELEASE_NAME> flyteorg/<HELM_CHART> -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
```
Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``), ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``), `<HELM_CHART>` with either `flyte-binary` or `flyte-core` and ``<YOUR_YAML_FILE>`` with the name of your YAML file.

  For Snowflake plugin on the Flyte cluster, please refer to `Snowflake Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/snowflake_plugin/snowflake.html>`_