Enabling Azure Key Vault

This documentation exists for customers who must use Azure Key Vault for organizational reasons. For everyone else, we strongly recommend using the Union.ai secrets manager to manage secrets rather than Azure Key Vault.

The Union.ai-managed userflyterole identity must be granted permission to access Azure Key Vault secrets.

Managing Azure Key Vault secrets

Refer to Azure official documentation for details on creating and managing secrets.

Providing permissions to Azure Key Vault

Union.ai data plane tasks employ Azure Workload Identity Federation to access Azure resources using an Azure user-assigned identity. Access to Azure Key Vault containers requires updating permissions to permit this Union.ai-managed user-assigned identity.

Create a role assignment assigning the Key Vault Secrets User role to the userflyterole user-assigned identity. Make sure it is scoped to the Azure Key Vault Secret.

Union.ai managed user-assigned identities

Refer to Azure portal’s user assigned managed identitites if assistance is required identifying the userflyterole user-assigned identity within the Union.ai data plane resource group.

Accessing the secret within Union.ai

  • Define a Secret object where
    • Secret.group is the a HTTP URI of the format https://<KEY_VAULT_NAME>.vault.azure.net/secrets/<SECRET_NAME>
    • Secret.group_version can be omitted to retrieve the latest version or set to an explicit secret version
    • Secret.mount_requirement is Secret.MountType.FILE
  • Pass that Secret object in the secret_requests parameter of the @union.task decorator.
  • Inside the task code, retrieve the value of the secret with:
    • union.current_context().secrets.get(<SECRET_NAME>) if Secret.group_version was omitted.
    • union.current_context().secrets.get(<SECRET_NAME>, group_version=SECRET_GROUP_VERSION) if Secret.group_version was specified.

Here are examples:

import union

VAULT_NAME = "examplevault"
SECRET_NAME = "example-secret"

SECRET_GROUP = f"https://{VAULT_NAME}.vault.azure.net/secrets/{SECRET_NAME}"
SECRET_GROUP_VERSION = "12345"

SECRET_REQUEST_WITH_VERSION = union.Secret(
  group=SECRET_GROUP,
  group_version=SECRET_GROUP_VERSION,
  mount_requirement=union.Secret.MountType.FILE
)

@union.task(secret_requests=[SECRET_REQUEST_WITH_VERSION])
def task_with_versioned_secret():
    secret_val = union.current_context().secrets.get(
        SECRET_NAME,
        group_version=SECRET_GROUP_VERSION
    )

SECRET_REQUEST_FOR_LATEST = union.Secret(
  group=SECRET_GROUP,
  mount_requirement=union.Secret.MountType.FILE
)

@union.task(secret_requests=[SECRET_REQUEST_FOR_LATEST])
def task_with_latest_secret():
    secret_val = union.current_context().secrets.get(
        SECRET_NAME
    )