Secret-based authentication

In this guide, we’ll deploy a FastAPI app that uses API key authentication with Flyte secrets. This allows you to invoke the endpoint from the public internet in a secure manner without exposing API keys in your code.

Create the secret

Before defining and deploying the app, you need to create the API_KEY secret in Flyte. This secret will store your API key securely.

Create the secret using the Flyte CLI:

flyte create secret API_KEY <your-api-key-value>

For example:

flyte create secret API_KEY my-secret-api-key-12345

The secret name API_KEY must match the key specified in the flyte.Secret() call in your code. The secret will be available to your app as the environment variable specified in as_env_var.

Define the FastAPI app

Here’s a simple FastAPI app that uses HTTPAuthorizationCredentials to authenticate requests using a secret stored in Flyte:

basic_auth.py

"""Basic FastAPI authentication using dependency injection."""

from fastapi import FastAPI, HTTPException, Security
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from starlette import status
import os
import pathlib
import flyte
from flyte.app.extras import FastAPIAppEnvironment

# Get API key from environment variable (loaded from Flyte secret)
# The secret must be created using: flyte create secret API_KEY <your-api-key-value>
API_KEY = os.getenv("API_KEY")
security = HTTPBearer()

async def verify_token(
    credentials: HTTPAuthorizationCredentials = Security(security),
) -> HTTPAuthorizationCredentials:
    """Verify the API key from the bearer token."""
    if not API_KEY:
        raise HTTPException(
            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
            detail="API_KEY not configured",
        )
    if credentials.credentials != API_KEY:
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
            detail="Could not validate credentials",
        )
    return credentials

app = FastAPI(title="Authenticated API")

@app.get("/public")
async def public_endpoint():
    """Public endpoint that doesn't require authentication."""
    return {"message": "This is public"}

@app.get("/protected")
async def protected_endpoint(
    credentials: HTTPAuthorizationCredentials = Security(verify_token),
):
    """Protected endpoint that requires authentication."""
    return {
        "message": "This is protected",
        "user": credentials.credentials,
    }

env = FastAPIAppEnvironment(
    name="authenticated-api",
    app=app,
    image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
        "fastapi",
        "uvicorn",
    ),
    resources=flyte.Resources(cpu=1, memory="512Mi"),
    requires_auth=False,  # We handle auth in the app
    secrets=flyte.Secret(key="API_KEY", as_env_var="API_KEY"),
)

if __name__ == "__main__":
    flyte.init_from_config(root_dir=pathlib.Path(__file__).parent)
    app_deployment = flyte.deploy(env)
    print(f"API URL: {app_deployment[0].url}")
    print(f"Swagger docs: {app_deployment[0].url}/docs")

As you can see, we:

Define a FastAPI app
Create a verify_token function that verifies the API key from the Bearer token
Define endpoints that use the verify_token function to authenticate requests
Configure the FastAPIAppEnvironment with:
- requires_auth=False - This allows the endpoint to be reached without going through Flyte’s authentication, since we’re handling authentication ourselves using the API_KEY secret
- secrets=flyte.Secret(key="API_KEY", as_env_var="API_KEY") - This injects the secret value into the API_KEY environment variable at runtime

The key difference from using env_vars is that secrets are stored securely in Flyte’s secret store and injected at runtime, rather than being passed as plain environment variables.

Deploy the FastAPI app

Once the secret is created, you can deploy the FastAPI app. Make sure your config.yaml file is in the same directory as your script, then run:

python basic_auth.py

Or use the Flyte CLI:

flyte serve basic_auth.py

Deploying the application will stream the status to the console and display the app URL:

            
        
✨ Deploying Application: authenticated-api
🔎 Console URL: https://<union-tenant>/console/projects/my-project/domains/development/apps/fastapi-with-auth
[Status] Pending: App is pending deployment
[Status] Started: Service is ready
🚀 Deployed Endpoint: https://rough-meadow-97cf5.apps.<union-tenant>

Invoke the endpoint

Once deployed, you can invoke the authenticated endpoint using curl:

            
curl -X GET "https://rough-meadow-97cf5.apps.<union-tenant>/protected" \
  -H "Authorization: Bearer <your-api-key-value>"

Replace <your-api-key-value> with the actual API key value you used when creating the secret.

For example, if you created the secret with value my-secret-api-key-12345:

            
curl -X GET "https://rough-meadow-97cf5.apps.<union-tenant>/protected" \
  -H "Authorization: Bearer my-secret-api-key-12345"

You should receive a response:

            
        
{
  "message": "This is protected",
  "user": "my-secret-api-key-12345"
}

Authentication for vLLM and SGLang apps

Both vLLM and SGLang apps support API key authentication through their native --api-key argument. This allows you to secure your LLM endpoints while keeping them accessible from the public internet.

Create the authentication secret

Create a secret to store your API key:

flyte create secret AUTH_SECRET <your-api-key-value>

For example:

flyte create secret AUTH_SECRET my-llm-api-key-12345

Deploy vLLM app with authentication

Here’s how to deploy a vLLM app with API key authentication:

vllm_with_auth.py

"""vLLM app with API key authentication."""

import pathlib
from flyteplugins.vllm import VLLMAppEnvironment
import flyte


# The secret must be created using: flyte create secret AUTH_SECRET <your-api-key-value>
vllm_app = VLLMAppEnvironment(
    name="vllm-app-with-auth",
    model_hf_path="Qwen/Qwen3-0.6B",  # HuggingFace model path
    model_id="qwen3-0.6b",  # Model ID exposed by vLLM
    resources=flyte.Resources(
        cpu="4",
        memory="16Gi",
        gpu="L40s:1",  # GPU required for LLM serving
        disk="10Gi",
    ),
    scaling=flyte.app.Scaling(
        replicas=(0, 1),
        scaledown_after=300,  # Scale down after 5 minutes of inactivity
    ),
    # Disable Union's platform-level authentication so you can access the
    # endpoint from the public internet
    requires_auth=False,
    # Inject the secret as an environment variable
    secrets=flyte.Secret(key="AUTH_SECRET", as_env_var="AUTH_SECRET"),
    # Pass the API key to vLLM's --api-key argument
    # The $AUTH_SECRET will be replaced with the actual secret value at runtime
    extra_args=[
        "--api-key", "$AUTH_SECRET",
    ],
)

if __name__ == "__main__":
    flyte.init_from_config(root_dir=pathlib.Path(__file__).parent)
    app = flyte.serve(vllm_app)
    print(f"Deployed vLLM app: {app.url}")

Key points:

requires_auth=False - Disables Union’s platform-level authentication so the endpoint can be accessed from the public internet
secrets=flyte.Secret(key="AUTH_SECRET", as_env_var="AUTH_SECRET") - Injects the secret as an environment variable
extra_args=["--api-key", "$AUTH_SECRET"] - Passes the API key to vLLM’s --api-key argument. The $AUTH_SECRET will be replaced with the actual secret value at runtime

Deploy the app:

python vllm_with_auth.py

Or use the Flyte CLI:

flyte serve vllm_with_auth.py

Deploy SGLang app with authentication

Here’s how to deploy a SGLang app with API key authentication:

sglang_with_auth.py

"""SGLang app with API key authentication."""

import pathlib
from flyteplugins.sglang import SGLangAppEnvironment
import flyte

# The secret must be created using: flyte create secret AUTH_SECRET <your-api-key-value>
sglang_app = SGLangAppEnvironment(
    name="sglang-app-with-auth",
    model_hf_path="Qwen/Qwen3-0.6B",  # HuggingFace model path
    model_id="qwen3-0.6b",  # Model ID exposed by SGLang
    resources=flyte.Resources(
        cpu="4",
        memory="16Gi",
        gpu="L40s:1",  # GPU required for LLM serving
        disk="10Gi",
    ),
    scaling=flyte.app.Scaling(
        replicas=(0, 1),
        scaledown_after=300,  # Scale down after 5 minutes of inactivity
    ),
    # Disable Union's platform-level authentication so you can access the
    # endpoint from the public internet
    requires_auth=False,
    # Inject the secret as an environment variable
    secrets=flyte.Secret(key="AUTH_SECRET", as_env_var="AUTH_SECRET"),
    # Pass the API key to SGLang's --api-key argument
    # The $AUTH_SECRET will be replaced with the actual secret value at runtime
    extra_args=[
        "--api-key", "$AUTH_SECRET",
    ],
)

if __name__ == "__main__":
    flyte.init_from_config(root_dir=pathlib.Path(__file__).parent)
    app = flyte.serve(sglang_app)
    print(f"Deployed SGLang app: {app.url}")

The configuration is similar to vLLM:

requires_auth=False - Disables Union’s platform-level authentication
secrets=flyte.Secret(key="AUTH_SECRET", as_env_var="AUTH_SECRET") - Injects the secret as an environment variable
extra_args=["--api-key", "$AUTH_SECRET"] - Passes the API key to SGLang’s --api-key argument

Deploy the app:

python sglang_with_auth.py

Or use the Flyte CLI:

flyte serve sglang_with_auth.py

Invoke authenticated LLM endpoints

Once deployed, you can invoke the authenticated endpoints using the OpenAI-compatible API format. Both vLLM and SGLang expose OpenAI-compatible endpoints.

For example, to make a chat completion request:

            
        
curl -X POST "https://your-app-url/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <your-api-key-value>" \
  -d '{
    "model": "qwen3-0.6b",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ]
  }'

Replace <your-api-key-value> with the actual API key value you used when creating the secret.

For example, if you created the secret with value my-llm-api-key-12345:

            
        
curl -X POST "https://your-app-url/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-llm-api-key-12345" \
  -d '{
    "model": "qwen3-0.6b",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ]
  }'

You should receive a response with the model’s completion.

The $AUTH_SECRET syntax in extra_args is automatically replaced with the actual secret value at runtime. This ensures the API key is never exposed in your code or configuration files.

Accessing Swagger documentation

The app also includes a public health check endpoint and Swagger UI documentation:

Health check: https://your-app-url/health
Swagger UI: https://your-app-url/docs
ReDoc: https://your-app-url/redoc

The Swagger UI will show an “Authorize” button where you can enter your Bearer token to test authenticated endpoints directly from the browser.

Security best practices

Use strong API keys: Generate cryptographically secure random strings for your API keys
Rotate keys regularly: Periodically rotate your API keys for better security
Scope secrets appropriately: Use project/domain scoping when creating secrets if you want to limit access:
```
flyte create secret --project my-project --domain development API_KEY my-secret-value
```
Never commit secrets: Always use Flyte secrets for API keys, never hardcode them in your code
Use HTTPS: Always use HTTPS in production (Flyte apps are served over HTTPS by default)

Troubleshooting

Authentication failing:

Verify the secret exists: flyte get secret API_KEY
Check that the secret key name matches exactly (case-sensitive)
Ensure you’re using the correct Bearer token value
Verify the as_env_var parameter matches the environment variable name in your code

Secret not found:

Make sure you’ve created the secret before deploying the app
Check the secret scope (organization vs project/domain) matches your app’s project/domain
Verify the secret name matches exactly (should be API_KEY)

App not starting:

Check container logs for errors
Verify all dependencies are installed in the image
Ensure the secret is accessible in the app’s project/domain

LLM app authentication not working:

Verify the secret exists: flyte get secret AUTH_SECRET
Check that $AUTH_SECRET is correctly specified in extra_args (note the $ prefix)
Ensure the secret name matches exactly (case-sensitive) in both the flyte.Secret() call and extra_args
For vLLM, verify the --api-key argument is correctly passed
For SGLang, verify the --api-key argument is correctly passed
Check that requires_auth=False is set to allow public access

Next steps

Learn more about managing secrets in Flyte
See app usage patterns for webhook examples and authentication patterns
Learn about vLLM apps and SGLang apps for serving LLMs