OpenAI

A Flyte task or app can call the OpenAI API with the OpenAI SDK and a key from your own account.

Setting	Value
Base URL	SDK default
API key	API keys
Environment variable	`OPENAI_API_KEY`

Store your OpenAI API key

Run this command and paste the key at the prompt:

        
    
flyte create secret openai-api-key --project <your-project> --domain development

Call OpenAI from a task

call_openai.py
                
import os

import flyte

env = flyte.TaskEnvironment(
    name="openai-inference",
    image=flyte.Image.from_debian_base().with_pip_packages("openai"),
    secrets=flyte.Secret(key="openai-api-key", as_env_var="OPENAI_API_KEY"),
)

@env.task
async def summarize(text: str) -> str:
    from openai import AsyncOpenAI

    client = AsyncOpenAI(api_key=os.environ["OPENAI_API_KEY"])
    response = await client.chat.completions.create(
        model="gpt-6-astra",
        messages=[{"role": "user", "content": f"Summarize in two sentences:\n\n{text}"}],
    )
    return response.choices[0].message.content

if __name__ == "__main__":
    flyte.init_from_config()
    run = flyte.run(summarize, text="First document...")
    print(run.url)

Call OpenAI from an app

An AppEnvironment takes the same secrets argument, and the app reads the same environment variable. See Call the provider from an app.

Route calls through the LLM Gateway

To add budgets, fallbacks, and guardrails, add OpenAI to the LLM Gateway using the same secret, then point your client at the gateway. See Choose direct calls or the gateway.