Durable Execution for Any AI Agent Framework

Flyte now runs agents built with the OpenAI Agents SDK, Claude Agent SDK, LangGraph, LangChain, Deep Agents, CrewAI, Pydantic AI, Google ADK, Mistral Agents and Hermes.

You keep the framework you already use: its agent loop, handoffs, tool calling and conventions. Flyte runs underneath it, recording model turns and tool calls so a run that dies halfway through can resume instead of starting the conversation over.

Agent retries are expensive in a slightly stupid way. Say an agent gets 30 turns into a research task and the worker dies. Retrying the task starts the agent from the top: those 30 model calls happen again, the tools it already called may run again, and you pay for all of it again. Because the model is non-deterministic, the second run isn’t guaranteed to retrace the first one either.

We ran into this while trying to run existing agent frameworks on Flyte. A normal Flyte task is straightforward to retry because its inputs and outputs give us a durable boundary. An agent loop is different: most of the useful state is sitting inside a long-running process, spread across model responses, tool calls and framework-specific objects. When that process disappears, starting the loop again is closer to a do-over than a retry.

We didn’t want to solve that by replacing the loop. If you already have an agent in the OpenAI Agents SDK, LangGraph, CrewAI or Pydantic AI, rewriting it in workflow primitives just to make it durable isn’t much of a solution. So we looked for the smallest boundary Flyte could own without changing how the framework works. In most cases, it came down to two places: the model call and the tool call.

The framework still owns the loop. Flyte makes the work underneath it durable. Completed model turns can replay without calling the model again, completed tools can come back from cache, and a retry moves through the work that already finished until it reaches the part that didn’t.

Across all ten adapters, we stuck to one rule: the framework owns the loop. Flyte doesn’t reimplement tool calling, turn management, handoffs or guardrails. Apart from being a maintenance headache, doing that would change the behavior you chose the framework for in the first place.

Instead, `run_agent` starts the framework’s own runner and wraps the boundaries we need for durability. The SDK still decides when to call the model, which tools to use and where to hand off. Flyte records that work and handles what happens when it fails.

The agent stays the same. It just runs on a runtime built to survive failures.

The interface

Every adapter presents the same basic surface, so an agent looks like this no matter which framework you’re using:

import flyte
from flyteplugins.agents.openai import tool, run_agent  # same shape for every adapter

env = flyte.TaskEnvironment(
    "agent",
    secrets=[flyte.Secret(key="openai_api_key", as_env_var="OPENAI_API_KEY")],
)

@tool
@env.task(cache="auto", retries=3)
async def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is sunny, 22°C."

@env.task(report=True, retries=3)
async def city_agent(question: str) -> str:
    return await run_agent(
        question,
        tools=[get_weather],
        model="gpt-4.1",
        memory_key="user-1",
    )

The integration mostly comes down to two pieces. `@tool` sits on top of `@env.task`, so the same Python function is both a regular Flyte task and a tool your framework can call.

`run_agent` does the other half: it runs the framework’s agent loop from inside a Flyte task.

From there, the agent maps naturally onto three layers of Flyte’s execution model:

If you haven’t used Flyte before, the important bit is that it keeps a durable record of what ran: inputs, outputs, failures and attempts for every action. That’s particularly useful for agents because their control flow is decided at runtime. You don’t know ahead of time whether the model will call search three times, hand off to another agent, run some code, or take a completely different path. Flyte records that graph as it happens, which is what lets you inspect the path afterward and reuse completed work when a run dies.

Recording a model turn

Making an individual model turn durable without taking over the framework’s loop was the interesting part. A model call isn’t a neat, serializable function call. Its arguments are live SDK objects: message lists containing provider-specific types, tool definitions holding function references, clients with open connections. You can’t persist those directly, and without a stable identifier there’s nothing to match against when the run is retried.

The adapters solve this by putting the real SDK call inside a closure and giving the durability layer two things: a fingerprint that identifies the request, and a function that knows how to execute it. The fingerprint is a SHA-256 hash over a canonical serialization of the parts of the request that affect the model’s answer: messages, model name, instructions and sorted tool names. Incidental state such as trace IDs, live clients and function handles stays out.

The closure makes the actual SDK call and serializes the response to JSON using the SDK’s own types, so it can round-trip faithfully while remaining readable in Flyte. `flyte.trace` ties the two together: the fingerprint becomes the basis for a deterministic action ID, and the response is stored with the run.

On retry, Flyte starts the task from the top, so the framework simply runs its normal loop again. When it reaches a model turn that already completed, the adapter produces the same fingerprint, Flyte finds the recorded response, and the model client is never touched. Completed tool calls come back as cache hits in the same way. The task is technically running again from the beginning, but the expensive work isn’t; execution moves through the completed steps and picks up at the first one that never finished.

Failures and non-determinism

Failures need slightly different treatment. Successful turns replay, and terminal failures replay too: there’s no reason to spend money reproducing an error we already know is permanent. Recoverable failures are the exception. A temporary network failure or a worker dying mid-turn should run again because replaying a stale transient error would poison every subsequent retry. So successful turns and terminal failures replay; recoverable failures re-run.

This also makes non-determinism much less painful than it is in traditional workflow replay. Those systems generally expect deterministic code, and a replay that takes a different branch becomes a determinism violation. Here, if a retried agent reaches a turn whose semantic request matches a completed turn, it gets the recorded response. If the request has changed, the fingerprint changes too, so there’s simply nothing to replay. It’s a cache miss: Flyte makes the model call, records the new result, and keeps going. You may pay for one additional model turn, but you haven’t corrupted the replay.

The crash-resume example makes the effect concrete. On the first attempt, the agent completes two model turns and two tool calls before we kill the worker. `get_weather` takes 5.1 seconds, `get_population` takes 16.6 seconds, and the agent timeline reaches 20.3 seconds before the crash. On retry there are zero live model calls. Both turns come back from their recorded results, the two tools resolve as cache hits in 59 ms and 102 ms, and the retry finishes in 0.44 seconds.

From the framework’s point of view, it ran its normal loop from the beginning. From the runtime’s point of view, almost none of the work happened twice.

Where each framework is wrapped

The idea is the same across frameworks. Every SDK routes model calls differently, and finding the seam where we can add durability without taking over the loop is most of the adapter work.
‍

Most frameworks follow the fingerprint-and-replay approach above. The two exceptions are: the Claude Agent SDK runs its loop in a subprocess Flyte can’t wrap, so durability there is the SDK’s own session resume backed by a Flyte checkpoint, and Hermes exposes no hook at all, so `durable=` is a documented no-op. The how-it-works page goes into more detail on both.

One thing is consistent across every framework though: tool calls are always durable. Tools don’t depend on us finding a way into the model loop. Once the framework calls a function decorated with `@tool`, the work crosses a boundary Flyte controls. And that boundary gives us more than just replay.

Tools are infrastructure

In a normal agent process, a tool is a function call. It runs on the same machine as the agent, under the same memory limit and in the same failure domain. If the tool OOMs, it can take the conversation with it.

Here, a tool is a Flyte task, so each call is a child action with its own container image, resources, retries and cache entry. If one tool needs 2 CPUs, another needs an A100, and a third needs a different set of Python dependencies, each declares that on its own `TaskEnvironment`. The model loop can stay on a small, cheap container while the heavy work runs somewhere else.

heavy = flyte.TaskEnvironment(
    "diagnostics",
    resources=flyte.Resources(cpu=2),
)

@tool
@heavy.task(retries=3)
async def run_diagnostic(service: str) -> str:
    """Run a diagnostic on a service. Runs in its own container, with its own resources."""
    ...

That changes the failure boundary too. If `run_diagnostic` crashes, the whole agent process doesn’t have to go with it. The framework gets a tool error and decides what to do next, which is exactly the kind of failure agent SDKs already know how to handle. Mark a tool with `cache="auto"` and identical calls can also reuse the previous result, both across retries and across separate runs.

More interestingly, a tool call is a durable suspension point. That lets you put things behind a tool that most agent SDKs don’t have an equivalent for, such as a human approval:

@tool
@env.task(retries=3)
async def issue_refund(account_id: str, amount_usd: float) -> str:
    """Issue a refund. Pauses for human approval before it runs."""
    condition = await flyte.new_condition.aio(
        f"approve_refund_{account_id}",
        prompt=f"Approve a ${amount_usd:.2f} refund to account {account_id}?",
        data_type=bool,
    )
    if not await condition.wait.aio():
        return f"Refund to {account_id} was declined by a human reviewer."
    return f"Refunded ${amount_usd:.2f} to account {account_id}."

The model decides whether to call `issue_refund`; Flyte controls what happens when it does. The run genuinely suspends while it waits instead of holding a thread open, survives restarts while pending, and resumes when a human responds. The same mechanism works for an hour-long wait or a week-long escalation window without consuming compute while nothing is happening.

The model decides which tool to call. Flyte handles what it takes to run it.

What replay means for your bill

Durable agents have a cost dimension that ordinary data pipelines usually don’t: repeated work can mean another paid API call. We wanted that to be visible rather than buried in logs.

Replayed turns never touch the model client, so they don’t incur another model charge. The live report from `report=True` reflects that: on a retry, turns restored from durable records appear as cached tokens rather than fresh spend, while tool cache hits show their original result with the much smaller replay time. The Claude adapter can go further because the SDK exposes its own cost information, so the report puts its dollar estimate next to the input, output, cache-read and cache-write token counts behind it.

Replay isn’t just about making a retry faster. Once an agent has been running for tens of minutes or hours, avoiding those repeated calls can be the more important part.

Memory that outlives the run

Replay handles one timescale: an agent run dies and needs to continue. `memory_key` handles another: the same agent gets called again tomorrow, on another worker, in a completely different run.

Pass `memory_key="user-1"` to `run_agent` and the conversation state is stored in a keyed, blob-backed memory store. Its path comes from the key rather than the run ID, so separate runs using the same key share the same state. Each adapter maps its framework’s own notion of memory onto that store rather than forcing everything into one abstraction.

For OpenAI, that means implementing the SDK’s `Session` protocol over the store instead of relying on local SQLite sessions that don’t survive a distributed backend. Mistral already keeps conversations server-side, so we persist the conversation ID and reconnect to the same thread. Deep Agents also restores the agent’s virtual filesystem, so a returning agent gets its working files back alongside the transcript. It can continue the work rather than merely remember that it was doing it.

Agents compose like tasks

Because an agent run is a task, multi-agent systems get two layers of orchestration that don’t compete. The framework keeps the orchestration that belongs inside the agent: OpenAI handoffs, ADK sub-agent transfer, Claude subagents and Mistral handoffs all work unchanged because we never replace the loop that implements them.

Flyte handles the layer outside the agent. A planner task can fan out research agents across separate workers and pass their outputs to a synthesizer. That’s actual distributed parallelism rather than several coroutines sharing one process. Each agent in the fan-out is a first-class node with its own retries, cache, resources and place in the run graph. Triggers can schedule them, conditions can gate them, and ordinary tasks can sit between them.

The SDK handles the micro-orchestration. Flyte handles the macro-orchestration.

It runs on your laptop first

You don’t need a backend just to write or test an agent. Outside a Flyte task context, the integration gets out of the way. The same file therefore works locally with:

flyte run --local

and on a backend with:

flyte run

without changing the agent code.

There is one caveat: a local run tells you whether the agent works, but not whether replay works because there’s no durable record locally. To watch an agent crash halfway through and then skip its completed model turns and tool calls on retry, you need a backend run.

Where this fits

Flyte already has a native Agent abstraction: a batteries-included loop with tools, MCP servers, memory and approval gates. If we were starting fresh and wanted the runtime and the loop from one place, that’s still what we’d reach for.

These adapters are for the other case, which in practice is most of the teams we talk to: you already have an agent. It’s written in LangGraph, CrewAI, the OpenAI Agents SDK or another framework, and it contains real product logic that people have spent time getting right. Rewriting its prompts, tools, handoffs and edge cases just to gain durability isn’t a serious migration plan.

So the plugins don’t ask you to. Use the framework you would have used anyway. Keep its loop and its semantics. Add Flyte underneath when you need the run to survive failures, move expensive tools onto different hardware, persist memory, fan agents out across workers, or understand what happened afterward.

The frameworks decide what the agent does next. Flyte decides where each step runs, how it’s sized, what happens when it fails, and what evidence remains afterward. What surprised us was how little we actually needed to change. By wrapping model calls and tool calls, we could make ten very different frameworks work with the same runtime without taking over their agent loops.

To try it with the OpenAI Agents SDK:

pip install flyteplugins-agents-openai

The agent frameworks documentation covers all ten adapters, the capability matrix and per-framework examples. If your framework isn’t on the list, the bring-your-own-framework guide walks through the same template the adapters themselves follow.