Flyte 2 Is Generally Available: The Durable, Open-Source AI Runtime

Today, Flyte 2 is generally available. It is faster, more scalable, and (my personal guarantee) genuinely fun to use. It remains open source and part of the Linux Foundation. Before I tell you what's in it, I want to tell you why it exists, because Flyte 2 is not an incremental release. It is a ground up rebuild of a system we have been refining for nearly a decade, rebuilt for a world that looks nothing like the one Flyte was born into.

This new evolution of Flyte has turned it into something more valuable than an orchestrator for AI and ML teams: a durable AI runtime. Where an orchestrator sits on top of systems to execute and forget, a runtime owns the infrastructure underneath your code, not just the code itself. Where a durable execution system can recover from logic failures, a durable runtime can use its infrastructure ownership to recover from compute failures (e.g., OOM), too, for end-to-end runtime durability. That is what we mean when we call Flyte 2 a durable AI runtime instead of an orchestrator, and it is the thread running through everything below.

Where we started

Flyte began at Lyft. We built it to prepare data, train models, run large scale simulations, and back test models: the unglamorous, essential machinery behind products like ETA and pricing. What we realized then still holds today: building ML products requires the reliability of user facing software, the dynamism of infrastructure that reshapes itself around each workload, and the agility to experiment constantly. No system at the time offered all three, so we built one.

Consider the world we built it in. Python was widely dismissed as a single threaded scripting language, type hints were rare, async/await barely existed. Spinning up a container per task was considered taboo: CUDA driver conflicts and dependency management made containerized ML painful, and rootless builds were an afterthought. Kubernetes barely worked; we ran on v1.10 with half its features experimental and no managed offerings to lean on. Apache Arrow had just gotten started. Running a cluster of a few hundred machines was nuts. Against that backdrop, we made bets, container native execution, strong typing, reproducibility by default, that felt contrarian then and feel obvious now. We even lived through the alternatives first: Flyte v0.1 was built on Airflow, v0.2 on AWS Step Functions, before we bet on Kubernetes in 2018. (For more about our story, read last year’s announcement)

What Flyte 1 taught us

We designed Flyte 1 to make sure users could orchestrate scalable workflows without shooting themselves in the foot. Static workflows, compiled and type checked before they ran, so correctness was an inherent property of the system. We thought of it as a workflow operating system: it should abstract complexity, scale with hardware, isolate tenants safely, and reproduce results every single time. That discipline is why Flyte earned trust for mission critical workloads, and it's the same discipline that makes Flyte 2 durable today.

But there was a cost to prioritizing scale and safety: Flyte became too hard to learn. Promises, dynamic workflows, our own conditionals: conceptually heavy abstractions users had to internalize before writing their first real pipeline. Better than YAML, sure, but that's a low bar. And our users kept pushing us, respectfully, relentlessly, toward dynamism.

We have been extraordinarily lucky in who those users are. We got to work with some of the best engineering organizations in the world: LinkedIn, Spotify, Stripe, Tesla, Expedia, Toyota, Mercedes-Benz, Amazon, NVIDIA, Apple, and a new generation of AI native companies like OpenAI, Runway, Luma Labs, Mistral, Wayve, Applied Intuition, and many more. 

Working alongside them, we learned. We watched new workloads emerge while the foundational pieces (data, models, compute, coordinated reliably) stayed intact. And we became more convinced than ever that the AI world needs one common, open layer of execution embedded within users’ infrastructure.

By 2024, the ecosystem had matured in exactly the directions we'd bet on. Async Python had serious momentum, with free threaded Python on the horizon. Container builds, CUDA, and containerized GPU workloads were not just acceptable but expected. Kubernetes was more stable than it had ever been, with managed clusters scaling to thousands of nodes. And a new type of workload was emerging: one that needed durability across a fully dynamic control flow and fully dynamic infrastructure, programs whose shape is decided at runtime, sometimes by a human, increasingly by a model. 

As one AI lab put it to us: “You can't hardcode a fixed path for an inherently dynamic, path-dependent process.”

So in early 2025, we made the hard call. We set out to build a whole new Flyte, from the ground up. Flyte 2.

The Flyte 2 Vision

We held ourselves to three goals:

Make learning easy, as close to nonexistent as possible. If you can write (or generate) Python, you can write Flyte. No DSL, no promises, no workflow language, no mental model tax. Tasks simply call tasks. Loops, conditionals, try/except: you write the logic, Flyte orchestrates it. Flyte should feel invisible. 

Build a foundation that is durable at the infrastructure level, not just the code level. The systems being built today are authored, invoked, debugged, and even repaired by agents, and most of what actually breaks a long running AI workload isn't a bug, it's an infrastructure failure: an OOM crash, a preempted node, a GPU that disappears mid-run. We needed Flyte 2 to have the power to recover from these failures, not just retry code against the same infrastructure assumptions.

Unify real time and batch. Whether something runs as a long-lived service or a scheduled pipeline should be a property of the problem, not an accident of the infrastructure you happened to choose. One programming model, one packaging model, both modalities.

The first open-source AI runtime

The result is Flyte 2: the first durable AI runtime for open source. 

Here's what that means in practice. This is a complete Flyte 2 program:

import asyncio
import flyte

env = flyte.TaskEnvironment(
    name="hello_world",
    image=flyte.Image.from_debian_base(python_version=(3, 12)),
)

@env.task
def calculate(x: int) -> int:
    return x * 2 + 5

@env.task
async def main(numbers: list[int]) -> float:
    results = await asyncio.gather(*[calculate.aio(n) for n in numbers])
    return sum(results) / len(results)

if __name__ == "__main__":
    flyte.init()
    run = flyte.run(main, numbers=list(range(10)))
    print(run.result)

Pure Python. No DSL. No workflow objects. Tasks call tasks; fanout is just asyncio.gather. And this exact program runs on your laptop, on a devbox, or across a cluster, each task in its own container, every input and output typed, versioned, and durable, every run recoverable from where it stopped. Your program doesn't get handed off to some other execution layer; Flyte is the execution layer. That's what runtime means.

Durable across code and infra

Let’s explore how Flyte 2 makes workflows durable from both code and infrastructure failures.

Flyte 2 is fundamentally different from your average durable execution platform because it is infra-aware. By letting you affect your compute resources at runtime, Flyte 2 can solve an entire world of failures that others can’t – the ones caused by infrastructure.

Durability comes from a replay log and traces that bring determinism even to non-deterministic, LLM-driven processes: crash, recover, and continue exactly where you left off. That part, replaying logic after a crash, is table stakes for any durable execution engine. 

What isn't table stakes is what happens when the crash comes from the infrastructure, not the code. An OOM kill, a preempted spot instance, a GPU that vanishes mid-training run: Flyte 2 recognizes the failure mode anchanges the resources it asks for on the retry, not just the code path it replays. That's the difference between a system that's durable against your bugs and one that's durable against the infrastructure your bugs run on. AI and ML workloads hit the second kind constantly.

Every execution is logged and reproducible from a single UI, so a failure is something you can inspect and replay, not something you re-run blind and hope goes better. It is still language agnostic at its core, but extremely simple to use in Python, and where performance mattered, we rewrote the hot paths in Rust: the engine is many times faster than Flyte 1, data movement holds its own against dedicated tools like s5cmd, and a single process can drive thousands of concurrent tasks. Everything feels instant.

Here's what agent native means in practice: your agent's workflow can literally be a while loop.

gpu = flyte.TaskEnvironment(name="worker", resources=flyte.Resources(gpu=1))

@flyte.trace
async def plan(history: list[str]) -> str:
    ...  # your LLM call, traced, so a crash resumes here, not from zero

@gpu.task
async def act(step: str) -> str:
    ...  # heavy lifting, on a GPU provisioned just for this step

@env.task
async def agent(goal: str) -> list[str]:
    history = [goal]
    while (step := await plan(history)) != "done":
        history.append(await act(step))
    return history

Dynamic control flow, durable non-deterministic steps, and a GPU that materializes mid-loop and disappears after: no agent framework required, and any framework you already use runs happily inside. 

Beyond the loop itself, Flyte 2 ships with orchestration sandboxes so LLM-generated pipelines run safely, isolated and network blocked by default, because if models are going to write workflows, the runtime must make that safe by construction. It ships human-in-the-loop as a first class construct, so a workflow can pause for a person's judgment and resume the moment it arrives. It speaks MCP, so your coding agent or IDE can operate Flyte 2 directly: launch runs, inspect state, fix failures.

Realtime + batch inference without a separate serving stack

Because we kept seeing teams stitch a separate inference stack next to their orchestrator, we included inference and serving in the same model. A long running app is just another environment:

from fastapi import FastAPI
import flyte
from flyte.app.extras import FastAPIAppEnvironment

app = FastAPI()
serving = FastAPIAppEnvironment(name="my-model", app=app)

@app.get("/predict")
async def predict(x: float) -> dict:
    return {"result": await score.aio(x)}  # calls the same task your pipeline uses

Same programming model, same packaging, same platform, whether the code runs as a scheduled pipeline, an agent loop, or a real time endpoint. The modality is a choice of the problem, not the infrastructure. Train, evaluate, serve, and orchestrate agents with one set of primitives.

Your data is your moat. Keep it.

There's one more belief embedded deep in Flyte's design, and it matters more in the AI era than ever before.

In this world, your data, your context, and your processes are your moat. The teams that win won't just consume intelligence, they'll build their own: models tuned on their data, agents shaped by their processes, systems that encode how they operate. Building proprietary intelligence means you need to own the foundations it stands on, and you need a platform flexible enough to evolve with ideas you haven't had yet.

Flyte is built for exactly that. It runs where your data lives: your laptop, your VPC, your cluster. Your code, your data, your secrets stay with you, on an open foundation that no vendor can take away or reshape underneath you. Own your foundations; keep your moat.

The best of Flyte 1 is here to stay

We rebuilt the foundation, but the values stay the same. Flyte 2 is still flexible, type safe, observable, durable, and reproducible. Every run is still versioned. You can still launch from the UI, the API, the CLI, or programmatically. It's pure Python, yet data aware: Flyte understands the data flowing between your tasks, typed, versioned, automatically offloaded and cached, so you get reproducibility and lineage without ever managing a bucket path. Everything you love about Flyte is here, and more, but simpler. Through all of it, Flyte's goal has not changed since the Lyft days: reduce the time it takes to try a new idea, and make everything durable, observable, and reproducible.

Generally available today. Let’s build.

We announced Flyte 2 last year, and the response has been beyond anything we hoped for: 1.3 million downloads already, before GA. To everyone who filed issues, tested betas, pushed on our APIs, and trusted us with production workloads: thank you. This monumental milestone would not exist without your trust.

Today, Flyte 2 reaches full general availability: an open, durable AI runtime that autoscales, all you need is a Kubernetes cluster. It spans the entire spectrum:

• Fully local: `pip install flyte` and run durable, observable workflows on your laptop. No cluster, no backend, nothing else required.
• Devbox: `flyte start devbox` gives you a durable single machine setup with the complete backend, UI, and storage in one command.
• Kubernetes: starting today, deploy to a k8s cluster in one line, or let your favorite agent do it for you.

Same code, same programming model, at every scale, and it's durable at every one of them.

Migrating from Flyte 1

We’re working hard to make migration as simple as possible for the thousands of teams relying on Flyte 1 today. 

• Free Migration Consultations (first come first serve): For Flyte 1 users and teams considering migrating from legacy orchestrators, we’re excited to offer for a limited time brief 1:1 sessions with our engineering team.
• Migration Hub: We’ve published a guide with valuable instructions and agents to make the Flyte 1 to Flyte 2 migration process smooth. This page includes information about live office hours and Flyte 1 support sunsetting.

What happens next

Our ambition is enormous: make Flyte the default AI runtime everywhere, from the largest enterprises to the smallest startups, across every type of AI workload, from data pipelines to training to agents to inference. Open, and owned by its community under the Linux Foundation.

And about that guarantee from the first paragraph: `pip install flyte`, run the snippet above, and watch your first run stream into the terminal. If you're not having fun by then, open an issue and tell me why, I read every one.

Nearly a decade in, it still feels like the beginning. 

Try Flyte 2 today at flyte.org. 

Join the community. 

Let's go further together.

— Ketan