Billions of routes, prices, & shipments. In pure Python.

Union is built on Flyte, the open-source AI runtime we create and maintain under the Linux Foundation AI & Data.

Pricing and routing pipelines run on a clock. A carrier feed lands late or malformed at two in the morning and the run dies four hours deep, and starting from the top means missing the window entirely. Union records what finished as the run happens, outside the node doing the work, and that record is what you resume from.

• Recover what failed. Point `recover` at a prior run. A late feed for one lane reruns that lane, not the night, even without caching enabled.
• Fork from any step. `flyte.rerun` starts a prior run again at the step you pick, with a corrected feed or a new pricing model.
• Replay with today's code. `--rerun-from` runs a prior run's inputs against the code on your machine right now.

Partition sizes in logistics are never even. One lane carries a hundred times the volume of the median, and a memory limit sized for the median dies on it while a limit sized for the peak overpays on every other partition. Union hands you a typed error instead of a stack trace, so the partition that blows memory comes back as an exception you can catch and re-run on a bigger box.

• Typed infrastructure errors. `OOMError`,` TaskInterruptedError`, `TaskTimeoutError`, `ImagePullBackOffError`. Failures you can branch on.
• Resources changed at runtime. `.override(resources=...)` re-runs the same partition with different memory, CPU, or accelerator.
• Right machine per step. Cheap wide workers for ingest, a high-memory solver node for optimization, decided in code rather than in a cluster config.
• The handler does not have to be human. Provisioning is an ordinary Python call, so the same API you write a retry policy against is one an agent can call while a run is in flight.

Hopper ran 22,300 tasks with 7,000 executing concurrently to visualize 4.4 billion trip records. Fanning out over lanes, markets, or dates is plain async, with no cluster semantics to reason about and no framework to fight.

• Concurrency you can cap. `flyte.map.aio(fn, lanes, concurrency=500)` bounds how many run at once, for a fan-out wider than the cluster.
• Retries with pacing. `retries=5`, or a `RetryStrategy` with exponential backoff so a flaky carrier API gets time to come back.
• Built for spot. `interruptible=True` puts branches on spot nodes, and container reuse keeps startup cost off thousands of short-lived tasks.

A demand forecast should not be a warehouse table whose name one person remembers. Artifacts are typed, versioned values that persist past the run that made them, so a routing workflow consumes this morning's price surface without re-running the model that produced it.

• Passed between workflows and apps. One pipeline's forecast is another's typed input. `flyte.io.DataFrame` moves a pointer to object storage, not the rows.
• Versioned, not overwritten. A new version is a new artifact, so a downstream run resolves to the exact forecast it priced against.
• Events, not polling. `flyte.OnArtifact` fires a run whenever a new forecast version lands, with no cron hacks in between.

Someone is asking which forecast actually priced the network on peak weekend, and what it ran against. Every run keeps what it takes to reproduce a result: the code that executed, the infrastructure it ran on, and the configuration applied to both. That is what makes a pipeline shareable rather than personal, and it is the same record an audit or a postmortem needs.

• Inputs and outputs per run. Open an execution from months ago and see exactly which feeds went in and which prices came out.
• The code that ran, not the code today. Each run resolves to its own code bundle and container image, down to the solver version.
• Full lineage for every forecast you shipped. Versioned workflows, containers, and artifacts, traced back to the partitions that produced them.