Open source at the core.
Union is built on Flyte, the open-source AI runtime we create and maintain under the Linux Foundation AI & Data.
companies using Flyte today
Flyte SDK downloads
Fan out on durable asyncio.
Vision work is one small task repeated a hundred million times. A week of fleet footage decodes into more frames than your cluster has cores, and fanning out over them is plain async, with the cluster scaling back to zero when the run is done.
- Concurrency you can cap. `flyte.map.aio(fn, frames, concurrency=500)` bounds how many run at once, for a fan-out wider than the cluster.
- Retries with pacing. `retries=5`, or a `RetryStrategy` with exponential backoff so a throttled object store gets time to come back.
- Built for spot. `interruptible=True` puts decode and inference on spot nodes and cuts compute costs by more than 90%. Losing one frame is expected here, not an incident.

Except blocks can change the hardware.
Typed infrastructure errors. `OOMError`, `TaskInterruptedError`, `TaskTimeoutError`, `ImagePullBackOffError`. Failures you can branch on.
Resources changed at runtime. `.override(resources=...)` re-runs the same batch with different memory, or moves it from an L4 to an A100.
The policy is yours. Write the handler once and every vision pipeline in your org can import it.
The handler does not have to be human. Provisioning is an ordinary Python call, so the same API you write a retry policy against is one an agent can call while a run is in flight.

Recover. Fork. Replay.
Recover what failed. Point `recover` at a prior run. Finished frames are reused and only what failed or changed runs again, even without caching enabled.
Fork from any step. `flyte.rerun` starts a prior run again at the step you pick, with a new backbone or a different augmentation policy.
Replay with today's code. `--rerun-from` runs a prior run's inputs against the code on your machine right now.

Outputs that outlive the run.
Passed between workflows and apps. One pipeline's curated dataset is another's typed input, without re-running the producer.
Versioned, not overwritten. A new version is a new artifact, so an eval run resolves to the exact weights it scored.
Events, not polling. `flyte.OnArtifact` fires a run whenever a new annotation batch lands, with no cron hacks in between.

Reproduce a run from months ago.
Inputs and outputs per run. Open a training run from months ago and see exactly which frames went in and which weights came out.
The code that ran, not the code today. Each run resolves to its own code bundle and container image, down to the CUDA and OpenCV versions.
Full lineage back to source footage. Every intermediate and final output traces to the raw images that produced it.

Vision pipelines in pure Python.
Decoding, detection, and evaluation, no YAML, no DSLs. Write it in Python, run it at scale on Union, with automatic retries, checkpointing, and recovery built in.
Running where a bad batch is expensive.
Autonomous driving, medical imaging, and satellite analysis at catalog scale. Long runs on infrastructure large enough to fail constantly.
See a training run survive a node failure.
Bring a pipeline you've lost data to before, and watch what happens when the machine under it goes away.







