From base model to served adapter. Without losing a run.
From a single-GPU LoRA run to multi-node FSDP across 64 GPUs, Union runs post-training with spot-safe checkpointing, reproducibility by design, and cloud-native scaling.
Trusted by leading AI teams
Open source at the core.
Union is built on Flyte, the open-source AI runtime we create and maintain under the Linux Foundation AI & Data.
companies using Flyte today
Flyte SDK downloads
Recover. Fork. Replay.
A three-day run has three days of value sitting inside it. Losing a spot node at hour sixty means starting over on the base weights and paying for all sixty again, which is why most teams quietly pay on-demand prices. Union records what finished as the run happens, outside the node doing the work, and that record is what you resume from.
- Recover what failed. Point `recover` at a prior run. Finished steps are reused and only what failed or changed runs again, even without caching enabled.
- Checkpoints that survive the node. `flyte.Checkpoint` syncs with HuggingFace Trainer, so a preemption resumes at the step it stopped rather than the epoch it started.
- Fork from any step. `flyte.rerun` starts a prior run again at the step you pick, with a new learning rate or a different preference dataset.
- Built for spot. `interruptible=True` puts multi-day training on spot nodes and cuts compute costs by more than 90%. Losing one is expected here, not an incident.
Except blocks can change the hardware.
Memory is the variable you tune last and discover first. A 7B LoRA run at 4k context fits on one L4, and the same config at 32k with a larger batch does not. Union hands you a typed error instead of a stack trace, so the run that blows memory comes back as an exception you can catch and re-run on a bigger accelerator.
- Typed infrastructure errors. `OOMError`, `TaskInterruptedError`, `TaskTimeoutError`, `ImagePullBackOffError`. Failures you can branch on.
- Resources changed at runtime. `.override(resources=...)` re-runs the same config with more memory, or moves it from an A100 to an H200.
- Mix accelerators across stages. Tokenize on CPU, train on H100s, evaluate on an L4. Each task declares what it needs instead of the pipeline sizing for its largest step.
- The handler does not have to be human. Provisioning is an ordinary Python call, so the same API you write a retry policy against is one an agent can call while a run is in flight.
Fan out on durable asyncio.
One fine-tune is a guess. Post-training is really a sweep: ranks, learning rates, and preference datasets crossed against each other, and running them one at a time is how a week disappears. Fanning out over the grid is plain async, and the cluster scales back to zero when the sweep is done.
- Concurrency you can cap. `flyte.map.aio(fn, configs, concurrency=32)` bounds how many train at once, so a wide sweep does not sit behind your whole GPU quota.
- Tokenize once, reuse everywhere. `cache="auto"` hashes inputs and code, so every arm of the sweep shares one tokenized dataset instead of rebuilding it.
- Distributed training when one GPU is not enough. The Elastic plugin spans nodes for FSDP and DeepSpeed, and workers rejoin after a failure instead of sinking the run.
Outputs that outlive the run.
The gap between a finished checkpoint and a served model is usually a manual copy. Artifacts are typed, versioned values that persist past the run that made them, so a merged adapter reaches a vLLM endpoint without anyone moving weights between buckets, and a promoted checkpoint can roll the deployment by itself.
- Passed between workflows and apps. A training run's adapter is an eval workflow's typed input and a serving app's weights, without re-running the producer.
- Versioned, not overwritten. A new version is a new artifact, so a rollback names the exact checkpoint that was live before.
- Events, not polling. `flyte.OnArtifact` fires an eval or a redeploy whenever a new checkpoint is promoted, with no cron hacks in between.
Reproduce a run from months ago.
A checkpoint on its own cannot tell you what made it. Every run keeps what it takes to reproduce a result: the code that executed, the infrastructure it ran on, and the configuration applied to both. That is what makes a training pipeline shareable rather than personal, and it is the same record a model card or a regulator asks for.
- Inputs and outputs per run. Open a training run from months ago and see the exact dataset version, hyperparameters, and base model behind a checkpoint.
- The code that ran, not the code today. Each run resolves to its own code bundle and container image, down to the transformers and CUDA versions.
- Full lineage back to the data. Every adapter traces to the tokenized dataset and the base weights it started from.
Post-training in pure Python.
Tokenize, sweep, checkpoint, and promote, no YAML, no DSLs. Write it in Python, run it across nodes on Union.
Model Training on Union
Case studies and technical deep dives from teams iterating on models at scale.
Start today and scale with confidence.
Talk to our team about how Union runs fine-tuning, alignment, and evaluation for teams training models on their own data.