Infrastructure for AI, ML & Data
For developers managing AI, ML, and data workflows in production, the challenges extend well beyond scheduling and orchestrating DAGs. Union.ai addresses these complexities by offering a comprehensive infrastructure management platform designed for the nuances of such environments.
Union optimizes resources across teams and implements cost-effective strategies that can reduce expenses by up to 66%. Moreover, it’s engineered to fit within your own cloud ecosystem, ensuring a robust and tailored infrastructure that scales with your technical demands.
Powerful DAGs, observability & cost-efficient engineering
Union is a fully-managed Flyte platform deployed in your VPC that provides a single-endpoint workflow orchestration and compute service to engineers building data and ML products.
Get built-in dashboards, live-logging, and task-level resource monitoring, enabling users to identify resource bottlenecks and simplifying the debugging process, resulting in optimized infrastructure and faster experimentation.
AI engineering for engineers
Union is an open AI orchestration platform that simplifies AI infrastructure so you can develop, deploy, and innovate faster. Unlike popular—but simple—AI engineering orchestrators, Union wrangles the infrastructure setup and management as well.
Write your code in Python, collaborate across departments, and enjoy full reproducibility and auditability. Union lets you focus on what matters.
Purpose-built for lineage-aware pipeline orchestration
Bring your own Airflow code (BYOAC) and take advantage of modern AI orchestration features—out of the box! Get full reproducibility, audibility, experiment tracking, cross-team task sharing, compile-time error checking, and automatic artifact capture.
Easily experiment and iterate in isolation with versioned tasks and workflows.
A centralized infrastructure for your team and organization, enables multiple users to share the same platform while maintaining their own distinct data and configurations.
Strongly typed inputs and outputs can simplify data validation and highlight incompatibilities between tasks making it easier to identify and troubleshoot errors before launching the workflow.
Caching the output of task executions can accelerate subsequent executions and prevent wasted resources.
As a data-aware platform, it can simplify rollbacks and error tracking.
Immutable executions help ensure reproducibility by preventing any changes to the state of an execution.
Rerun only failed tasks in a workflow to save time, resources, and more easily debug.
Enable human intervention to supervise, tune and test workflows - resulting in improved accuracy and safety.
Globally trusted & tested
Join our developer community
“Flyte has this concept of immutable transformation — it turns out the executions cannot be deleted, and so having immutable transformation is a really nice abstraction for our data-engineering stack.”
“With Flyte™, we want to give the power back to biologists. We want to stand up something that they can play around with different parameters for their models because not every … parameter is fixed. We want to make sure we are giving them the power to run the analyses.”
“FlyteFile is a really nice abstraction on a distributed platform. [I can say,] ‘I need this file,’ and Flyte™ takes care of downloading it, uploading it and only accessing it when we need to. We generate large binary files in netcdf format, so not having to worry about transferring and copying those files has been really nice.”
“You can say, ‘Give me imputation’ and [Flyte™ will] launch 40 spot instances that are cheaper than your on-demand instance that you're using for your notebook and return the results back in memory.”
“One thing that I really like compared to my previous experience with some of these tools: the local dev experience with pyflyte and the sandbox are super, super nice to reduce friction between production and dev environment.”
“Workflow versioning is quite important: When it comes to productionizing a pipeline, there are only a few platforms that provide this kind of versioning. To us, it’s critical to be able to roll back to a certain workflow version in case there is a bug introduced into our production pipeline.”
“To my great surprise, the migration to Flyte™ was as smooth and easy as the development of our initial active learning pipeline in Airflow had been painful: It literally took just a few weeks to revamp our platform’s main pipeline entirely, to the delight of users and developers alike.”
“We get a lot of reusable workflows, and Flyte™ makes it fairly easy to share complex machine learning and different dependencies between teams without actually having to put all the dependencies into one container.”
“Versioning, caching and the different domains we can have in Flyte™ prompted us to move from Airflow to Flyte™ because you don’t really need to think about them and they are … available out of the box in Flyte™.”