Converge and Simplify AI, Data and Analytics

Union Cloud reduces time to market: Its production-grade AI orchestration offers a common architecture pattern for people and processes. Cut boilerplate code and total pipeline cost by up to 90%.

Converge and Simplify AI, Data and Analytics
Union.ai

Globally trusted and tested

10
k+
Community members
1
m+
Downloads per month
30
+
Fortune 100 companies

Introducing

Bring your data, AI and teams together

Union Cloud unifies data and machine learning pipelines, people and processes while reducing compute cost and time to value. Union Cloud is a hosted version of Flyte™, an open-source solution backed by a vibrant user community and used in production by more than 30 companies in the Fortune 100.

Effortless and efficient

Union Cloud is the infinitely scalable workflow platform that is resource optimized. With caching, task-aware resourcing and more, Union Cloud is the most efficient cloud-native fabric for the entire data and ML stack.

Learn about ML in production
Efficient Workflows

Converge data and AI pipelines and systems

Traditional data-science and AI platforms aren’t built for dynamic business demands. Union Cloud’s data and AI platform supports hybrid and multicloud data ecosystems. It’s a fabric for rapid data, AI and analytics development and deployment — at any scale.

Snowflake, SQAlchemy, DoltHub, BigQuery, Hive and DuckDBDatabricks, AWS Athena and Apache SparkPandera and Great ExpectationsVaex, Polars and FlyteKuneflow, Dask, Ray, Dask, HuggingFace, DeepSpeed, UnionML, AWS Sagemaker and JaxW&B and WhylogsOnnx, BentoML, AWS Sagemaker and BananaWhylogs and MLFlowKuneflow, Dask, Ray, Dask, HuggingFace, DeepSpeed, AWS Sagemaker and Jax
Extract
Transform
Validate
Explore
Train
Test
Deploy
Monitor
Retrain

“As [data] engineers, a lot of this might be table stakes for us. But for data scientists, being able to get up and running [financial analytics] on Flyte™ and getting all of this stuff for free has been a really big win for them.”

— Dylan Wilder, Engineering Manager at Spotify

Ensure reliability, reusability and repeatability

Union Cloud is powered by Flyte™, a Kubernetes-native orchestrator that’s helping organizations like Spotify and Lyft build a new generation of products that make elegant use of complex data and machine learning workflows while reducing technology duplication.

Learn more about Flyte™

“We got over 66% reduction in orchestration code when we moved to Flyte™ — a huge win!”

— Seth Miller-Zhang, Senior Software Engineer at ZipRecruiter

Union Cloud is powered by Flyte™

See what the community has to say about it

To develop excellent products, leading companies must coordinate complex workflows that integrate data, analytics and machine learning. Whether they use their own infrastructure or host on Union Cloud, these organizations choose Flyte™-based solutions over Airflow, Kubeflow and other orchestrators.

To reduce harmful greenhouse gases 45% by 2025, MethaneSAT will need to measure daily global methane emissions from 30 different targets every day. It will use Union Cloud’s powerhouse Flyte™ engine as its production task orchestrator.

“We’re really excited about the capabilities that this project will enable us to do, and Flyte™’s going to get us there.”

– Nicholas LoFaso, Senior Platform Software Engineer at MethaneSAT

In 2022, fintech company Stash compared orchestration platforms and chose Union Cloud’s core engine, Flyte™. Stash cut its provision time by 50%, reduced its model execution time by 65% and lowered its compute costs by 67%.

“We want to make writing a workflow for people’s jobs pleasant and straightforward. … [I]f it’s not simple and easy, people aren’t going to adopt the technology, so this one was a really big one for us.”

— Katrina Palen, ML Platform Engineer at Stash

Blackshark.ai uses AI models and satellite imagery to generate real-time, accurate, searchable and photorealistic digital 3D maps of the entire planet. To run AI detection and content generation, Blackshark turned to the Flyte™ orchestration engine behind Union Cloud. 

“We don't want to be limited to a single cloud provider, so having the ability to run everything through Kubernetes is is amazing for us.”

— Maarten de Jong, Python Developer at Blackshark.ai

Creating quarterly two-year P&L projections for the digital media giant required three to four weeks’ work from 15 teams. With Union Cloud’s Flyte™ engine, Spotify’s finance team built a single workflow to crunch the data in real time.

“For data scientists, being able to get up and running on Flyte™, and getting all of this stuff for free, is a really big win.”

— Dylan Wilder, Engineering Manager at Spotify

Gojek’s services range from food and transportation to payments and entertainment. When it was time to graduate from its homegrown orchestration system to a unified platform, the Gojek team chose Union Cloud’s Flyte™ orchestration engine.

“For data, our platform has to scale to handle an order or two of magnitude [more] than what we currently have. We are happy with our experience so far with Flyte™.”

— Pradithya Aria Pura, Principal Software Engineer at Gojek

Stay on top of your data and AI pipelines

Union Cloud is a cloud-native platform that provides the infrastructure and tools you need to manage and optimize data and ML workflows at any scale. It provides valuable insights into performance, behavior and compliance of your pipelines, making it easier to debug issues, optimize performance and ensure that the workflows are operating as expected.

AI needs constant maintenance

Making AI products work means retraining and processing vast amounts of data — over and over again. That can be time-consuming and computationally expensive. When teams can analyze and monitor workflows proactively, they make make better data-driven decisions, achieve faster execution times, optimize resource allocation and arrive at better outcomes.

Union Cloud’s detailed metrics on performance resources make it easier to identify bottlenecks and optimize pipeline performance.

AI is a
team sport

From start to finish, AI projects are multidisciplinary. They require data scientists, domain experts, and machine learning and infrastructure engineers. To succeed, AI teams must work together and align goals, processes and tooling.

Union Cloud provides a centralized platform for teams to coordinate and collaborate through a standard workflow definition, versioning, task reusability, access controls and an extensible plugin system.

AI at scale
is complex

Scaling AI into production-level systems introduces a level of complexity that can feel impossible to navigate.

Union Cloud empowers organizations to scale their infrastructure, data processing, applications — even their organization. By integrating with scalable technologies (such as Spark and Ray), supporting collaborative workflows and providing granular and custom access controls, your team can scale their operations without sacrificing efficiency or performance.

The data and AI platform for faster product innovation

Union Cloud is a hosted version of Flyte™, managed by Flyte™’s creators.

As your growing data and ML pipelines require more computing power, your workflow orchestration platform must keep pace. Union Cloud is designed to scale, so you won’t have to worry about constant monitoring and maintenance. With Union Cloud, you can easily handle changing workloads and resource requirements, so your data and ML workflows always have the resources they need to succeed.

Kubernetes-native Union Cloud’s advanced resource management for large-scale data processing and AI workloads includes automatic resource allocation, automatic scaling and self-healing. Precisely adjust resources within your code — either during runtime or with real-time resource calculations — without modifying the underlying infrastructure. Optimize resources seamlessly, and enhance the overall efficiency of your system.

Union Cloud lets you package applications as containers and deploy them to your preferred cloud environments (like AWS or GCP) and even on-premise data centers. Deploying cloud-agnostic AI and data pipelines ensures workflows run in the best environment for the specific workload.

According to Gartner’s 2021 AI in Organizations Survey, organizations take an average of 7.3 months to move an AI initiative from prototype to production, and 47% of organizations take more than six months on average. Union Cloud reduces up to 90% of boilerplate code and delivers data, AI and analytics faster than any other platform.

Don’t just take our word for it

Infinome Biosciences cut pipeline runtimes by 98%. MethaneSAT swiftly crunches 2 terabytes of data across 10,000 CPUs. ZipRecruiter cut its orchestration code by two-thirds. Learn how companies like yours use Union Cloud’s Flyte™ engine to achieve their business goals.

Github

Github

Spread the love and help us build awareness of this project!

Star us on Github
Slack

Slack

We’d love to connect and hear your feedback, questions and ideas!

Join us on Slack
Contribute

Community

Help us to help you build amazing open-source software!

Get involved