Accelerate GenAI-style workloads by pre-loading models and data into “warm” execution environments. Leverage actors across multiple workflows to amortize container startup cost across many executions.

Flexible environment specification 

Just like Flyte tasks, actor environments allow you to customize infrastructure and dependencies. Load a custom model onto a GPU and use torch to run on-demand predictions with low latency.

Startup and teardown logic

Define operations which should take place before and after your main code logic runs. Download and load an LLM into memory as the actor comes online so subsequent prompts run quickly. Perform teardown operations like entering chat history into a database to monitor and improve performance over time.

Integration with Map Tasks

Run actors inside map tasks to speed up executions and use fewer resources. Instead of spinning up one environment per job (as is the case with regular map tasks), mapping over Actors allows you spread the cost of starting each environment over multiple jobs.

Scale out to multiple clusters

Go beyond the limits of a single cluster while maintaining one access point. Union automatically load balances across multiple clusters, allowing you to run workloads spanning hundreds of compute nodes and tens of thousands of concurrent jobs.

Leverage hard-to-find GPU resources

Route workloads to different cloud providers based on GPU availability or pricing. A team running primarily on AWS can use Union to send GPU-based workloads to a cluster running in GCP. Union currently supports AWS and GCP, with Azure coming soon.

Isolate important workloads

Union lets you create hard physical and network isolation between normal jobs and mission-critical workloads. Run development, staging, and production workflows in separate AWS accounts or GCP projects as suggested by the AWS and GCP cloud architecture frameworks.

Execute workflows in parallel

Union allows you to map over entire workflows, dramatically raising the ceiling for the number of parallel jobs. Union supports workflows with nested parallelism of up to 100,000 tasks. Mapping over workflows allows users to write more logical code, with each sub-workflow greedily moving forward to completion and handling errors and retries independently.

Leverage full-workflow caching to dramatically increase efficiency and execution speed. When using nested parallelism, caching the results of each sub-workflow ensures that no computation is repeated unnecessarily.

Perform large data reads in seconds, not minutes. Accelerated datasets are pre-mounted to each compute node, dramatically reducing startup time for tasks which would otherwise need to download data over the network before running.

Run large data processing jobs on Spark, Dask, or Ray and distributed training jobs on Torch Elastic or Tensorflow natively on Union. Framework-specific logs and metrics are available in hosted Spark History Server, Ray Dashboard, and Dash Dashboard. Switch between running Spark jobs on Union and hosted providers such as Databricks with a single-line code change.

Union’s image builder makes pulling in ad-hoc dependencies seamless. Just add your dependency and run your workflow. Union will build your image in the cloud and automatically pull it down into your execution. Previously-built images are automatically cached, so subsequent runs will skip building.

Create and manage secrets on demand. Define secrets directly in the SDK and avoid the hassle of interfacing with cloud provider secrets managers. Secrets can be scoped at an organization-wide level or for specific projects. Union securely stores secrets in your cloud provider’s secrets manager, in your VPC.

Union displays runtime logs directly in the UI, avoiding the need to authenticate and search through third party logging services. 

When a remotely-running job fails, Union can automatically freeze the memory state and attach a browser-based VS Code debugger to the remote environment. You can then step through the code line-by-line, debug the failure, then continue the execution. Observe what is actually happening at runtime and avoid spending costly cycles trying to replicate remote environments.

A registry for models and data across the organization

Track important task and workflow outputs as artifacts to make them searchable, discoverable, and consumable by other teams. Store important runtime information such as hyperparameter values using model cards. Use Union’s UI to easily search across all the artifacts in a given project.

Enable separation of concerns between teams

Set up your workflows to automatically consume the latest version of an upstream team’s outputs using artifact queries. Maintain reproducibility while modularizing workflows into team-specific functions, allowing teams to iterate independently of each other. 

Understand lineage across the AI lifecycle

When you use artifacts to stitch workflows together, lineage is automatically tracked across the meta-workflow. Trace a given model’s predictions all the way back to the precursor datasets on which the model was trained.

Reactive workflows let you automatically trigger workflows in response to external events. Artifacts form the “contracts” between workflows, so a downstream team can automate their workflows in response to an upstream team’s output becoming available. Cross-workflow lineage is automatically tracked, and everything is versioned, giving you deep visibility into historical operations.

Union enables you to efficiently utilize fractional GPUs, harness cost-effective Spot instances, and seamlessly integrate custom silicon like TPUs. Streamline your AI workflows to achieve peak performance with optimal resource allocation.

Union integrates latest-generation cluster management with Karpenter to optimize your resource allocation and scalability. This setup enables you to automatically adjust capacity and performance, ensuring operational efficiency. Experience streamlined management and reduced costs, letting you focus on innovation without infrastructure constraints.

Automatically scale your operations to zero, effectively minimizing resource wastage by avoiding underutilized nodes. This capability ensures you only pay for the cloud resources you actually use, enhancing overall efficiency. Optimize your costs and environmental footprint seamlessly with Union's smart scaling.

Leverage task-level monitoring for precise tracing of CPU, memory, and GPU usage per task. This feature allows you to right-size resources efficiently, ensuring optimal allocation based on actual workload demands. Streamline your operations and enhance cost efficiency by tailoring resource use to each specific task.

Union enhances your operational oversight with cluster-wide observability, allowing you to easily monitor resource consumption by different teams over time. This comprehensive visibility helps you optimize resource allocation and improve efficiency across your organization. Gain control and insight into your operations, ensuring each team uses resources effectively.

Union can be seamlessly deployed into your cloud environment, enabling you to maintain stringent security and data locality standards while benefiting from a managed platform. This integration allows you to leverage Union's powerful features without compromising on control or compliance. Enhance your operations with robust security and efficient data management, tailored to your organizational needs.

Union provides robust role-based access control (RBAC), enabling you to precisely manage team access to specific projects while efficiently sharing underlying resources. This feature ensures that sensitive data and operations are safeguarded, maintaining organizational integrity. Customize permissions seamlessly, aligning with your team's structure and workflow requirements.

Union's deployment across global regions ensures you can meet diverse regulatory requirements and optimize costs and resource availability, including GPUs. This geographical flexibility allows you to position your services closer to your data sources and users, enhancing performance and compliance. Tailor your deployment strategy with Union to maximize efficiency and adhere to local standards effortlessly.

Union ensures high availability for your mission-critical workloads, providing a highly-resilient system that allows you to operate with confidence. This managed Flyte offering minimizes downtime and ensures your operations continue seamlessly, even under demanding conditions. Rely on Union for stability and peace of mind, knowing your crucial processes are always supported.

Union enhances your operational security by adhering to SOC 2 Type II compliance, ensuring rigorous oversight and management of data protection. This certification reflects Union’s commitment to high standards of security and confidentiality, giving you confidence in the integrity of your data. Rely on Union for a secure, managed platform that meets stringent industry requirements and safeguards your sensitive information.

Leverage the full potential of your AI operations with Union, benefiting directly from the expertise of the team that created Flyte. This managed offering provides you with unparalleled insights and support, ensuring you maximize the capabilities of your AI platform. Work with the originators of Flyte to optimize, innovate, and excel in your AI initiatives.