ArteraAI scales personalized cancer therapy while cutting GPU inference costs 60% with Union.ai

Artera is a pioneer in AI-guided cancer care, and one of the first to bring FDA-authorized, AI-based cancer prognostics into clinical usage and a commercial reality. It builds AI tests that analyze a patient's own tumor to predict future outcomes, turning a digitized biopsy image into a precision medicine insight an oncologist can act on. 

The ArteraAI Prostate Test is the first and only test that is both prognostic and predictive for localized prostate cancer, recognized in national clinical practice guidelines, and has already helped guide treatment decisions for thousands of patients across the US and beyond. Artera is now carrying that approach into new cancers, new countries, and new health systems, with production running across many AWS regions on several different continents.

Every ArteraAI Prostate Test runs on a GPU-heavy inference pipeline that includes several key steps: First, the patient's biopsy is digitized into a whole-slide image and decomposed into thousands of patches. A classifier then runs, keeping only the tissue patches. Next, a self-supervised model transforms these patches into features. Finally, a fusion model combines those features with clinical variables to generate a calibrated prediction. 

Controlling costs, maintaining accuracy and meeting SLAs are critical as Artera adds new cancer tests, expands into new countries, and serves more patients. With each growth phase the volume of inference climbs quickly, and every new pipeline has to be fast and cost-efficient. Typically, running faster requires more resources which means paying more – but not with Union.ai. 

Artera runs all of this on Union.ai, and a good deal of the efficiency comes with the platform itself. Union.ai is the enterprise platform for Flyte and has enhanced capabilities beyond Flyte native features like cached results so work is not recomputed. Union can place jobs on cheaper interruptible capacity, and retries automatically when that capacity is pulled away. Dynamic Batcher saturates GPU RAM, Cold Boot starts containers in milliseconds, Reusable Containers remove stop/start events. In all these ways, Union.ai lowers the cost of running at scale. 

In Artera’s case, almost all of the cost in these workflows is GPU time. Before Union, every GPU task carried a hidden tax: container image-pull time. When a task in a pipeline is scheduled, a node spins up and the container image is pulled. Only after it is fully downloaded and booted up can the task run. Like most GPU workloads, Artera's image is large due to its stacked layers: a base image, CUDA, and PyTorch. Building and pulling the image took minutes on every fresh node. Image pull time sits in the critical path of each task. Across a wide fan-out, this overhead compounds into costly waste and slow, unpredictable start times.

That cost doesn't stay fixed: it grows with every node. A pipeline that fans out to thousands of GPUs pays the image-pull tax thousands of times over. Artera's pipelines fan out to thousands of parallel jobs on GPUs.The largest pipelines span multiple EKS clusters, with thousands of nodes.  At that scale, a pipeline that encounters an issue partway through would be incredibly expensive to rerun from a monolithic checkpoint. Running 8000 nodes for a single pipeline requires reliability that is hard to build and maintain in-house, and it only gets harder with more scale.

Running inference on Union changes how images are pulled and loaded in two ways:

• ‍Union Remote Image Builder. Hand-written Dockerfiles were replaced by images defined natively (flyte.Image) and built and pushed by the Union Image Builder as part of deployment. The images are lean by construction: dependencies installed in a cache-friendly layer, workflow code ships separately so ordinary code changes never trigger a rebuild, and model checkpoints and calibration tables are baked into the image at build time instead of pulled at runtime. Remote image builds mean the Artera team doesn’t need to worry about linux kernels, uv caches, or other build environment consistencies, while at the same time accelerating development speed by not being bogged down by frequent rebuilds. ‍
• Cold Boot Optimization. Union.ai reduces multi-minute cold-starts down to milliseconds. Rather than downloading the whole image before a container can start, the node pulls the image metadata in a fraction of a second and then loads the actual image contents lazily, on demand, as the task runs. The one tradeoff is that a few seconds of what used to be "pull" time now happen during the task's runtime, as content is fetched on first use.

Artera's customer SLA demands a durable runtime, where interruptions don't result in expensive reprocessing, and no data is ever redownloaded. Every pipeline task runs with automated retries and backoff, so a reclaimed spot node or a transient infrastructure failure is recovered automatically instead of sinking the run or reprocessing from the beginning. Union and Flyte both self-heal exactly at the point of failure. Automated checkpointing, versioning and caching combine to ensure no data is unnecessarily redownloaded and no step is unnecessarily rerun. Union's focus on durability and the features that support it enable Artera to confidently run analyses that rapidly span 8000 nodes and deliver patient cancer test results across multiple regions. As demand grows, more nodes and more clusters come online to absorb it, and this resilience is what keeps that growth dependable rather than fragile.

Measured in production. On the migrated inference workflow, the numbers moved like this:

The image pull dropped from roughly three minutes to about a second, and the end-to-end run nearly halved. The tradeoff is the ~5 seconds of added GPU time per task: with lazy loading, image content that used to arrive during the pull now streams in during the first moments of the task. That is a few seconds per task, weighed against a pull saving measured in minutes per node.

What it means in dollars. The math starts with a single GPU task. For Artera, every pipeline task runs on a node, and based on their image size and legacy orchestrator, a fresh node previously spent over 3 minutes pulling the image before the task could start. A T4 node in AWS costs $0.526 an hour, which means 3-minutes waiting costs $0.028 of GPU time (3 x $0.526 / 60). By implementing Union Remote Image Building and Cold Boot Optimization 3 minutes is reduced to sub-second. Saving ~$0.028 on every new node.

At scale, Artera’s pipelines fan-out to execute n tasks on GPU nodes. The concurrency setting of the fan-out impacts how many GPU nodes are spun up at once and how many pipeline tasks land on a new node with an image pull vs reusing an existing node. 

At full concurrency all n start in parallel. Artera previously implemented a concurrency of n/2, meaning the pipeline goes in two waves and n/2 tasks land on new nodes. For example, a pipeline that fans out to 100 GPU tasks runs at a concurrency of 50 today, or 100 at full concurrency. 

At full concurrency, where every task lands on a fresh node, the economics shift: instead of 3 minutes of pulling and startup, followed by 2 minutes of compute for the actual inference, the node just computes, cutting GPU-node time by 60% (5min -> 2min). By removing this image pull tax, Union.ai reduces GPU costs and produces results faster. The ratio itself doesn't change with fan-out width. What changes is how much of the fan-out gets to claim it.

The savings matter, but the bigger effect is on how Artera scales. With the image-pull tax gone, running at higher concurrency no longer means a bigger bill, so Artera can push pipelines faster without paying more to do it. And that gap widens every time Artera opens a new region or ships a new test.

From Training to Production on Union.ai

Production inference is only half of what Artera runs on Union. Its growth depends on new cancer assays and international expansion, and both start with model development and training, a different kind of workload that Union handles as well. The scale here is even  larger. Artera runs millions of GPU tasks a month, on bigger hardware that ranges from single-GPU nodes up to 8-GPU nodes to distributed, multi-GPU multi-node training, with hyperparameter sweeps and training jobs that run for hours to days at a stretch. Remote Image Builder and Cold Boot Optimization help on this side too, but most of the savings here come from Reusable Containers and interruptible jobs that let Artera run on spot instances with intra-task checkpointing.

The result is one platform behind the tests Artera runs in the clinic today and the ones it's building next. Faster starts, lower GPU cost, and a runtime that recovers on its own lets Artera scale across regions and clusters without giving up speed or reliability to get there, which is what it takes to bring these tests to more patients, in more places, over time.

Authors: Daniel Sola and Kevin He