Union Artifacts part 1: naming the things your tasks produce

Artifacts are named, versioned references to an object in storage, plus a record of where it came from and what depends on it.

This is part one of a two part series about Union Artifacts. See part 2.

Every pipeline already produces the things a model registry stores. The training task writes weights, the preprocessing task writes a parquet file, the eval task writes a report. They all land in object storage with a URI, and then the URI becomes somebody's problem: pasted into a Slack thread, hardcoded in a serving config, glued together with a naming convention that holds until the person who invented it goes on vacation.

The usual fix is to buy a second system. You stand up a model registry next to the orchestrator, and now you have two records of the same event: the orchestrator knows a run produced a file, the registry knows a model exists, and nothing knows those are the same fact. Reconciling them is a job.

An artifact is a named, versioned reference to an object your tasks already wrote. Your data path doesn't change: files, directories, and dataframes are offloaded to object storage exactly as before. What gets added is a name, a version, and a record of provenance, recorded by the same system that ran the task.

"Artifacts is where our teams manage and share the datasets and models they produce. Every artifact carries the labels our pipelines set, thousands of values on some keys, and we can still find the exact clip we need by key, value, date, and who produced it. Collaborating with Union has been amazing: we asked for scoped filters on a Wednesday and they were live on Thursday."
— Eryk Gorzynski, Third Dimension

What an artifact is

Three properties, each of which rules something out:

• It's a reference, not a copy. The bytes stay where the task put them. Registering an artifact doesn't move or duplicate data, so there's no second storage bill and no sync to fall behind.
• It's offloaded. Only a `flyte.io.File`, `flyte.io.Dir`, or `flyte.io.DataFrame` can be an artifact. Primitives, dataclasses, and Pydantic models cannot. An artifact is an addressable asset, not a scalar, and the API rejects the scalar case rather than quietly storing a number.
• It's versioned by production. Registering under the same name again creates a new version. You don't bump anything; running the task again is the version bump.

Registering a task output

The most common way to create an artifact is to wrap a task's return value. Two things are required, and both of them:

import flyte
from flyte import artifacts
from flyte.io import File

env = flyte.TaskEnvironment(name="training")

@env.task(produces_artifacts=True)
async def train() -> File:
    weights = await File.from_local("model.pt")
    return artifacts.new(
        weights,
        artifacts.Metadata(
            name="trained-model",
            description="Classifier trained on the latest training set",
            attrs={"framework": "pytorch"},
        ),
    )

`artifacts.new()` returns a zero-copy wrapper that behaves exactly like the original `File`, so the task's declared return type doesn't change and neither does anything downstream. `produces_artifacts=True` is what tells the platform to extract the metadata on the way out.

It's worth pausing on the failure mode here, because it's the one people hit. The wrapper is stripped from the output value whether or not the flag is set. Leave the flag off and the task still succeeds, still returns a perfectly good `File`, and registers nothing: no artifact, no version, no trigger fires, no app has anything to bind to. Nothing warns you. If you're debugging an artifact that never showed up, check the decorator before you check anything else.

The wrapped value also has to be a top-level output. Nesting it inside a list, a dict, or a model fails at serialization. In a multi-output task only the wrapped slot becomes an artifact, which is usually what you want:

@env.task(produces_artifacts=True)
async def train() -> tuple[File, float]:
    ...
    return artifacts.new(weights, metadata), accuracy

Metadata, kind, and cards

`flyte.artifacts.Metadata` carries identity. Only `name` is required. Omit `version` and it comes from the producing run, so every execution registers a distinct version without you managing a counter.

`kind` is the field that makes a registry queryable. It's one of `"model"`, `"data"`, or `"generic"`, stored under a reserved `flyte.io/kind` attr and read back through the `Artifact.kind` property, which never returns `None`. Artifacts published before the key existed fall back to their card's type, so an older model with a model card still classifies. Anything with neither marker reads as `"generic"`, on the reasoning that "unlabelled" and "not a model" are the same answer to the caller.

For models there's a helper that sets `kind="model"` and the standard model attrs in one call:

card = artifacts.Card.create_from(content=card_html, format="html", card_type="model")

metadata = artifacts.Metadata.create_model_metadata(
    name="trained-model",
    description="A toy classifier",
    framework="PyTorch",
    architecture="ResNet50",
    task="Image Classification",
    serial_format="pt",
    card=card,
)

A card is a document attached to the artifact and rendered in the UI. `Card.create_from()` takes inline content or a local path, in `html`, `md`, `json`, `yaml`, `csv`, `tsv`, or an image format. `kind` and `card_type` are doing different jobs and it's easy to conflate them: `kind` is what the artifact is, `card_type` is how its card renders. An artifact can have either without the other.

Free-form `attrs` are string-valued and searchable, which is where the model card metadata actually earns its keep. `--attr framework=torch` is a filter later.

Publishing things a task didn't produce

Not every asset comes out of a pipeline. A partner drops a CSV, a colleague hands you weights from their laptop, an external system writes a dataset on a schedule. `flyte.remote.Artifact.create()` uploads and registers in one call, from anywhere:

import flyte
from flyte.io import File
from flyte.remote import Artifact

flyte.init_from_config()

published = Artifact.create(
    File.from_local_sync("data/2026-08-18.csv"),
    name="incoming_dataset",
    description="Dataset dropped off by a partner",
    external_ref="s3://partner-bucket/drop/2026-08-18.csv",
)
print(published.name, published.version)

`external_ref` is the provenance you attach when there's no run to point at. Inside a task, the producing action is stamped on the version automatically; outside one, `external_ref` is what keeps the version from being an orphan. The call is synchronous by default, with `Artifact.create.aio(...)` for async contexts.

The same thing from the CLI, which is what you want in a CI job:

flyte create artifact my-model --from-file model.pt --kind model --attr framework=torch
flyte create artifact llama3 --from-file weights.bin --external-ref hf://meta-llama/Meta-Llama-3-8B

Hugging Face gets its own path. `flyte.prefetch.hf_model()` pulls weights from the Hub into your own object storage and registers the result as a model artifact:

import flyte.prefetch

run = flyte.prefetch.hf_model(repo="HuggingFaceTB/SmolLM2-135M-Instruct")
run.wait()

The version is the upstream commit ID, which makes the prefetch idempotent: re-running it for a model that hasn't moved republishes the same version rather than filling the registry with duplicates, and a real upstream change arrives as a genuinely new version. The repo's README comes along as the model card, and `source_repo` and `source_commit` land in searchable attrs, so `flyte get artifact --attr source_repo=HuggingFaceTB/SmolLM2-135M-Instruct` gets you back to where a set of weights came from.

Consuming: three places a name resolves

A name is only worth having if things can refer to it. Artifacts resolve in three places, and in all three the consuming code is artifact-unaware.

In a task. Fetch a reference and pass it to `flyte.run()`:

from flyte.remote import Artifact

model = Artifact.get("trained-model")               # latest version
data = Artifact.get("training-set", version="v3")   # pinned version

run = flyte.run(evaluate, model=model, data=data)

The task body receives a plain `File`, `Dir`, or `DataFrame`. There is no artifact-specific code inside `evaluate`, which means the same task runs identically whether you hand it an artifact or a raw path. One sharp edge: `Artifact.listall()` is an iterator, so materialize it with `list(...)` before binding it as a run input.

In a trigger. `flyte.OnArtifact` starts a run whenever a new version of a name lands:

retrain = flyte.Trigger(
    name="retrain-on-new-model",
    automation=flyte.OnArtifact(name="customer_model"),
    inputs={"model": flyte.TriggeredArtifact, "threshold": 0.5},
    description="Validate every new customer_model version",
)

@env.task(triggers=(retrain,))
async def validate(model: File, threshold: float = 0.5) -> str:
    ...

`flyte deploy validation.py env` registers it. The `flyte.TriggeredArtifact` sentinel marks which input receives the new version; at most one input can be it, and it requires the automation to be `OnArtifact`. Every other input needs a value in the trigger definition or a default on the task.

The interesting property is that the trigger doesn't care who published. A task output, an `Artifact.create()` from a CI job, a CLI upload, a Hugging Face prefetch: all of them are a new version, all of them fire it. That makes an artifact name a clean handoff point between systems that don't otherwise know about each other. A partner drops a dataset, something publishes it, your processing pipeline starts. Nobody wrote an integration.

In an app. `flyte.app.ArtifactValue` mounts weights into a serving container instead of baking them into the image or downloading them in startup code:

import flyte.app

env = flyte.app.AppEnvironment(
    name="classifier-api",
    resources=flyte.Resources(cpu="2", memory="8Gi"),
    parameters=[
        flyte.app.Parameter(
            name="model",
            value=flyte.app.ArtifactValue(name="trained-model", type="directory"),
            mount="/tmp/model",
        )
    ],
)

The app reads `/tmp/model` like any local directory. Versions are pinned at deploy time: leave out `version` and the latest at deploy is chosen and frozen, so publishing a new version doesn't swap weights under a running app. Redeploy to move. That default is the right one, and it's the opposite of what a "latest" pointer usually does to you at 2am.

Lineage, and what it actually covers

Because the same system runs the task and records the artifact, lineage is a byproduct rather than an integration. Each version records how it came to exist: a task output records the producing run and task with a link to the execution, its inputs, and its logs; an upload records who published it and the `external_ref`; a prefetch records the source repo and commit.

On the consumer side, Union records where a version is bound to compute: apps whose deployment resolved the artifact, down to the exact pinned version, and triggers watching the name along with the runs they started. The UI draws this as a graph with the producing action on the left and dependents fanning out on the right. The practical use is the boring one: before you delete or rework a dataset, you can see every model trained on it and every app serving those models. When a served model misbehaves, you can walk back to the training run and the exact dataset version it consumed.

Lineage today is not universal, and the gaps are specific. Calling `Artifact.get()` in your own code and reading the data is a plain fetch and creates no consumer edge. Passing an artifact as a task input doesn't record one either, not yet. The tracked dependencies are apps that declare the artifact as a parameter, and runs started by artifact triggers.

The producer side, then, is complete; the consumer side covers apps and triggers and stops there. If your mental model is "everything that touched this dataset shows up here," that's not what you get today. If it's "every version knows where it came from, and every deployment knows what it's serving," that is what you get, and it covers the two questions people actually page each other about.

You can query the same relationships without the UI:

flyte get artifact                     # all names, with latest version info
flyte get artifact my-model            # every version, newest first
flyte get artifact --search model
flyte get artifact --source-run my_run
flyte get artifact --kind model --attr framework=torch

Artifacts are scoped to a project and domain, and every call takes `project` and `domain`, defaulting to your config.

Recap

Registration: `produces_artifacts=True` plus `artifacts.new()`. Both, or nothing happens and nothing tells you.

Identity: `Metadata` carries name, optional version, description, string attrs, `kind`, and a card. Omit the version and the run supplies it.

Publication: `Artifact.create()` and `flyte create artifact` for assets no task produced; `flyte.prefetch.hf_model()` for the Hub, versioned by commit and therefore idempotent.

Resolution: `Artifact.get()` into a run, `OnArtifact` into a trigger, `ArtifactValue` into an app. Consuming code stays artifact-unaware in all three.

Lineage: recorded automatically, complete on the producer side, currently covering apps and triggers on the consumer side.

None of this is a second system. It's the orchestrator writing down what it already knew. In the next post, we’ll talk about how this primitive building block enables event-driven AI and data systems.

Try it

pip install flyte
flyte create artifact demo-model --from-file model.pt --kind model --attr framework=torch
flyte get artifact demo-model

Then wire a trigger onto the name and publish a second version to watch it fire. The Artifacts guide has the full API surface, and Automatic lineage tracking documents the current edges, including the ones that aren't drawn yet.