* This page is part of the open-source Flyte v2 docs.
* Full index: https://www.union.ai/docs/v2/flyte/llms.txt
* Flyte is the open-source project. Union.ai is a commercial superset of it and uses the same `flyte` SDK and CLI, so the Union.ai docs cover everything here and more.
* If a question is not answered in these docs, or is about a feature that may be commercial, check the Union.ai v2 docs at https://www.union.ai/docs/v2/union/llms.txt
* These are the v2 docs. v1 is out of date and deprecated. Only refer to v1 docs if asked explicitly about v1. The v1 index is at https://www.union.ai/docs/v1/flyte/llms.txt

Gate a change on its way from pull request to production, including changes to a model or dataset that never appear as a diff.

# Review and release gates

A gate is a check that must pass before a change can proceed. CI handles cheap checks well. Use Flyte for the checks that need real data, real compute, or a person, and for changes that never appear as a diff.

## Match the gate to the change

| The change is | It arrives as | Gate it with |
|---|---|---|
| Code | A commit | Lint, type checks, and unit tests, in CI |
| Code that changes behavior | A commit | An [evaluation run](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/evaluation-gates.md) on real data |
| A model, prompt, or dataset | No commit: a training run finished, or an upstream table changed | An evaluation run against what is currently live |

For the third kind of change there is no pull request to attach a check to. The gate has to start when the new model or dataset appears, and it has to record its verdict somewhere other than a PR status.

## Gate a pull request

The [GitHub integration](https://www.union.ai/docs/v2/flyte/integrations/software-development-tools/github.md) receives the pull request event and launches a task. The receiver is the one shown in [Event-driven automation](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/event-driven-automation.md#launch-once-per-event). The launched task does the work and writes the result back to the pull request. This one labels the pull request by size:

```
# /// script
# requires-python = "==3.13"
# dependencies = [
#    "flyte>=2.10.6",
#    "flyteplugins-github[review,auth]>=2.10.6",
#    "PyGithub>=2",
# ]
# main = "replay_sample_delivery"
# params = ""
# ///
"""Tasks launched by the GitHub webhook receiver.

- `triage_pr` labels a pull request by size, calling PyGithub directly.
- `gated_merge` uses `review_pr` to pause the run until a person reviews the
  pull request in the Flyte UI.
- `clone_at_head` mints a short-lived GitHub App installation token.

`replay_sample_delivery` runs without a GitHub account, webhook, or credentials:

    flyte run --local github_tasks.py replay_sample_delivery
"""

import flyte

# {{docs-fragment task}}
env = flyte.TaskEnvironment(
    name="github-triage",
    image=flyte.Image.from_debian_base().with_pip_packages("PyGithub"),
    # Token for reading the pull request. The webhook secret is mounted on
    # the receiver app, not here.
    secrets=[flyte.Secret(key="github-token", as_env_var="GITHUB_TOKEN")],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@env.task
async def triage_pr(repo: str, number: int) -> str:
    """Label a pull request by size."""
    import os

    from github import Auth, Github

    client = Github(auth=Auth.Token(os.environ["GITHUB_TOKEN"]))
    pull = client.get_repo(repo).get_pull(number)
    changed = pull.additions + pull.deletions
    label = "size/s" if changed < 50 else "size/m" if changed < 500 else "size/l"
    pull.add_to_labels(label)
    return label
# {{/docs-fragment task}}

# {{docs-fragment review-gate}}
review_env = flyte.TaskEnvironment(
    name="github-review",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github[review]"),
    secrets=[flyte.Secret(key="github-token", as_env_var="GITHUB_TOKEN")],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@review_env.task
async def gated_merge(repo: str, number: int) -> str:
    """Wait for a review decision in the Flyte UI, then act on it.

    `review_pr` creates a condition carrying the pull request's metadata and
    waits for a reviewer to answer it. The condition is stored on the backend,
    so the run survives restarts while it waits.
    """
    from flyteplugins.github import review_pr

    decision = await review_pr(repo, number, instructions="Block on missing tests.")
    if decision.is_approved:
        return f"approved by {decision.reviewer}: {decision.summary}"
    blocking = ", ".join(c.path for c in decision.blocking_comments)
    return f"{decision.verdict}: {decision.summary} ({blocking})"
# {{/docs-fragment review-gate}}

# {{docs-fragment app-token}}
agent_env = flyte.TaskEnvironment(
    name="github-agent",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github[auth]"),
    # Each key maps to the environment variable `mint_installation_token`
    # reads (github-app-id -> GITHUB_APP_ID), so `as_env_var=` isn't needed.
    secrets=[
        flyte.Secret(key="github-app-id"),
        flyte.Secret(key="github-app-installation-id"),
        flyte.Secret(key="github-app-private-key"),
    ],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@agent_env.task
async def clone_at_head(repo: str) -> str:
    """Build an authenticated clone URL with a GitHub App installation token.

    The token expires after one hour.
    """
    import asyncio

    from flyteplugins.github import clone_url, mint_installation_token

    # mint_installation_token is synchronous; run it off the event loop.
    # It returns None if the App credentials are missing.
    token = await asyncio.to_thread(mint_installation_token)
    url = clone_url(repo, token)
    # Never return or log the token itself.
    return url.replace(token, "***") if token else url
# {{/docs-fragment app-token}}

# {{docs-fragment replay}}
# A separate environment with no secrets, so the replay runs before any
# secret is created.
replay_env = flyte.TaskEnvironment(
    name="github-replay",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github"),
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@replay_env.task
async def replay_sample_delivery() -> dict[str, str]:
    """Verify and parse the sample delivery bundled with the plugin.

    `SAMPLE_DELIVERY` is a recorded GitHub payload and a function that signs
    it. The body is real; the signature header is generated by the plugin.
    """
    import flyteplugins.github as plugin

    secret = "a-test-signing-secret"
    sign, body = plugin.SAMPLE_DELIVERY
    headers = sign(body, secret)

    assert plugin.verify(body, headers, secret), "a correctly signed delivery must verify"
    assert not plugin.verify(body, headers, "wrong-secret"), "a bad signature must not"

    event = plugin.parse(headers, body)
    return {
        # Matched by `on_event`. `events.PullRequest.OPENED` equals this string.
        "qualified_type": event.qualified_type,
        # Matched against `scopes`.
        "scope": event.scope or "",
        "title": event.title or "",
        "actor": event.actor or "",
        # The key `run_once` deduplicates on.
        "dedupe_key": event.dedupe_key(),
    }
# {{/docs-fragment replay}}

if __name__ == "__main__":
    flyte.init_from_config()
    run = flyte.run(replay_sample_delivery)
    print(run.name)
    print(run.url)
    run.wait()
```

*Source: https://github.com/unionai/unionai-examples/blob/main/v2/integrations/flyte-plugins/github/github_tasks.py*

To make the result a required status, have the task publish a check run through the GitHub API (for example, `Repository.create_check_run` in PyGithub), and require that check in branch protection. GitHub accepts check runs only from a GitHub App, so authenticate with an [installation token](#gate-agent-pull-requests) rather than a personal access token.

Running the gate as a Flyte task instead of a CI job lets it:

- Hold a GPU for only the step that needs it, with per-task [resources](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/resources.md).
- Skip the expensive work on a re-run, with [caching](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/caching.md).
- Run a thousand-case suite in parallel, with [fan-out](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/fanout.md).
- Absorb a provider's transient 503 with [retries](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/retries-and-timeouts.md), instead of failing the build.

## Gate on a person's review

`review_pr` pauses a run on a condition that carries the pull request's metadata, waits for a person to answer in the Flyte UI, and returns a typed verdict:

```
# /// script
# requires-python = "==3.13"
# dependencies = [
#    "flyte>=2.10.6",
#    "flyteplugins-github[review,auth]>=2.10.6",
#    "PyGithub>=2",
# ]
# main = "replay_sample_delivery"
# params = ""
# ///
"""Tasks launched by the GitHub webhook receiver.

- `triage_pr` labels a pull request by size, calling PyGithub directly.
- `gated_merge` uses `review_pr` to pause the run until a person reviews the
  pull request in the Flyte UI.
- `clone_at_head` mints a short-lived GitHub App installation token.

`replay_sample_delivery` runs without a GitHub account, webhook, or credentials:

    flyte run --local github_tasks.py replay_sample_delivery
"""

import flyte

# {{docs-fragment task}}
env = flyte.TaskEnvironment(
    name="github-triage",
    image=flyte.Image.from_debian_base().with_pip_packages("PyGithub"),
    # Token for reading the pull request. The webhook secret is mounted on
    # the receiver app, not here.
    secrets=[flyte.Secret(key="github-token", as_env_var="GITHUB_TOKEN")],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@env.task
async def triage_pr(repo: str, number: int) -> str:
    """Label a pull request by size."""
    import os

    from github import Auth, Github

    client = Github(auth=Auth.Token(os.environ["GITHUB_TOKEN"]))
    pull = client.get_repo(repo).get_pull(number)
    changed = pull.additions + pull.deletions
    label = "size/s" if changed < 50 else "size/m" if changed < 500 else "size/l"
    pull.add_to_labels(label)
    return label
# {{/docs-fragment task}}

# {{docs-fragment review-gate}}
review_env = flyte.TaskEnvironment(
    name="github-review",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github[review]"),
    secrets=[flyte.Secret(key="github-token", as_env_var="GITHUB_TOKEN")],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@review_env.task
async def gated_merge(repo: str, number: int) -> str:
    """Wait for a review decision in the Flyte UI, then act on it.

    `review_pr` creates a condition carrying the pull request's metadata and
    waits for a reviewer to answer it. The condition is stored on the backend,
    so the run survives restarts while it waits.
    """
    from flyteplugins.github import review_pr

    decision = await review_pr(repo, number, instructions="Block on missing tests.")
    if decision.is_approved:
        return f"approved by {decision.reviewer}: {decision.summary}"
    blocking = ", ".join(c.path for c in decision.blocking_comments)
    return f"{decision.verdict}: {decision.summary} ({blocking})"
# {{/docs-fragment review-gate}}

# {{docs-fragment app-token}}
agent_env = flyte.TaskEnvironment(
    name="github-agent",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github[auth]"),
    # Each key maps to the environment variable `mint_installation_token`
    # reads (github-app-id -> GITHUB_APP_ID), so `as_env_var=` isn't needed.
    secrets=[
        flyte.Secret(key="github-app-id"),
        flyte.Secret(key="github-app-installation-id"),
        flyte.Secret(key="github-app-private-key"),
    ],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@agent_env.task
async def clone_at_head(repo: str) -> str:
    """Build an authenticated clone URL with a GitHub App installation token.

    The token expires after one hour.
    """
    import asyncio

    from flyteplugins.github import clone_url, mint_installation_token

    # mint_installation_token is synchronous; run it off the event loop.
    # It returns None if the App credentials are missing.
    token = await asyncio.to_thread(mint_installation_token)
    url = clone_url(repo, token)
    # Never return or log the token itself.
    return url.replace(token, "***") if token else url
# {{/docs-fragment app-token}}

# {{docs-fragment replay}}
# A separate environment with no secrets, so the replay runs before any
# secret is created.
replay_env = flyte.TaskEnvironment(
    name="github-replay",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github"),
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@replay_env.task
async def replay_sample_delivery() -> dict[str, str]:
    """Verify and parse the sample delivery bundled with the plugin.

    `SAMPLE_DELIVERY` is a recorded GitHub payload and a function that signs
    it. The body is real; the signature header is generated by the plugin.
    """
    import flyteplugins.github as plugin

    secret = "a-test-signing-secret"
    sign, body = plugin.SAMPLE_DELIVERY
    headers = sign(body, secret)

    assert plugin.verify(body, headers, secret), "a correctly signed delivery must verify"
    assert not plugin.verify(body, headers, "wrong-secret"), "a bad signature must not"

    event = plugin.parse(headers, body)
    return {
        # Matched by `on_event`. `events.PullRequest.OPENED` equals this string.
        "qualified_type": event.qualified_type,
        # Matched against `scopes`.
        "scope": event.scope or "",
        "title": event.title or "",
        "actor": event.actor or "",
        # The key `run_once` deduplicates on.
        "dedupe_key": event.dedupe_key(),
    }
# {{/docs-fragment replay}}

if __name__ == "__main__":
    flyte.init_from_config()
    run = flyte.run(replay_sample_delivery)
    print(run.name)
    print(run.url)
    run.wait()
```

*Source: https://github.com/unionai/unionai-examples/blob/main/v2/integrations/flyte-plugins/github/github_tasks.py*

See [Human gates and approvals](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/human-gates-and-approvals.md) for timeouts, who may answer, and how long a run can wait.

## Promote the same version

Treat a release as a promotion: the same versioned code moves between domains as it passes each gate, instead of being rebuilt for each environment.

1. On every merge, deploy to `development` pinned to the commit, with `flyte deploy --version ${{ github.sha }}`. See [CI/CD deployments](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/cicd.md).
2. Run the gate there, against real data.
3. If it passes, promote the same version to `staging`, then to `production`.

Deploying the same commit twice produces the same version, and tasks already registered at that version are not registered again. Every run traces back to its commit. If you rebuild for each environment, you can no longer show that what you tested is what you shipped.

## Gate agent pull requests

When an agent opens pull requests, change two things.

**Credentials.** Authenticate the agent as a GitHub App. It mints a short-lived installation token for each operation instead of holding a personal access token:

```
# /// script
# requires-python = "==3.13"
# dependencies = [
#    "flyte>=2.10.6",
#    "flyteplugins-github[review,auth]>=2.10.6",
#    "PyGithub>=2",
# ]
# main = "replay_sample_delivery"
# params = ""
# ///
"""Tasks launched by the GitHub webhook receiver.

- `triage_pr` labels a pull request by size, calling PyGithub directly.
- `gated_merge` uses `review_pr` to pause the run until a person reviews the
  pull request in the Flyte UI.
- `clone_at_head` mints a short-lived GitHub App installation token.

`replay_sample_delivery` runs without a GitHub account, webhook, or credentials:

    flyte run --local github_tasks.py replay_sample_delivery
"""

import flyte

# {{docs-fragment task}}
env = flyte.TaskEnvironment(
    name="github-triage",
    image=flyte.Image.from_debian_base().with_pip_packages("PyGithub"),
    # Token for reading the pull request. The webhook secret is mounted on
    # the receiver app, not here.
    secrets=[flyte.Secret(key="github-token", as_env_var="GITHUB_TOKEN")],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@env.task
async def triage_pr(repo: str, number: int) -> str:
    """Label a pull request by size."""
    import os

    from github import Auth, Github

    client = Github(auth=Auth.Token(os.environ["GITHUB_TOKEN"]))
    pull = client.get_repo(repo).get_pull(number)
    changed = pull.additions + pull.deletions
    label = "size/s" if changed < 50 else "size/m" if changed < 500 else "size/l"
    pull.add_to_labels(label)
    return label
# {{/docs-fragment task}}

# {{docs-fragment review-gate}}
review_env = flyte.TaskEnvironment(
    name="github-review",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github[review]"),
    secrets=[flyte.Secret(key="github-token", as_env_var="GITHUB_TOKEN")],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@review_env.task
async def gated_merge(repo: str, number: int) -> str:
    """Wait for a review decision in the Flyte UI, then act on it.

    `review_pr` creates a condition carrying the pull request's metadata and
    waits for a reviewer to answer it. The condition is stored on the backend,
    so the run survives restarts while it waits.
    """
    from flyteplugins.github import review_pr

    decision = await review_pr(repo, number, instructions="Block on missing tests.")
    if decision.is_approved:
        return f"approved by {decision.reviewer}: {decision.summary}"
    blocking = ", ".join(c.path for c in decision.blocking_comments)
    return f"{decision.verdict}: {decision.summary} ({blocking})"
# {{/docs-fragment review-gate}}

# {{docs-fragment app-token}}
agent_env = flyte.TaskEnvironment(
    name="github-agent",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github[auth]"),
    # Each key maps to the environment variable `mint_installation_token`
    # reads (github-app-id -> GITHUB_APP_ID), so `as_env_var=` isn't needed.
    secrets=[
        flyte.Secret(key="github-app-id"),
        flyte.Secret(key="github-app-installation-id"),
        flyte.Secret(key="github-app-private-key"),
    ],
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@agent_env.task
async def clone_at_head(repo: str) -> str:
    """Build an authenticated clone URL with a GitHub App installation token.

    The token expires after one hour.
    """
    import asyncio

    from flyteplugins.github import clone_url, mint_installation_token

    # mint_installation_token is synchronous; run it off the event loop.
    # It returns None if the App credentials are missing.
    token = await asyncio.to_thread(mint_installation_token)
    url = clone_url(repo, token)
    # Never return or log the token itself.
    return url.replace(token, "***") if token else url
# {{/docs-fragment app-token}}

# {{docs-fragment replay}}
# A separate environment with no secrets, so the replay runs before any
# secret is created.
replay_env = flyte.TaskEnvironment(
    name="github-replay",
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github"),
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

@replay_env.task
async def replay_sample_delivery() -> dict[str, str]:
    """Verify and parse the sample delivery bundled with the plugin.

    `SAMPLE_DELIVERY` is a recorded GitHub payload and a function that signs
    it. The body is real; the signature header is generated by the plugin.
    """
    import flyteplugins.github as plugin

    secret = "a-test-signing-secret"
    sign, body = plugin.SAMPLE_DELIVERY
    headers = sign(body, secret)

    assert plugin.verify(body, headers, secret), "a correctly signed delivery must verify"
    assert not plugin.verify(body, headers, "wrong-secret"), "a bad signature must not"

    event = plugin.parse(headers, body)
    return {
        # Matched by `on_event`. `events.PullRequest.OPENED` equals this string.
        "qualified_type": event.qualified_type,
        # Matched against `scopes`.
        "scope": event.scope or "",
        "title": event.title or "",
        "actor": event.actor or "",
        # The key `run_once` deduplicates on.
        "dedupe_key": event.dedupe_key(),
    }
# {{/docs-fragment replay}}

if __name__ == "__main__":
    flyte.init_from_config()
    run = flyte.run(replay_sample_delivery)
    print(run.name)
    print(run.url)
    run.wait()
```

*Source: https://github.com/unionai/unionai-examples/blob/main/v2/integrations/flyte-plugins/github/github_tasks.py*

Installation tokens expire after one hour. That is long enough for a clone or a `gh pr create`, and a leaked token stops working soon after.

**Review load.** An agent can open more pull requests than reviewers can read carefully. To keep review meaningful:

- Make an automated gate the only gate for mechanical changes, and reserve human review for changes that alter behavior.
- Test generated code before review. The [code generation integration](https://www.union.ai/docs/v2/flyte/integrations/codegen.md) runs it in a sandbox first.

[Agent frameworks](https://www.union.ai/docs/v2/flyte/integrations/agents.md) covers running agent SDKs as durable tasks, and [Agents](https://www.union.ai/docs/v2/flyte/user-guide/agents.md) covers Flyte's own agent harness.

## Gate a dataset change

Gate a dataset change like a model change, and add a schema check in front of it. [Pandera](https://www.union.ai/docs/v2/flyte/integrations/pandera.md) validates dataframes at task boundaries, so a changed column type or an unexpected null fails at the step that introduced it. See [Evaluation gates](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/evaluation-gates.md#gate-the-input) for input checks on model output.

## Gates to avoid

A gate that people work around gives false assurance. Avoid these:

- **A non-deterministic gate with no tolerance.** If it fails one run in five, people re-run it until it passes.
- **An absolute threshold.** "Accuracy above 0.9" turns permanently red when the data shifts, or permanently passes. Compare against what is live instead. See [Evaluation gates](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/evaluation-gates.md).
- **A slow gate in front of a fast loop.** A 40-minute gate belongs after merge, with the ability to roll back.

## See also

- [Evaluation gates](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/evaluation-gates.md) for measuring quality well enough to gate on it.
- [Human gates and approvals](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/human-gates-and-approvals.md) for putting a person in the loop.
- [CI/CD deployments](https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/cicd.md) for deployment from CI.

---
**Source**: https://github.com/unionai/unionai-docs/blob/main/content/user-guide/project-patterns/software-development-lifecycle/review-and-release-gates.md
**HTML**: https://www.union.ai/docs/v2/flyte/user-guide/project-patterns/software-development-lifecycle/review-and-release-gates/
