Event-driven automation

Every automation starts with a cause. The cause determines whether the automation runs on time, whether it fires twice, and whether anyone can later tell why it ran.

Choose the cause

Use a schedule only when time is the cause, such as a weekly report or a nightly compaction. A nightly job that rebuilds a dataset “after the upstream load finishes” runs late when the load is early, and on stale data when the load is late. Neither failure raises an error.

Receive a webhook

A webhook receiver is an app that verifies each inbound delivery with the sending product’s own scheme, normalizes it, and launches a run. One app can serve any combination of the five supported products:

import flyte
from flyte.extras.webhooks import WebhookAppEnvironment, WebhookEvent, run_once
from flyteplugins.github import GitHubProvider, events

# Serves the receiver at /webhook/github and a setup dashboard at /.
# GITHUB_WEBHOOK_SECRET is mounted automatically.
#
# `scopes` lists the repositories to act on. Deliveries from other
# repositories, or with no repository, are acknowledged but not dispatched.
app_env = WebhookAppEnvironment(
    name="github-webhooks",
    providers=[GitHubProvider()],
    scopes=["octo/repo"],
    image=flyte.Image.from_debian_base().with_pip_packages("flyteplugins-github[app]"),
    resources=flyte.Resources(cpu=1, memory="512Mi"),
)

Each provider’s secret is mounted automatically. The dashboard at / shows the payload URL to paste into the product’s webhook settings.

Launch once per event

Webhook senders retry on any non-2xx response, pollers overlap their windows, and operators re-trigger by hand. Without protection, one pull request can launch four triage runs and leave four comments.

run_once deduplicates on the event rather than the delivery:

@app_env.on_event(events.PullRequest.OPENED)
async def on_pull_request_opened(event: WebhookEvent) -> dict:
    """Launch triage once per pull request.

    Retried and resent deliveries have the same `dedupe_key()`, so `run_once`
    launches only one run for them. Use `await run_once.aio(...)`: the blocking
    form stalls the app's event loop, and GitHub times out a delivery after
    ten seconds.
    """
    import flyte.remote as remote

    task = remote.Task.get(name="github-triage.triage_pr", auto_version="latest")
    result = await run_once.aio(
        task,
        key=event.dedupe_key(),
        repo=event.scope,
        number=event.payload["pull_request"]["number"],
    )
    if not result.created:
        # An earlier delivery of this event already launched a run.
        return {"skipped": result.run.name, "url": result.run.url}
    return {"run": result.run.name, "url": result.run.url}

How it behaves:

• Every run launched this way carries the label dedupe=<key>. A launch is refused if a run with that label is live or has succeeded.
• Failed, aborted, and timed-out runs don’t block a new launch, so re-triggering after a failure retries the work.
• dedupe_key() includes the provider’s timestamp, so a later change to the same resource gets a new key and launches a new run.
• The key is a string. Build your own for a different scope, such as one run per thread instead of one per message.

Keep the receiver thin

The receiver should verify, decide, and launch. Do the work in the launched task.

The receiver is an app with a request deadline: GitHub waits 10 seconds for a response. Work done in the handler isn’t retried and doesn’t appear as a run. Work done in the launched task is retried, observable, and recorded.

• Call await run_once.aio(...) in handlers. The blocking form stalls the app’s event loop, and under load the receiver stops responding.
• Keep the app and the tasks in separate files. Importing a module that constructs a WebhookAppEnvironment requires fastapi, so a shared file puts fastapi in every task image.

Restrict scopes

scopes is an allowlist of repositories, channels, teams, lists, or project keys. Events from anywhere else are acknowledged, so the sender stops retrying, but they are not dispatched.

Always set scopes. Without it, the receiver acts on any repository whose webhook points at it. The webhook secret authenticates the sender, not the repository.

Events that carry no scope are also not dispatched. If a webhook seems not to fire, check the product page for where that provider reads its scope from.

Test in stages

When nothing happens, the cause could be credentials, parsing, or networking. Test in this order to separate them:

1. Run the task directly with flyte run <file> <task>. This tests the credentials and the logic without a webhook.
2. Replay the plugin’s SAMPLE_DELIVERY. Every provider plugin ships one, so this tests verification and parsing without a product account.
3. Deploy the receiver and send a real delivery. This tests the remaining wiring.

See also

• Review and release gates for what the launched run can do.
• Software development tools for each provider’s events and setup.
• Triggers for schedules.