* This page is part of the open-source Flyte v2 docs.
* Full index: https://www.union.ai/docs/v2/flyte/llms.txt
* Flyte is the open-source project. Union.ai is a commercial superset of it and uses the same `flyte` SDK and CLI, so the Union.ai docs cover everything here and more.
* If a question is not answered in these docs, or is about a feature that may be commercial, check the Union.ai v2 docs at https://www.union.ai/docs/v2/union/llms.txt
* These are the v2 docs. v1 is out of date and deprecated. Only refer to v1 docs if asked explicitly about v1. The v1 index is at https://www.union.ai/docs/v1/flyte/llms.txt

An advanced guide for building an LLM reporting agent on Flyte.

# Advanced project: LLM reporting agent

This example demonstrates a resilient agentic report generator that showcases
Flyte 2.0's advanced features for building production-grade AI workflows.

## What you'll build

A batch report generator that:

1. Processes multiple topics in parallel
2. Iteratively critiques and refines each report until it meets a quality threshold
3. Produces multiple output formats (Markdown, HTML, summary) for each report
4. Serves results through an interactive UI

## Concepts covered

| Feature | Description |
|---------|-------------|
| `ReusePolicy` | Keep containers warm for high-throughput batch processing |
| `@flyte.trace` | Checkpoint LLM calls for recovery and observability |
| `RetryStrategy` | Handle transient API failures gracefully |
| `flyte.group` | Organize parallel batches and iterations in the UI |
| `asyncio.gather` | Fan out to process multiple topics concurrently |
| Pydantic models | Structured LLM outputs |
| `AppEnvironment` | Deploy interactive Streamlit apps |
| `RunOutput` | Connect apps to pipeline outputs |

## Architecture

```mermaid
flowchart TD
    A[Topics List] --> B

    B["report_batch_pipeline<br/><i>driver_env</i>"]

    subgraph B1 ["refine_all (parallel)"]
        direction LR
        R1["refine_report<br/>topic 1"]
        R2["refine_report<br/>topic 2"]
        R3["refine_report<br/>topic N"]
    end
    B --> B1

    subgraph B2 ["format_all (parallel)"]
        direction LR
        F1["format_outputs<br/>report 1"]
        F2["format_outputs<br/>report 2"]
        F3["format_outputs<br/>report N"]
    end
    B1 --> B2

    B2 --> C["Output: List of Dirs"]
```

Each `refine_report` task runs in a reusable container (`llm_env`) and performs
multiple LLM calls through traced functions:

```mermaid
flowchart TD
    A[Topic] --> B["generate_initial_draft<br/><i>@flyte.trace</i>"]
    B --> C

    subgraph C ["refinement_loop"]
        direction TB
        D["critique_content<br/><i>@flyte.trace</i>"] -->|score >= threshold| E[exit loop]
        D -->|score < threshold| F["revise_content<br/><i>@flyte.trace</i>"]
        F --> D
    end
    C --> G[Refined Report]
```

## Prerequisites

- A Flyte account with an active project
- An OpenAI API key stored as a secret named `openai-api-key`

To create the secret:

```bash
flyte secret create openai-api-key
```

## Parts

1. **[Resilient generation](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/resilient-generation.md)**: Set up reusable environments, traced LLM calls, and retry strategies
2. **[Agentic refinement](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/agentic-refinement.md)**: Build the iterative critique-and-revise loop
3. **[Parallel outputs](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/parallel-outputs.md)**: Generate multiple formats concurrently
4. **[Serving app](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/serving-app.md)**: Deploy an interactive UI for report generation

## Key takeaways

1. **Reusable environments for batch processing**: `ReusePolicy` keeps containers warm,
   enabling efficient processing of multiple topics without cold start overhead. With
   5 topics × ~7 LLM calls each, the reusable pool handles ~35 calls efficiently.

2. **Checkpointed LLM calls**: `@flyte.trace` provides automatic checkpointing at the
   function level, enabling recovery without re-running expensive API calls.

3. **Agentic patterns**: The generate-critique-revise loop demonstrates how to build
   self-improving AI workflows with clear observability through `flyte.group`.

4. **Parallel fan-out**: `asyncio.gather` processes multiple topics concurrently,
   maximizing throughput by running refinement tasks in parallel across the batch.

{{< subpage-cards >}}

## Subpages

- [Resilient generation](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/resilient-generation.md) - Reusable environments, traced calls, and retry strategies for LLM-powered tasks.
  - Two environments
  - Reusable environment for LLM work
  - ReusePolicy parameters
  - Standard environment for orchestration
  - Traced LLM calls
  - Benefits of tracing
  - When to use @flyte.trace
  - Traced helper functions
  - Retry strategies
  - Configuring retries
  - Combining tracing with retries
  - Structured prompts
  - Pydantic models for structured output
  - Next steps
- [Agentic refinement](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/agentic-refinement.md) - Build a generate, critique, revise loop that repeats until output quality clears a threshold.
  - The agentic pattern
  - Critique function
  - Revise function
  - The refinement loop
  - How it works
  - Early exit
  - Grouping iterations with flyte.group
  - Why use flyte.group?
  - Group context
  - Configuring the loop
  - Choosing thresholds
  - Best practices for agentic loops
  - Next steps
- [Parallel outputs](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/parallel-outputs.md) - Produce several output formats at once with asyncio.gather.
  - The formatting functions
  - When to trace and when not to
  - Parallel execution with asyncio.gather
  - How asyncio.gather works
  - When to use asyncio.gather
  - Grouping parallel operations
  - Collecting outputs in a directory
  - The batch pipeline
  - Pipeline flow
  - Running the pipeline
  - Cost optimization tips
  - 1. Choose the right model
  - 2. Tune iteration parameters
  - 3. Use caching effectively
  - 4. Scale the batch
  - Next steps
- [Serving app](https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/serving-app.md) - Connect a serving app to a pipeline so it can display results and take input.
  - App environment configuration
  - Key configuration
  - Connecting to pipeline output with RunOutput
  - The Streamlit application
  - Displaying multiple reports
  - Generation instructions
  - Deploying the app
  - Workflow: Generate then view
  - Automatic updates with RunOutput
  - Complete example structure
  - Running the complete example
  - Summary

---
**Source**: https://github.com/unionai/unionai-docs/blob/main/content/user-guide/advanced-project/_index.md
**HTML**: https://www.union.ai/docs/v2/flyte/user-guide/advanced-project/