Configure tasks

An LLM-optimized bundle of this entire section is available at section.md. This single file contains all pages in this section, optimized for AI coding agent context.

As we saw in Quickstart, you can run any Python function as a task in Flyte just by decorating it with @env.task.

This allows you to run your Python code in a distributed manner, with each function running in its own container. Flyte manages the spinning up of the containers, the execution of the code, and the passing of data between the tasks.

The simplest possible case is a TaskEnvironment with only a name parameter, and an env.task decorator, with no parameters:

task_config.py
                
env = flyte.TaskEnvironment(name="my_env")

@env.task
async def my_task(name:str) -> str:
    return f"Hello {name}!"

Notice how the TaskEnvironment is assigned to the variable env and then that variable is used in the @env.task. This is what connects the TaskEnvironment to the task definition.

In the following we will often use @env.task generically to refer to the decorator, but it is important to remember that it is actually a decorator attached to a specific TaskEnvironment object, and the env part can be any variable name you like.

This will run your task in the default container environment with default settings.

But, of course, one of the key advantages of Flyte is the ability to control the software environment, hardware environment, and other execution parameters for each task, right in your Python code. In this section we will explore the various configuration options available for tasks in Flyte.

Task configuration levels

Task configuration is done at three levels. From most general to most specific, they are:

The TaskEnvironment level: setting parameters when defining the TaskEnvironment object.
The @env.task decorator level: Setting parameters in the @env.task decorator when defining a task function.
The task invocation level: Using the task.override() method when invoking task execution.

Each level has its own set of parameters, and some parameters are shared across levels. For shared parameters, the more specific level will override the more general one.

Example

Here is an example of how these levels work together, showing each level with all available parameters:

task_config.py
                
                    
                
            
                
            
# Level 1: TaskEnvironment - Base configuration
env_2 = flyte.TaskEnvironment(
    name="data_processing_env",
    image=flyte.Image.from_debian_base(),
    resources=flyte.Resources(cpu=1, memory="512Mi"),
    env_vars={"MY_VAR": "value"},
    # secrets=flyte.Secret(key="openapi_key", as_env_var="MY_API_KEY"),
    cache="disable",
    # pod_template=my_pod_template,
    # reusable=flyte.ReusePolicy(replicas=2, idle_ttl=300),
    depends_on=[another_env],
    description="Data processing task environment",
    # plugin_config=my_plugin_config
)

# Level 2: Decorator - Override some environment settings
@env_2.task(
    short_name="process",
    # secrets=flyte.Secret(key="openapi_key", as_env_var="MY_API_KEY_2"),
    cache="auto",
    # pod_template=my_pod_template,
    report=True,
    max_inline_io_bytes=100 * 1024,
    retries=3,
    timeout=60,
    docs="This task processes data and generates a report."
)
async def process_data(data_path: str) -> str:
    return f"Processed {data_path}"

@env_2.task
async def invoke_process_data() -> str:
    result = await process_data.override(
        resources=flyte.Resources(cpu=4, memory="2Gi"),
        env_vars={"MY_VAR": "new_value"},
        # secrets=flyte.Secret(key="openapi_key", as_env_var="MY_API_KEY_3"),
        cache="auto",
        max_inline_io_bytes=100 * 1024,
        retries=3,
        timeout=60
    )("input.csv")
    return result

Parameter interaction

Here is an overview of all task configuration parameters available at each level and how they interact:

Parameter	`TaskEnvironment`	`@env.task` decorator	`override` on task invocation
name	✅ Yes (required)	❌ No	❌ No
short_name	❌ No	✅ Yes	✅ Yes
image	✅ Yes	❌ No	❌ No
resources	✅ Yes	❌ No	✅ Yes (if not `reusable`)
env_vars	✅ Yes	❌ No	✅ Yes (if not `reusable`)
secrets	✅ Yes	❌ No	✅ Yes (if not `reusable`)
cache	✅ Yes	✅ Yes	✅ Yes
pod_template	✅ Yes	✅ Yes	✅ Yes
reusable	✅ Yes	❌ No	✅ Yes
depends_on	✅ Yes	❌ No	❌ No
description	✅ Yes	❌ No	❌ No
plugin_config	✅ Yes	❌ No	❌ No
report	❌ No	✅ Yes	❌ No
max_inline_io_bytes	❌ No	✅ Yes	✅ Yes
retries	❌ No	✅ Yes	✅ Yes
timeout	❌ No	✅ Yes	✅ Yes
triggers	❌ No	✅ Yes	❌ No
links	❌ No	✅ Yes	✅ Yes
interruptible	✅ Yes	✅ Yes	✅ Yes
queue	✅ Yes	✅ Yes	✅ Yes
docs	❌ No	✅ Yes	❌ No

Task configuration parameters

Each parameter is documented in detail on its dedicated page or in the API reference. For full type signatures and constraints, see the TaskEnvironment API reference.

Parameter	Details
name, short_name, description, docs	Additional task settings
image	Container images • `Image` API ref
resources	Resources • `Resources` API ref
env_vars	Additional task settings
secrets	Secrets • `Secret` API ref
cache	Caching • `Cache` API ref
pod_template	Pod templates • `PodTemplate` API ref
reusable	Reusable containers • `ReusePolicy` API ref
depends_on	Multiple environments
plugin_config	Task plugins
report	Additional task settings
max_inline_io_bytes	Additional task settings
retries, timeout	Retries and timeouts • `RetryStrategy`, `Timeout` API refs
triggers	Triggers • `Trigger` API ref
links	Additional task settings
interruptible, queue	Interruptible tasks and queues