Projects and domains

Projects and domains are the principle organizational categories into which you group your workflows in Flyte.

Projects define groups of task, workflows, launch plans and other entities that share a functional purpose. Domains represent distinct steps through which the entities in a project transition as they proceed through the development cycle.

By default, Flyte provides three domains: development, staging, and production. During onboarding, you can configure your Flyte instance to have different domains. Speak to the Flyte team for more information.

Projects and domains are orthogonal to each other, meaning that a project has multiple domains and a domain has multiple projects.

Here is an example arrangement:

	Development	Staging	Production
Project 1	workflow_1 (v2.0)	workflow_1 (v1.0)	workflow_1 (v1.0)
Project 2	workflow_2 (v2.0)	workflow_2 (v1.0)	workflow_2 (v1.0)

Projects

Projects represent independent workflows related to specific teams, business areas, or applications. Each project is isolated from others, but workflows can reference entities (workflows or tasks) from other projects to reuse generalizable resources.

Domains

Domains represent distinct environments orthogonal to the set of projects in your org within Flyte, such as development, staging, and production. These enable dedicated configurations, permissions, secrets, cached execution history, and resource allocations for each environment, preventing unintended impact on other projects and/or domains.

Using domains allows for a clear separation between environments, helping ensure that development and testing don’t interfere with production workflows.

A production domain ensures a “clean slate” so that cached development executions do not result in unexpected behavior. Additionally, secrets may be configured for external production data sources.

When to use different Flyte projects?

Projects help group independent workflows related to specific teams, business areas, or applications. Generally speaking, each independent team or ML product should have its own Flyte project. Even though these are isolated from one another, teams may reference entities (workflows or tasks) from other Flyte projects to reuse generalizable resources. For example, one team may create a generalizable task to train common model types. However, this requires advanced collaboration and common coding standards.

When setting up workflows in Flyte, effective use of projects and domains is key to managing environments, permissions, and resource allocation. Below are best practices to consider when organizing workflows in Flyte.

Domains: Clear Environment Separation

Domains represent distinct environments within Flyte, allowing clear separation between development, staging, and production. This structure helps prevent cross-environment interference, ensuring that changes made in development or testing do not affect production workflows. Using domains for this separation ensures that workflows can evolve in a controlled manner across different stages, from initial development through to production deployment.

Projects: Organizing Workflows by Teams, Business Areas, or Applications

Projects in Flyte are designed to group independent workflows around specific teams, business functions, or applications. By aligning projects to organizational structure, you can simplify access control and permissions while encouraging a clean separation of workflows across different teams or use cases. Although workflows can reference each other across projects, it’s generally cleaner to maintain independent workflows within each project to avoid complexity.

Flyte’s CLI tools and SDKs provide options to specify projects and domains easily:

CLI Commands: In most commands within the pyflyte and flytectl CLIs, you can specify the project and domain by using the --project and --domain flags, enabling precise control over which project-domain pair a command applies to. More details here and here.

Python SDK: When working with the flytekit SDK, you can leverage FlyteRemote to define the project and domain for workflow interactions programmatically, ensuring that all actions occur in the intended environment. More details here.