Preview Flyte 2 for production, hosted on Union.ai
Register now ↗
Flyte OSS Docs
Signup
User guide
Tutorials
Reference
Platform deployment
Integrations
Architecture
Community
Flyte OSS Docs
  • User guide
    • Introduction
    • Getting started
      • Local setup
      • First project
      • Understanding the code
      • Running your workflow
    • Core concepts
      • Workflows
        • Standard workflows
        • Subworkflows and sub-launch plans
        • Dynamic workflows
        • Imperative workflows
        • Launching workflows
        • Viewing workflows
        • Viewing workflow executions
      • Tasks
        • Map tasks
        • Task types
        • Task parameters
        • Launching tasks
        • Viewing tasks
        • Task software environment
          • Local image building
          • ImageSpec with ECR
          • ImageSpec with GAR
          • ImageSpec with ACR
          • Environment variables
        • Viewing logs
        • Reference tasks
        • Task hardware environment
          • Customizing task resources
          • Accelerators
          • Retries and timeouts
          • Interruptible instances
      • Launch plans
        • Defining launch plans
        • Viewing launch plans
        • Notifications
        • Schedules
        • Activating and deactivating
        • Running launch plans
        • Reference launch plans
        • Concurrency control
      • Caching
      • Named outputs
      • ImageSpec
    • Development cycle
      • Project structure
      • Projects and domains
      • Building workflows
      • Setting up a production project
      • Local dependencies
      • ImageSpec
      • Running your code
      • Overriding parameters
      • Run details
      • Debugging with interactive tasks
      • Task resource validation
      • Running in a local cluster
      • Jupyter notebooks
      • Decks
      • Migrating from Airflow to Flyte
      • Remote management
        • Remote examples
      • Testing
        • Mocking tasks
    • Data input/output
      • FlyteFile and FlyteDirectory
      • Downloading with FlyteFile and FlyteDirectory
      • Task input and output
      • Accessing attributes
      • Dataclass
      • Enum type
      • Pickle type
      • Pydantic BaseModel
      • PyTorch type
      • StructuredDataset
      • TensorFlow types
    • Programming
      • Chaining entities
      • Conditionals
      • Decorating tasks
      • Decorating workflows
      • Intratask checkpoints
      • Waiting for external inputs
      • Nested parallelism
      • Failure node
  • Tutorials
    • Bioinformatics
      • Nucleotide Sequence Querying with BLASTX
        • BLASTX example
    • Feature engineering
      • EDA, Feature Engineering, and Modeling with Papermill
        • EDA and feature engineering in Jupyter notebook and modeling in a Flyte task
        • EDA and feature engineering in one Jupyter notebook and modeling in the other
        • Flyte pipeline in one Jupyter notebook
        • Supermarket regression 1 notebook
        • Supermarket regression 2 notebook
        • Supermarket regression notebook
      • Feast integration
        • Feature engineering tasks
        • Flyte pipeline with Feast
        • How to trigger the Feast workflow using FlyteRemote
    • Flytelab
      • Weather forecasting
    • Model training
      • Diabetes classification
        • Train and Validate a diabetes classification XGBoost model
      • Forecasting Rossman store sales with Horovod and Spark
        • Data-parallel distributed training using Horovod on Spark
      • House price regression
        • Predicting house price in a region using XGBoost
        • Predicting house price in multiple regions using XGBoost and dynamic workflows
      • MNIST classification with PyTorch and W & B
        • Single N=node, multi-GPU training
        • Single node, single GPU training
      • NLP processing
        • Word E=embeddings and topic modelling with Gensim
  • Reference
    • LLM-optimized documentation
    • Pyflyte CLI
    • Flytectl CLI
      • flytectl
      • flytectl version
      • flytectl append
        • flytectl append identityassignments
      • flytectl apply
        • flytectl apply app
        • flytectl apply clusterconfig
        • flytectl apply clusterconfigid
        • flytectl apply clusterpoolconfig
      • flytectl config
        • flytectl config discover
        • flytectl config docs
        • flytectl config init
        • flytectl config validate
      • flytectl create
        • flytectl create app
        • flytectl create clusterpool
        • flytectl create clusterpoolassignment
        • flytectl create execution
        • flytectl create policy
        • flytectl create project
        • flytectl create role
      • flytectl delete
        • flytectl delete app
        • flytectl delete cluster
        • flytectl delete cluster-pool-attributes
        • flytectl delete cluster-resource-attribute
        • flytectl delete clusterconfig
        • flytectl delete clusterpool
        • flytectl delete clusterpoolassignment
        • flytectl delete execution
        • flytectl delete execution-cluster-label
        • flytectl delete execution-queue-attribute
        • flytectl delete identityassignments
        • flytectl delete plugin-override
        • flytectl delete policy
        • flytectl delete role
        • flytectl delete task-resource-attribute
        • flytectl delete workflow-execution-config
      • flytectl demo
        • flytectl demo exec
        • flytectl demo reload
        • flytectl demo start
        • flytectl demo status
        • flytectl demo teardown
      • flytectl get
        • flytectl get app
        • flytectl get cluster
        • flytectl get cluster-pool-attributes
        • flytectl get cluster-resource-attribute
        • flytectl get clusterconfig
        • flytectl get clusterconfigs
        • flytectl get clusterpool
        • flytectl get clusterpoolconfig
        • flytectl get clusterswithconfig
        • flytectl get echo
        • flytectl get execution
        • flytectl get execution-cluster-label
        • flytectl get execution-queue-attribute
        • flytectl get executionoperation
        • flytectl get identityassignment
        • flytectl get launchplan
        • flytectl get plugin-override
        • flytectl get policy
        • flytectl get project
        • flytectl get role
        • flytectl get task
        • flytectl get task-resource-attribute
        • flytectl get workflow
        • flytectl get workflow-execution-config
      • flytectl register
        • flytectl register examples
        • flytectl register files
      • flytectl update
        • flytectl update cluster-pool-attributes
        • flytectl update cluster-resource-attribute
        • flytectl update execution
        • flytectl update execution-cluster-label
        • flytectl update execution-queue-attribute
        • flytectl update launchplan
        • flytectl update launchplan-meta
        • flytectl update plugin-override
        • flytectl update project
        • flytectl update task-meta
        • flytectl update task-resource-attribute
        • flytectl update workflow-execution-config
        • flytectl update workflow-meta
    • Flyteidl
    • FlyteKit Plugins
      • Apache Airflow
        • Classes
        • Packages
          • flytekitplugins.airflow.connector
          • flytekitplugins.airflow.task
      • Async FSSpec
        • Classes
        • Packages
          • flytekitplugins.async_fsspec.s3fs.constants
          • flytekitplugins.async_fsspec.s3fs.s3fs
      • AWS Athena
        • AWS Batch
          • AWS SageMaker
            • BigQuery
              • Classes
              • Packages
                • flytekitplugins.bigquery.connector
                • flytekitplugins.bigquery.task
            • Comet ML
              • Dask
                • DBT
                  • Deck
                    • Dolt
                      • DuckDB
                        • Envd
                          • Flyte Interactive
                            • FSSpec
                              • Google IAP
                                • Classes
                                • Packages
                                  • flytekitplugins.identity_aware_proxy.cli
                              • Great Expectations
                                • Hive
                                  • Classes
                                  • Packages
                                    • flytekitplugins.hive.task
                                • Hugging Face
                                  • Classes
                                  • Packages
                                    • flytekitplugins.huggingface.sd_transformers
                                • Inference
                                  • Classes
                                  • Packages
                                    • flytekitplugins.inference.nim.serve
                                    • flytekitplugins.inference.ollama.serve
                                    • flytekitplugins.inference.sidecar_template
                                    • flytekitplugins.inference.vllm.serve
                                • Kubeflow MPI
                                  • Classes
                                  • Packages
                                    • flytekitplugins.kfmpi.task
                                • Kubeflow PyTorch
                                  • Classes
                                  • Packages
                                    • flytekitplugins.kfpytorch.error_handling
                                    • flytekitplugins.kfpytorch.pod_template
                                    • flytekitplugins.kfpytorch.task
                                • Kubeflow TensorFlow
                                  • Classes
                                  • Packages
                                    • flytekitplugins.kftensorflow.task
                                • Kubernetes Pod
                                  • Classes
                                  • Packages
                                    • flytekitplugins.pod.task
                                • Memory Machine Cloud
                                  • Classes
                                  • Packages
                                    • flytekitplugins.mmcloud.connector
                                    • flytekitplugins.mmcloud.task
                                    • flytekitplugins.mmcloud.utils
                                • Memray Profiling
                                  • Classes
                                  • Packages
                                    • flytekitplugins.memray
                                • MLflow
                                  • Modin
                                    • Neptune
                                      • flytekitplugins.neptune
                                    • OmegaConf
                                      • Classes
                                      • Packages
                                        • flytekitplugins.omegaconf
                                    • ONNX PyTorch
                                      • ONNX ScikitLearn
                                        • Classes
                                        • Packages
                                          • flytekitplugins.onnxscikitlearn.schema
                                      • ONNX TensorFlow
                                        • OpenAI
                                          • Classes
                                          • Packages
                                            • flytekitplugins.openai.batch.connector
                                            • flytekitplugins.openai.batch.task
                                            • flytekitplugins.openai.batch.workflow
                                            • flytekitplugins.openai.chatgpt.connector
                                            • flytekitplugins.openai.chatgpt.task
                                        • Optuna
                                          • Classes
                                          • Packages
                                            • flytekitplugins.optuna
                                        • Pandera
                                          • Classes
                                          • Packages
                                            • flytekitplugins.pandera.config
                                            • flytekitplugins.pandera.pandas_renderer
                                            • flytekitplugins.pandera.pandas_transformer
                                        • Papermill
                                          • Classes
                                          • Packages
                                            • flytekitplugins.papermill.task
                                        • Perian Job Platform
                                          • Classes
                                          • Packages
                                            • flytekitplugins.perian_job.agent
                                            • flytekitplugins.perian_job.task
                                        • Polars
                                          • Classes
                                          • Packages
                                            • flytekitplugins.polars.sd_transformers
                                        • Ray
                                          • Classes
                                          • Packages
                                            • flytekitplugins.ray.models
                                            • flytekitplugins.ray.task
                                        • Slurm
                                          • Classes
                                          • Packages
                                            • flytekitplugins.slurm.ssh_utils
                                        • Snowflake
                                          • Classes
                                          • Packages
                                            • flytekitplugins.snowflake.connector
                                            • flytekitplugins.snowflake.task
                                        • Spark
                                          • Classes
                                          • Packages
                                            • flytekitplugins.spark.connector
                                            • flytekitplugins.spark.generic_task
                                            • flytekitplugins.spark.models
                                            • flytekitplugins.spark.pyspark_transformers
                                            • flytekitplugins.spark.schema
                                            • flytekitplugins.spark.sd_transformers
                                            • flytekitplugins.spark.task
                                            • flytekitplugins.spark.utils
                                        • SQLAlchemy
                                          • Classes
                                          • Packages
                                            • flytekitplugins.sqlalchemy.task
                                        • Vaex
                                          • Weights & Biases
                                            • Classes
                                            • Packages
                                              • flytekitplugins.wandb
                                          • whylogs
                                            • Classes
                                            • Packages
                                              • flytekitplugins.whylogs
                                        • Flytekit SDK
                                          • Classes & Protocols
                                          • Packages
                                            • flytekit
                                            • flytekit.bin.entrypoint
                                            • flytekit.clients.auth_helper
                                            • flytekit.clients.auth.auth_client
                                            • flytekit.clients.auth.authenticator
                                            • flytekit.clients.auth.default_html
                                            • flytekit.clients.auth.exceptions
                                            • flytekit.clients.auth.keyring
                                            • flytekit.clients.auth.token_client
                                            • flytekit.clients.friendly
                                            • flytekit.clients.grpc_utils.auth_interceptor
                                            • flytekit.clients.grpc_utils.default_metadata_interceptor
                                            • flytekit.clients.grpc_utils.wrap_exception_interceptor
                                            • flytekit.clients.helpers
                                            • flytekit.clients.raw
                                            • flytekit.clis.helpers
                                            • flytekit.clis.sdk_in_container.backfill
                                            • flytekit.clis.sdk_in_container.build
                                            • flytekit.clis.sdk_in_container.constants
                                            • flytekit.clis.sdk_in_container.executions
                                            • flytekit.clis.sdk_in_container.helpers
                                            • flytekit.clis.sdk_in_container.metrics
                                            • flytekit.clis.sdk_in_container.package
                                            • flytekit.clis.sdk_in_container.pyflyte
                                            • flytekit.clis.sdk_in_container.run
                                            • flytekit.clis.sdk_in_container.serialize
                                            • flytekit.clis.sdk_in_container.serve
                                            • flytekit.clis.sdk_in_container.utils
                                            • flytekit.clis.version
                                            • flytekit.configuration
                                            • flytekit.configuration.default_images
                                            • flytekit.configuration.feature_flags
                                            • flytekit.configuration.file
                                            • flytekit.configuration.internal
                                            • flytekit.configuration.plugin
                                            • flytekit.constants
                                            • flytekit.core.annotation
                                            • flytekit.core.array_node
                                            • flytekit.core.array_node_map_task
                                            • flytekit.core.artifact
                                            • flytekit.core.artifact_utils
                                            • flytekit.core.base_sql_task
                                            • flytekit.core.base_task
                                            • flytekit.core.cache
                                            • flytekit.core.checkpointer
                                            • flytekit.core.class_based_resolver
                                            • flytekit.core.condition
                                            • flytekit.core.constants
                                            • flytekit.core.container_task
                                            • flytekit.core.context_manager
                                            • flytekit.core.data_persistence
                                            • flytekit.core.docstring
                                            • flytekit.core.environment
                                            • flytekit.core.gate
                                            • flytekit.core.hash
                                            • flytekit.core.interface
                                            • flytekit.core.launch_plan
                                            • flytekit.core.legacy_map_task
                                            • flytekit.core.local_cache
                                            • flytekit.core.local_fsspec
                                            • flytekit.core.mock_stats
                                            • flytekit.core.node
                                            • flytekit.core.node_creation
                                            • flytekit.core.notification
                                            • flytekit.core.options
                                            • flytekit.core.pod_template
                                            • flytekit.core.promise
                                            • flytekit.core.python_auto_container
                                            • flytekit.core.python_customized_container_task
                                            • flytekit.core.python_function_task
                                            • flytekit.core.reference
                                            • flytekit.core.reference_entity
                                            • flytekit.core.resources
                                            • flytekit.core.schedule
                                            • flytekit.core.shim_task
                                            • flytekit.core.task
                                            • flytekit.core.testing
                                            • flytekit.core.tracked_abc
                                            • flytekit.core.tracker
                                            • flytekit.core.type_engine
                                            • flytekit.core.type_helpers
                                            • flytekit.core.type_match_checking
                                            • flytekit.core.utils
                                            • flytekit.core.worker_queue
                                            • flytekit.core.workflow
                                            • flytekit.deck.deck
                                            • flytekit.deck.renderer
                                            • flytekit.exceptions.base
                                            • flytekit.exceptions.eager
                                            • flytekit.exceptions.scopes
                                            • flytekit.exceptions.system
                                            • flytekit.exceptions.user
                                            • flytekit.exceptions.utils
                                            • flytekit.experimental.eager_function
                                            • flytekit.extend.backend.base_connector
                                            • flytekit.extend.backend.connector_service
                                            • flytekit.extend.backend.utils
                                            • flytekit.extras.accelerators
                                            • flytekit.extras.cloud_pickle_resolver
                                            • flytekit.extras.pydantic_transformer.transformer
                                            • flytekit.extras.sklearn.native
                                            • flytekit.extras.sqlite3.task
                                            • flytekit.extras.tasks.shell
                                            • flytekit.extras.webhook
                                            • flytekit.extras.webhook.connector
                                            • flytekit.extras.webhook.constants
                                            • flytekit.extras.webhook.task
                                            • flytekit.image_spec.default_builder
                                            • flytekit.image_spec.image_spec
                                            • flytekit.image_spec.noop_builder
                                            • flytekit.interaction.click_types
                                            • flytekit.interaction.parse_stdin
                                            • flytekit.interaction.rich_utils
                                            • flytekit.interaction.string_literals
                                            • flytekit.interactive
                                            • flytekit.interactive.constants
                                            • flytekit.interactive.utils
                                            • flytekit.interactive.vscode_lib.config
                                            • flytekit.interactive.vscode_lib.decorator
                                            • flytekit.interactive.vscode_lib.vscode_constants
                                            • flytekit.interfaces.cli_identifiers
                                            • flytekit.interfaces.random
                                            • flytekit.interfaces.stats.client
                                            • flytekit.interfaces.stats.taggable
                                            • flytekit.lazy_import.lazy_module
                                            • flytekit.loggers
                                            • flytekit.models.admin.common
                                            • flytekit.models.admin.task_execution
                                            • flytekit.models.admin.workflow
                                            • flytekit.models.annotation
                                            • flytekit.models.array_job
                                            • flytekit.models.common
                                            • flytekit.models.concurrency
                                            • flytekit.models.core.catalog
                                            • flytekit.models.core.compiler
                                            • flytekit.models.core.condition
                                            • flytekit.models.core.errors
                                            • flytekit.models.core.execution
                                            • flytekit.models.core.identifier
                                            • flytekit.models.core.types
                                            • flytekit.models.core.workflow
                                            • flytekit.models.documentation
                                            • flytekit.models.domain
                                            • flytekit.models.dynamic_job
                                            • flytekit.models.event
                                            • flytekit.models.execution
                                            • flytekit.models.filters
                                            • flytekit.models.interface
                                            • flytekit.models.launch_plan
                                            • flytekit.models.literals
                                            • flytekit.models.matchable_resource
                                            • flytekit.models.named_entity
                                            • flytekit.models.node_execution
                                            • flytekit.models.presto
                                            • flytekit.models.project
                                            • flytekit.models.qubole
                                            • flytekit.models.schedule
                                            • flytekit.models.security
                                            • flytekit.models.task
                                            • flytekit.models.types
                                            • flytekit.models.workflow_closure
                                            • flytekit.remote.backfill
                                            • flytekit.remote.data
                                            • flytekit.remote.entities
                                            • flytekit.remote.executions
                                            • flytekit.remote.interface
                                            • flytekit.remote.lazy_entity
                                            • flytekit.remote.metrics
                                            • flytekit.remote.remote
                                            • flytekit.remote.remote_callable
                                            • flytekit.remote.remote_fs
                                            • flytekit.sensor.base_sensor
                                            • flytekit.sensor.file_sensor
                                            • flytekit.sensor.sensor_engine
                                            • flytekit.tools.fast_registration
                                            • flytekit.tools.ignore
                                            • flytekit.tools.interactive
                                            • flytekit.tools.module_loader
                                            • flytekit.tools.repo
                                            • flytekit.tools.script_mode
                                            • flytekit.tools.serialize_helpers
                                            • flytekit.tools.subprocess
                                            • flytekit.tools.translator
                                            • flytekit.types.directory
                                            • flytekit.types.directory.types
                                            • flytekit.types.error.error
                                            • flytekit.types.file
                                            • flytekit.types.file.file
                                            • flytekit.types.file.image
                                            • flytekit.types.iterator.iterator
                                            • flytekit.types.iterator.json_iterator
                                            • flytekit.types.numpy.ndarray
                                            • flytekit.types.pickle.pickle
                                            • flytekit.types.schema.types
                                            • flytekit.types.schema.types_pandas
                                            • flytekit.types.structured
                                            • flytekit.types.structured.basic_dfs
                                            • flytekit.types.structured.bigquery
                                            • flytekit.types.structured.snowflake
                                            • flytekit.types.structured.structured_dataset
                                            • flytekit.utils.asyn
                                            • flytekit.utils.dict_formatter
                                            • flytekit.utils.pbhash
                                            • flytekit.utils.rate_limiter
                                      • Platform deployment
                                        • Flyte deployment
                                          • Planning your deployment
                                          • Installing Flyte
                                          • Multi-cluster
                                        • Platform configuration
                                          • Authentication
                                          • Monitoring
                                          • Logging links in the UI
                                          • Access to GPUs
                                          • Using Pod Templates
                                          • Cloud events
                                          • Customizing resources
                                          • Platform events
                                          • Workflow notifications
                                          • Optimizing performance
                                          • Resource Manager
                                          • Secrets
                                          • Security overview
                                          • API playground
                                        • Connector setup
                                          • Airflow connector
                                          • BigQuery connector
                                          • ChatGPT connector
                                          • Databricks connector
                                          • K8s Data Service connector
                                          • MMCloud connector
                                          • OpenAI Batch connector
                                          • Sagemaker inference connector
                                          • Sensor connector
                                          • SLURM connector
                                          • Snowflake connector
                                          • DGXC Lepton connector
                                        • Plugins setup
                                          • Kubernetes plugins
                                          • AWS Athena plugin
                                          • AWS Batch plugin
                                          • AWS Sagemaker plugin
                                          • BigQuery plugin
                                          • Databricks plugin
                                          • Snowflake plugin
                                        • Configuration reference
                                          • Datacatalog configuration
                                          • FlyteAdmin configuration
                                          • FlytePropeller configuration
                                          • FlyteScheduler configuration
                                      • Integrations
                                        • Connectors
                                          • Airflow connector
                                            • Airflow connector example usage
                                          • BigQuery connector
                                            • BigQuery connector example usage
                                          • ChatGPT connector
                                            • ChatGPT connector
                                          • Databricks connector
                                            • Running Spark on Databricks
                                          • Memory Machine Cloud connector
                                            • Memory Machine Cloud connector example usage
                                          • OpenAI Batch connector
                                            • Batching Requests for Asynchronous Processing
                                          • Perian connector
                                            • Perian connector example usage
                                          • SageMaker connector
                                            • Deploy and serve an XGBoost model on AWS SageMaker using FastAPI
                                          • Sensor connectror
                                            • File Sensor
                                          • Slurm connector
                                            • Slurm connector example usage
                                          • Snowflake connector
                                            • Querying data in Snowflake
                                        • Flytekit plugins
                                          • Comet ML
                                            • Comet Example
                                          • DBT
                                            • DBT Example
                                          • Dolt
                                            • Dolt Branches
                                            • Quickstart
                                          • Great Expectations
                                            • Task Example
                                            • Type Example
                                          • Memray Profiling
                                            • Memray Profiling Example
                                          • MLflow
                                            • MLflow Example
                                          • Modin
                                            • KNN Classifier
                                          • Neptune plugin
                                            • Neptune example
                                          • NIM
                                            • Serve Generative AI Models with NIM
                                          • Ollama
                                            • Serve LLMs with Ollama
                                          • ONNX
                                            • PyTorch Example
                                            • ScikitLearn Example
                                            • TensorFlow Example
                                          • Pandera
                                            • Basic Schema Example
                                            • Validating and Testing Machine Learning Pipelines
                                          • Papermill
                                            • Jupyter Notebook Tasks
                                            • Simple Papermill Notebook
                                          • Slurm connector
                                            • DuckDB Example
                                          • SQL
                                            • SQLAlchemy
                                            • Sqlite3
                                          • Weights and Biases
                                            • Weights and Biases Example
                                          • WhyLogs
                                            • Whylogs Example
                                        • Native backend plugins
                                          • Dask
                                            • Running a Dask Task
                                          • MPI
                                            • Running Distributed Training Using Horovod and MPI
                                          • PyTorch Distributed
                                            • Run PyTorch Distributed
                                            • Use PyTorch Lightning to Train an MNIST Autoencoder
                                          • Ray
                                            • Running Ray Tasks
                                          • Spark
                                            • Converting a Spark DataFrame to a Pandas DataFrame
                                            • Running a Spark Task
                                          • TensorFlow Distributed
                                            • Run Distributed TensorFlow Training
                                        • External service backend plugins
                                          • AWS Athena
                                            • Athena Query
                                          • AWS Batch
                                            • AWS Batch
                                          • FlyteInteractive
                                            • FlyteInteractive Jupyter Decorator
                                            • FlyteInteractive Vscode Decorator
                                          • Hive
                                            • Hive Tasks
                                        • Flyte operators
                                          • Airflow Provider
                                            • FlyteOperator Example
                                        • Deprecated integrations
                                          • BigQuery plugin
                                            • BigQuery plugin example
                                          • Databricks plugin
                                            • Databricks plugin example
                                          • Kubernetes Pods
                                            • Pod Example
                                          • Snowflake plugin
                                            • Snowflake plugin example
                                      • Architecture
                                        • Registration
                                        • Executions
                                        • Workflow state transitions
                                        • Workflow timeline
                                        • Data handling
                                        • Data catalog
                                        • Versions
                                        • Workflow lifecycle
                                        • Component Architecture
                                          • FlytePropeller architecture
                                          • Flyte Native Scheduler Architecture
                                        • Control Plane
                                          • Projects
                                          • Domains
                                          • Flyte Admin
                                          • FlyteConsole
                                          • Dynamic Job Spec
                                        • Extending Flyte
                                          • Custom types
                                          • Prebuilt container task plugins
                                          • User container task plugins
                                          • Backend plugins
                                          • Container interface
                                      • Community
                                        • Joining the community
                                        • Contributing code
                                        • Contributing docs and examples
                                          • Quick start
                                          • Variants
                                          • Versions
                                          • Authoring
                                          • Shortcodes
                                          • Redirects
                                          • API docs
                                          • Publishing
                                        • Roadmap
                                        • Troubeshooting
                                      Documentation navigation for Flyte including user guides, tutorials, API references, and integration examples
                                      • Connectors
                                        • Airflow connector
                                          • Airflow connector example usage
                                        • BigQuery connector
                                          • BigQuery connector example usage
                                        • ChatGPT connector
                                          • ChatGPT connector
                                        • Databricks connector
                                          • Running Spark on Databricks
                                        • Memory Machine Cloud connector
                                          • Memory Machine Cloud connector example usage
                                        • OpenAI Batch connector
                                          • Batching Requests for Asynchronous Processing
                                        • Perian connector
                                          • Perian connector example usage
                                        • SageMaker connector
                                          • Deploy and serve an XGBoost model on AWS SageMaker using FastAPI
                                        • Sensor connectror
                                          • File Sensor
                                        • Slurm connector
                                          • Slurm connector example usage
                                        • Snowflake connector
                                          • Querying data in Snowflake
                                      • Flytekit plugins
                                        • Comet ML
                                          • Comet Example
                                        • DBT
                                          • DBT Example
                                        • Dolt
                                          • Dolt Branches
                                          • Quickstart
                                        • Great Expectations
                                          • Task Example
                                          • Type Example
                                        • Memray Profiling
                                          • Memray Profiling Example
                                        • MLflow
                                          • MLflow Example
                                        • Modin
                                          • KNN Classifier
                                        • Neptune plugin
                                          • Neptune example
                                        • NIM
                                          • Serve Generative AI Models with NIM
                                        • Ollama
                                          • Serve LLMs with Ollama
                                        • ONNX
                                          • PyTorch Example
                                          • ScikitLearn Example
                                          • TensorFlow Example
                                        • Pandera
                                          • Basic Schema Example
                                          • Validating and Testing Machine Learning Pipelines
                                        • Papermill
                                          • Jupyter Notebook Tasks
                                          • Simple Papermill Notebook
                                        • Slurm connector
                                          • DuckDB Example
                                        • SQL
                                          • SQLAlchemy
                                          • Sqlite3
                                        • Weights and Biases
                                          • Weights and Biases Example
                                        • WhyLogs
                                          • Whylogs Example
                                      • Native backend plugins
                                        • Dask
                                          • Running a Dask Task
                                        • MPI
                                          • Running Distributed Training Using Horovod and MPI
                                        • PyTorch Distributed
                                          • Run PyTorch Distributed
                                          • Use PyTorch Lightning to Train an MNIST Autoencoder
                                        • Ray
                                          • Running Ray Tasks
                                        • Spark
                                          • Converting a Spark DataFrame to a Pandas DataFrame
                                          • Running a Spark Task
                                        • TensorFlow Distributed
                                          • Run Distributed TensorFlow Training
                                      • External service backend plugins
                                        • AWS Athena
                                          • Athena Query
                                        • AWS Batch
                                          • AWS Batch
                                        • FlyteInteractive
                                          • FlyteInteractive Jupyter Decorator
                                          • FlyteInteractive Vscode Decorator
                                        • Hive
                                          • Hive Tasks
                                      • Flyte operators
                                        • Airflow Provider
                                          • FlyteOperator Example
                                      • Deprecated integrations
                                        • BigQuery plugin
                                          • BigQuery plugin example
                                        • Databricks plugin
                                          • Databricks plugin example
                                        • Kubernetes Pods
                                          • Pod Example
                                        • Snowflake plugin
                                          • Snowflake plugin example

                                      The page you navigated to () does not exist, so we brought you to the closest page to it.

                                      You have switched from the to the variant of this site. There is no equivalent of . We have taken you to the closest page in the variant.

                                      Flyte OSS
                                      Integrations
                                      Flyte operators

                                      Flyte operators

                                      This section covers Flyte operators.

                                      ← Hive TasksAirflow Provider →
                                      LLM-optimized
                                      This page Full docs index
                                      On this page