Crunch data however you want. Automatically transition data from one dataframe type to the other using Structured Dataset.
Use HuggingFace dataset as a native Flyte™ type.
Visualize and explore big tabular datasets.
Use Polars dataframe as a native Flyte™ type.
Scale your Pandas workflows.
Validate data at every step of your Flyte™ workflow.
Databases & Data Warehouses
Manage and connect to databases and warehouses seamlessly.
Execute SQL queries as Flyte™ tasks.
Query a Snowflake service.
Query a Hive service.
Run intricate analytical queries with DuckDB.
Apply git-like versioning to your SQL databases.
Query a BigQuery table.
Process and analyze your data with data-crunchers.
Schedule, monitor and orchestrate Databricks jobs.
Transform data in your warehouses with DBT.
Run Spark jobs on ephemeral clusters.
Query an AWS Athena service.
Store, share and manage features for ML models.
Simplify the model training process.
Distributed Model Training
Perform distributed model training to speed up the model development process.
Connect to Ray cluster to perform distributed model training and hyperparameter tuning.
Run distributed PyTorch training jobs.
Run distributed TensorFlow training jobs.
Run distributed training with an MPI operator.
Run distributed deep learning workflows.
Run Dask jobs natively on a Kubernetes cluster.
Streamline the model deployment process.
Generate ONNX models from TensorFlow models.
ONNX Scikit Learn
Generate ONNX models from Scikit Learn models.
Generate ONNX models from PyTorch models.
Monitor data and models from within Flyte™.
Exercise greater control over Kubernetes resources.
Build your own integration
Create your own integration and submit it to the Flyte™ repository.