The machine learning team at Stash manages the personal finance app’s activity to protect the system from fraud, optimize model execution time and lower compute costs and time to market. They needed to replace their “homegrown” workflow management process with a more efficient, less costly solution.
When Lyft open-sourced Flyte™, it provided the market with a structured programming and distributed processing platform that it had already used to create highly concurrent, scalable and maintainable workflows. Flyte™ has been serving production-model training and data processing at Lyft for four years.
It has become the company’s preferred platform among teams running high-demand workflows including Pricing, Locations, Estimated Time of Arrivals (ETA), Mapping and Self-Driving. In fact, Flyte™ manages more than 10,000 unique workflows at Lyft, comprising monthly totals of more than 1 million executions, 20 million tasks and 40 million containers.
Flyte™’s primary programming interface is based in plain and simple Python. Configuration is done inline, no YAML is required, and workflows can be tested locally before shipping. Java and Scala SDKs are also available, and powerful programmatic access ships standard.
The Flyte™ Console gives you visibility into your workflows, tasks, schedules, executions, failures, timeouts, retries, and more. Track performance, view logs and DAG failures as you iterate on your model development.
Flyte™’s an ML orchestrator, and so is Kubeflow; Flyte™ has a Python SDK, and so does Kubeflow Pipelines; Flyte™’s built on Kubernetes, and so is Kubeflow. Look beyond these basics, however, and you’ll find Flyte™ and Kubeflow offer markedly different developer experiences and levels of effort to scale and deploy them. Get to know the differences in this compare-and-contrast article.
Kubernetes-native Flyte™ workflows are typesafe, directed graphs of independent tasks that are readily visualized and analyzed.
Flyte™ has been serving production-model training and data processing at Lyft for more than four years. The platform manages more than 10,000 unique workflows at Lyft. Every month, that’s more than 1 million executions, 20 million tasks and 40 million containers.
Blackshark.ai is an Austrian startup that generates photorealistic digital 3D maps of the entire planet. Processing up to 2.5 petabytes of content for 1.5 billion building footprints, Blackshark uses Flyte™ for AI detection and content generation.
Flyte™ is helping organizations like Spotify and Lyft build a new generation of products that make elegant use of complex data and machine learning. Now Union AI, the team behind Flyte™, has created a managed version of the workflow orchestrator, freeing data and ML teams from infrastructure constraints and setup.
Open-source, zero-configuration data testing and validation for more readable and robust data processing pipelines.
Unifying the vast ecosystem of ML tools into a single interface.
We’re building the production-grade Kubernetes-native ML and data orchestration platform.
