How Kineo achieves business goals with Flyte™
The company
“We really think that there's business value using AI in every industry,” said Jan Fiedler, MLOps engineer with Kineo. Helping companies find that value is the mission of Berlin-based Kineo. “We provide customized AI solutions — not only to one specific industry, but all the industries.”
Providing AI as a service requires nimble problem-solving from Kineo’s team of data scientists, whether they’re applying machine learning to forecast global sales of automatic door openers or extrapolating data about press times to tighten quality control for a manufacturer of decorative wooden boards.
Fiedler’s job is to empower those Kineo scientists with a robust infrastructure for AI workflows. “I'm not developing any machine learning solutions myself,” he explained. “It's my goal to provide the best tools so that it's as pleasant as possible for my colleagues to develop machine learning solutions and deploy them as well.”
Those tools include a platform for orchestrating the complex software tasks Kineo’s bespoke AI solutions demand. And when Kineo found itself hampered by the limitations of its previous orchestrator, it launched a search that brought it to Flyte.
The challenge
Fiedler said that after about two years working with the Kubeflow orchestrator, Kineo found itself contending with a number of pain points that impeded its operation:
No local execution. To test the processes its data scientists created, Kineo had to schedule development runs to a large cloud-based Kubernetes cluster instead of running the code locally. This expensive and cumbersome process also required MLOps support, limiting the flexibility of Kineo’s data scientists.
Notebook code != pipeline code. Kineo develops its products locally on the Jupyter Notebook. In fact, Fiedler said, the company runs its first client-facing experiments on the notebook to demonstrate business value. With Kubeflow, putting this code into production required time-consuming modifications. “Just this conversion of notebook code to pipeline code was taking a lot of a lot of time, doing really simple, stupid stuff like if-else statements and loops,” he said. “It was not like learning a new language, but it took us time.”
No ML type support. Kubeflow enforces a type system, but it doesn’t support data types beyond fundamental Python types and artifacts/files.
The solution
With these four pain points in mind, Kineo embarked on a search for an orchestrator that supported local development that could be deployed seamlessly to the cloud.
The team narrowed the search down to two candidates that fit the bill — Prefect and Flyte — and created a spreadsheet to do a comprehensive comparison of both orchestrators’ feature sets compared with Kubeflow. Among the factors in Flyte’s favor, Fiedler said, was team members’ familiarity with Flyte and the simplicity of creating pipeline code. Automated testing scored another point in Flyte’s favor. “In Kubeflow, we were able to write unit tests, not to test the whole pipeline or workflow locally.”
Another major factor in Flyte’s favor was its rich assortment of components. “In Flyte, there are already a lot of functionalities or tools included in Flyte itself. With Kubeflow, we had to develop Kubeflow components.” When the team researched Flyte, by contrast, “we thought, ‘OK, a lot of things that we actually want to use in a workflow on the pipeline are already covered by Flyte.’”
“Because Flyte is so much more lightweight than Kubeflow, Kineo has cut the cost of AWS base operations by 50%. More important are the cost savings of being able to develop locally.”
Fiedler credited the Flyte community with helping get Kineo up and running quickly. “It’s amazing. I think we figured out how to deploy Flyte in a couple of weeks, really, but this could only happen because, if we asked a question of the Flyte community, we could receive the answer in a couple of minutes. So this was huge.”
The results
According to Fiedler, Kineo is already to quantify operational savings as a result of switching to Flyte. “Even with our orchestration tool always online, we expect the overall costs to be roughly 50% of the previous setup.”
“Ninety percent of our projects are happening in the cloud,” he said, Because Flyte is so much more lightweight than Kubeflow, Kineo has cut the cost of AWS base operations by 50%. More important are the cost savings of being able to develop locally. “Just being able to develop fully locally, our AWS costs drop drastically.” He shared an analysis of the cost of developing a data pipeline for two weeks:
‘This obviously also scales with developing machine learning forecasts. I'm not saying that all costs go down to zero now, but what we’ve already seen from the team is that we develop with a subset of data, make use of the great caching of tasks, and then only really in the last day of a project we take the workflow online. That also goes really really fast.
“Looking at a two-week sprint, reducing the need of provisioning a Kubernetes cluster in the cloud from 10 days to one or two comes with great cost reduction,” Fiedler continued. “When developing data workflows, we were able to observe a cost reduction from $160 to $39 (roughly 70%).
“In addition to costs for cloud resources we also decrease costs for personnel resources,” he said. “People just migrate Kubeflow pipelines in a couple of days. ... They’re adapting to Flyte super, super quickly because we're just dealing with Python. It just enables the engineers to be much much faster.”