Why Blackshark.ai Chose Flyte™ to Support Earth’s ‘Digital Twin’

Blackshark.ai is balancing the world on its shoulders. The Austrian startup uses AI models and satellite imagery to generate real-time, accurate, searchable and photorealistic digital 3D maps of the entire planet. 

To get there, Blackshark extracts insights of the earth’s infrastructure from current satellite and aerial imagery using machine learning, and fills in missing attributes using AI for a photorealistic, geo-typical or asset-specific digital twin. (In one use case, Blackshark.ai enabled Microsoft’s Flight Simulator team to construct over 1.5 billion photorealistic 3D buildings, giving users an unprecedented immersive 3D flight experience.)

The work requires enormous amounts of data and staggering computing power. Blackshark currently runs its computations on hundreds of machines working on up to 2.5 petabytes of data. 

From the start of Blackshark’s trajectory, the internal team was solely responsible for handling the infrastructure, code and resources. Now scale presented fresh challenges: New models were being trained, people were doing explorations with novel techniques, and it all had to happen in parallel, which henceforth, posed challenges to Blackshark. 

“At a certain point, your company grows from from 30 to 100 employees, and there are few people who know how to work with the platform,” said Flyte™ Python developer Maarten de Jong. “It's just not scalable anymore.”

To get ahead of its growing data demands, Blackshark turned to Flyte™, the orchestration platform that powers Union. Today, Flyte™ runs the AI detection and content generation pieces of the Blackshark pipeline, effectively handling all the data the company can throw at it.

“The reason we’re mainly using Flyte™ is that it has cloud-native capabilities,” de Jong said. “We do everything in the cloud, and we don’t want to be limited to a single cloud provider, so having the ability to run everything through Kubernetes is amazing for us.”

Maarten De Jong from the Blackshark team explains they chose Flyte™ and how they use it in the following video:

Some noteworthy excerpts from the video:

“… the reason we’re mainly using Flyte™ is that it has cloud-native capabilities. We do everything in the cloud, and we don’t want to be limited to a single cloud provider, so having the ability to run everything through Kubernetes is amazing for us.”

“... it’s pretty robust. We haven’t really been able to break it too much, …”