How we coded the world’s coolest ocean drone
Blueye Pioneer is the world’s first ocean drone for consumers. Serving as a digital diving mask, it allows people to explore the world beneath the surface of the sea using their smartphone or tablet, and then share high-quality videos online. As part of an integrated team of designers and developers at Blueye Robotics, my colleague Alexander Vanvik and I, Martin Rechsteiner, helped design and code the app that controls and interacts with Blueye Pioneer. Here’s how we did it...
We were involved in the development process of Blueye Pioneer from the very beginning. Our mission: to create an intuitive way of controlling it. We knew that the drone should be controlled by users’ smartphones, but we weren’t precisely sure how. Whether it was best to control it using a hand-controller, by touch, a head-mounted display or other solutions. Everything was completely open. Therefore, we spent a lot of time prototyping different ways of operating the drone, including many types of input mechanisms. What we learned from all this prototyping and testing formed the basis for the first iterations of the app.
I’ve lost count of how many different types of technical solutions we prototyped and tested before we landed on the solution we are now refining.
We tested in two ways: first, we took a boat out and experimented with the web-based prototype that Blueye had already created. This gave us an opportunity to gain experience operating the drone as well as iterating on their existing prototype. Secondly, since we didn’t always have access to a physical drone, we created a simulator in Unreal Engine (a game engine used to develop many high-end game titles). This enabled us to prototype and test a lot from our desks at EGGS, where we could not only iterate interaction principles and user interface elements but also try out different ways of interacting with the drone. Furthermore, we could see the impact on viewing by adjusting the placement and angle of the camera on it.
When it was time to start developing the actual app, we spent many hours with Blueye’s chief software developer discussing what technology stack to use. We knew that we needed to support both iOS and Android, so we looked at numerous cross platform options. We didn’t want the user experience to suffer from our choice of framework, so anything that didn’t allow us to use native interface elements wasn’t considered an option. This basically left us with three alternatives: to use React-native, Xamarin or just create native implementations for both iOS and Android.
We quickly decided that React-native probably wouldn’t give us the fine-grained control we needed without having to tap into the native code anyway. Because Blueye already had experience using Xamarin, and we knew that a lot of the same functionality had to be written for both apps if we were to go completely native, we decided that Xamarin would be the most suitable fit.
With Xamarin you can write apps in C#, that compile down to native iOS and Android apps. This meant that we could develop a common core that was shared between platforms, but still have the fine-grained control of writing native interface elements. Things like video transfer, drone communication, video uploading etc. This turned out to be a smart move because it saved us a lot of time by not having to double the programming work.
Blueye plays a significant role in the development of the drone. We’ve also worked as part of a tight-knit team of product designers, especially on the simulation and testing of the thrusters, camera positioning and different modes of control and steering. Because Blueye has a solid, top-level design strategy that infiltrates all aspects of the product, there has been a well-founded fusion between branding, product design, strategy, and development. This allows us to create a seamless experience for the user.
We’ve developed and innovated in teams, closely interacting with both product designers and digital designers in iterative processes.
Today, we have a fully functioning iOS app, along with having smartly prepared for the development of the Android app. We’re now focusing on the social aspect of using Blueye Pioneer. The intention is to create a Blueye user community within the app - a social forum where users can share film footage, interact live via video stream and then be able to store recorded material in the cloud. With an official worldwide launch planned for spring 2018, Blueye Pioneer is available to pre-order now.
Groundbreaking technology to get your eyes below the surface