A playful way of flying a drone using Augmented Reality.
Aquila is a project that I thought of doing to exercise my skills, learn something new and to push some boundaries in the process. For my readers, this is not a tutorial, rather a journey through my experience creating Aquila. Also, a disclaimer, I don’t claim this is the best of works, but a really good learning experience.
Aquila is Latin for Eagle. Being a hardcore fan of Assassins Creed, and dreaming about having a pet eagle someday, I thought of exploring how virtual pets could be used for interactive technology.
As like any project, being a designer, I started with sketches. Lot of them. The understanding of the look and feel of the eagle, its motion and details are something which will come out as a result of such sketches. What I realised quickly is that the eagle has a simple yet beautiful and elegant form. I especially love the beak.
Once all these sketches were done, I used some of them as reference images for sculpting. Mind you, it has been ages since I sculpted and it took a while to get up to speed. I was looking at a simple model, low in polygon count, but not having a low-poly aesthetic. I had a hard time deciding whether to poly-model it or sculpt it. Sculpting is easier and I daresay more creative. But poly-modelling gives you a better mesh for painting and animation. I went the sculpting route as I was interested in honing my artistic skills. Again, I daresay I succeeded. (I could hear a professional artist saying WTF at my work :P ). I started off with ZBrush. ZBrush used to be one of my favourite tools in the past, but opening ZBrush after ages gave me a headache. The interface is anything but beginner friendly. Also it crashed a lot adding to my frustration (maybe I did things wrong?). I quickly switched to Autodesk Mudbox. Although it had simple tools, it suited my needs. And I started making the basic form.
Once the basic form was made, I moved into detailing. I left the wings and tail as it was, planning to do poly-paint on it for details. I started working on the head and the beak.
When I found it satisfactory enough, I moved on to detailing the legs. After this I decimated the mesh, that is, reduced its polycount. I knew triangles would be a headache compared to poly models, but I decided to go with it. The final turntable sculpt came out as this. (Glossy material, because life :) )
Then I moved into poly painting. I used Cinema4D in conjunction with my favourite Adobe Photoshop for the poly-painting. As I mentioned before my mesh was full of triangles and poly painting was a headache. I wanted a very simple painting and it took me hours to come up with this:
I used Photoshop to add more details, which came down to this:
And this as the final design:
Looking good, not exactly what I had in mind, but works well.
Here again I faced the wrath of animating triangles, meshes bursting out and everything, but I went vertex by vertex and smoothened it out. The rigging was fun, I played with different skeleton models, but what worked for me based on videos of eagle flying and my sketches was this:
Looking good again! I then started animating the model. This was fun. Although I could’nt learn how to create multiple motion clips, I realised the best workflow from C4D to Unity3D was to use all animations in one single timeline. I made a quick export as an FBX file and tried it out in Unity3D:
Looked good. Now to add details to the animations. I made an idle, fly-up, hover, fly-down and eating animations. Trying this out on Unity3D gave me a good idea on how to map animations in the Animator and some scripts to go with it:
The animations were a bit stiff for my taste, so I tried refining it again. The final refined idle state animation was this:
All set and done! Now to bring in the new dimension…Augmented Reality. I got my hands on a Google Pixel and an iPhone 7. So I used the ARKit and ARCore examples of Unity and put the eagle in AR and tested it out:
Worked well! And looking good. Except for the yellow spots on the eagle’s back which I let remain so that it gives character and identity to Aquila. Yes, the eagle’s name is Aquila as well.
Next up was integrating voice into the mix. I tried a couple of different options available in the Asset Store but I found one speech recognizer particularly best suited because of its simplicity and ability to recognise phrases. However, continuous recognition was not available yet. So I decided to test out how good the available toolkit is:
Pretty good, although ambient noise was a problem. I started digging deep into the code and re-factored it for continuous and better recognition. I included the cycles of animation and attached it to a variety of voice commands. Also it was fun creating the motion path of the eagle through code. Although not smooth, it gave the effect which I wanted. I decided to leave the fly-down animation and use hover while flying down also (it looks like its gonna crash, but guess again!)
Took it outside for tests and this is what I came up with:
All well and good. Now to get myself a drone. Drones are hard to come by, especially in India with all the laws governing flying of drones. So I gave a shout out to my friends to help me with one. More on it later. In the mean time I started refining the interface and making some graphics:
That’s the logo I came up with. I had many words in mind, AR, sharp, glide, chute, future and I hope I could capture all that in the logo.
Simple graphics to go with it as UI elements:
I finally got a drone, thanks to a very special friend! Integrating the drone was a headache though. Auto-start when I say “Be my eyes” and auto-land when I say “that’s it”. And of-course I had a low quality hackable drone, with bad video feed, but hey, lets say the eagle sees it like that :P
Well it took a while to smooth things out, but finally I made it. Here is the final video: