Development of a Virtual Reality Environment (VRE) for Intuitive Drone Operations 2017-01-2070

Recent advances in Small Unmanned Aerial Systems (SUAS) or drone technologies has resulted in their widespread use in a number of civilian applications, such as aerial imaging, infrastructure inspection, precision agriculture, among others. While this technology is accessible for everyone, it still requires a highly skilled operator to be able to successfully operate these drones in a safe and efficient manner. At the same time, developments in Virtual/Augmented Reality (V/AR) technologies present opportunities for combining the two into novel applications and use cases by providing an intuitive interface for interacting with the drones - opening up possibilities for safe and effective use of drones by relatively untrained operators. This effort addresses the development and implementation of an interface that provides an operator wearing an Oculus Rift virtual reality headset interfaced with a Leap Motion controller the ability to control drones in a virtual reality environment and translate the commands to a physical implementation, in a motion capture volume. This includes actions such as selecting drones, take-off and landing, and commanding the drones to fly a pre-defined flight pattern. DroneKit-Python was used to communicate commands to drones while OptiTrack motion capture cameras and the NatNet SDK (both provided by Naturalpoint Inc.) combine to provide the precise physical location of each drone in an indoor laboratory setting. Unreal Engine 4 was used as the development platform to create the virtual scene the operator resides in. A QAV250 quadcopter from Lumenier Labs was used as the UAS platform, with a Pixhawk flight controller, interfaced with a Raspberry Pi 3 Single Board Computer (SBC) as the companion computer.

In this effort, the virtual environment was created and successfully integrated with the motion capture system. In addition, the QAV 250 quadcopter was successfully controlled through the operator interface in the VR environment and take-offs and flights along pre-defined flight paths (triangles) were successfully achieved. Further tests are planned to increase user interaction and achieve more complex flight paths.