Human-Robot Interface: Efficient Control and Flight Data Display of a Quadcopter through Google Glass

Download 127.73 Kb.
Size127.73 Kb.
1   2


There are several types of data involved in this project: flight data, video data, and image data. Flight data consist of GPS coordinates, drone speed, altitude, and error codes. Video data is 1080p/30fps HD video in the “.MP4” file format. Image data is RAW or JPEG file formats. The transferring and storing of this data is a project task. In addition, a database must be set up for long-term storage.

Data Transfer

The program transfers data from the quadcopter to Glass through API calls. Once transferred to Glass, the data will remain locally on Glass’s memory until Glass connects to the internet. Then upon connecting to the internet, Glass will automatically upload the data to the database. Figure 5 illustrates this design.

We designed data transfer in two stages because internet connectivity may not be available during drone use. Glass has about 10GB of storage, so a single flight should be completely storable until Glass is able to connect to the internet. However, a user must consider memory limitations when using the drone for multiple flights before transferring data to the database.
Note: The following design information on data preparation is tentative.
To transfer the data to the database a program on Glass automatically prepares the data for database storage. This preparation is storing the data in a database object that the database will access later. Once the database object is ready, Glass sends the data object to a web page interface that will carry out the storage process for the object. This means the web page will check permissions, forward the data object, and issue the appropriate commands for the database to carry out.
Data Storage
Note: The following design information on data preparation is tentative.
When the data is stored on Glass the data is in appropriate files. For example, the video data is a “.MP4” file. Flight data such as speed and altitude will be stored in a text file with key information such as time and an identifier for the Phantom drone.
The database is a relational database with the following entry formats.
Meta data format: (ID, Time, Latitude, Longitude, Altitude, Speed)

Picture data format: (ID, Time, picture)

Video data format: (ID, Time, video)
The fields ID and Time make up the key relating all the data. The setup for the database includes the creation of the tables for each data format.


There are two types of controls that this project implements. The gimbal controls for the camera and the camera controls. The controller that comes with the drone handles the controls for the drone.

Gimbal Controls

Google Glass offers head tracking, which provides values based on the tilt and positioning of a person’s head. Glass’s sensors determine the values and have little to do with a person’s actual head. The program translates the values obtained from Glass to values for the API calls for the gimbal. For example, if a person tilts there head down 5 degrees then the corresponding gimbal API call may be for a tilt of 2 degrees. Controlling the gimbal of Glass is just a matter of collecting the sensor data and translating it into gimbal values for the API call.

Camera Controls

To create a more hands free experience voice commands can make taking pictures easy. This requires setting up a phrase command, which recognizes spoken words and executes its corresponding code. Figure 6 illustrates how camera controls work.


There are three main interaction points for the user: Google Glass, the database, and the drone controller. Each has a distinctive interface preference. This project requires the creation of the database and Glass user interfaces.

Glass UI

Google suggest a simplistic user interface for Glass programs. The following images illustrate the current interface design.

Image 1: Before Photo Image 2: After Photo

Image 3: Video Stream

Database UI

The database information will be accessible through a web page. The current design decision is to make an easy to navigate file system. This system will be similar to a file directory on most operating systems. When a user selects the desired file, the web page will then query the database for the actual file.

Additional Notes

  • The Phantom has flight zone restrictions such as major airports. Entering these zones is not possible. You can find more information at the DJI website.

  • Sideloaded android applications on Glass do not run as expected. Returning to the timeline does not put the application in the background rather it exits the program. In addition, launching a program does not always bring the program into view. We recommend a special android application launcher to resolve these issues.

  • DJI might update both its SDK and drone firmware. This may introduce bugs into the application.

  • DJI issues an application key for developers to associate with their program. If DJI invalidates this key then developing this program further may not be possible without obtaining a new key.

Share with your friends:
1   2

The database is protected by copyright © 2019
send message

    Main page