Autonomous Drone Navigation with Deep Learning. Flight over 250 meter Forest Trail

This video demonstrates our autonomous visual navigation system for drones and mobile robotics. The system uses Deep Neural Network for visual perception and path finding. This video features 250 meter autonomous flight over a forest trail (visual navigation only, no GPS, no maps) and shows drone's first person "vision".

This video accompanies our IROS 2017 paper:
"Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness".
Paper: arXiv:1705.02550 [cs.RO] : https://arxiv.org/abs/1705.02550
Our code is open sourced here: https://github.com/NVIDIA-AI-IOT/redtail

Paper Abstract - We present a micro aerial vehicle (MAV) system, built with inexpensive off-the-shelf hardware, for autonomously following trails in unstructured, outdoor environments such as forests. The system introduces a deep neural network (DNN) called TrailNet for estimating the view orientation and lateral offset of the MAV with respect to the trail center. The DNN utilizes a novel application of transfer learning, and it achieves stable flight without oscillations by avoiding overconfident behavior through a loss function that includes both label smoothing and entropy reward. In addition to the TrailNet DNN, the system also utilizes vision modules for environmental awareness, including another DNN for object detection and a visual odometry component for estimating depth for the purpose of low-level obstacle detection. All vision systems run in real time on board the MAV via a Jetson TX1. We provide details on the hardware and software used, as well as implementation details. We present experiments showing the ability of our system to navigate forest trails more robustly than previous techniques, including autonomous flights of 1 km.