Real-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. We propose a novel, accurate tightly-coupled visual-inertial odometry pipeline for such cameras that leverages their outstanding properties to estimate the camera ego-motion in challenging conditions, such as high-speed motion or high dynamic range scenes. The method tracks a set of features (extracted on the image plane) through time. To achieve that, we consider events in overlapping spatio-temporal windows and align them using the current camera motion and scene structure, yielding motion-compensated event frames. We then combine these feature tracks in a keyframe-based, visual-inertial odometry algorithm based on nonlinear optimization to estimate the camera’s 6-DOF pose, velocity, and IMU biases. The proposed method is evaluated quantitatively on the public Event Camera Dataset and significantly outperforms the state-of-the-art, while being computationally much more efficient: our pipeline can run much faster than real-time on a laptop and even on a smartphone processor. Furthermore, we demonstrate qualitatively the accuracy and robustness of our pipeline on a large-scale dataset, and an extremely high-speed dataset recorded by spinning an event camera on a leash at 850 deg/s.

Reference:
Rebecq, Horstschaefer, Scaramuzza
Real-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization
PDF: http://rpg.ifi.uzh.ch/docs/BMVC17_Reb...

Our research page on event based vision:
http://rpg.ifi.uzh.ch/research_dvs.html

For event-camera datasets and event camera simulator, see here: http://rpg.ifi.uzh.ch/davis_data.html

Other resources on event cameras (publications, software, drivers, where to buy, etc.):
https://github.com/uzh-rpg/event-base...

Robotics and Perception Group, University of Zurich, 2017
http://rpg.ifi.uzh.ch/