Secrets of Event-Based Optical Flow (ECCV 2022, Oral)

PDF: https://arxiv.org/pdf/2207.10022
Code: https://github.com/tub-rip/event_base...
Poster: https://drive.google.com/file/d/1mF-m...

Event cameras respond to scene dynamics and offer advantages to estimate motion. Following recent image-based deep-learning achievements, optical flow estimation methods for event cameras have rushed to combine those image-based methods with event data. However, it requires several adaptations (data conversion, loss function, etc.) as they have very different properties. We develop a principled method to extend the Contrast Maximization framework to estimate optical flow from events alone. We investigate key elements: how to design the objective function to prevent overfitting, how to warp events to deal better with occlusions, and how to improve convergence with multi-scale raw events. With these key elements, our method ranks first among unsupervised methods on the MVSEC benchmark, and is competitive on the DSEC benchmark. Moreover, our method allows us to expose the issues of the ground truth flow in those benchmarks, and produces remarkable results when it is transferred to unsupervised learning settings. We release the code open source.

Reference:
Shintaro Shiba, Yoshimitsu Aoki, Guillermo Gallego
Secrets of Event-Based Optical Flow,
European Conference on Computer Vision (ECCV) 2022, Tel Aviv.

Event-based Vision:
Research: https://sites.google.com/view/guiller...
Code: https://github.com/tub-rip/event-visi...
Event Collapse in Contrast Maximization Frameworks: https://arxiv.org/abs/2207.04007
Survey paper: https://arxiv.org/abs/1904.08405
Course at TU Berlin: https://sites.google.com/view/guiller...

Affiliation:
Shintaro Shiba and Yoshimitsu Aoki are with the Faculty of Science and Technology, Keio University, Kanagawa, Japan.
Guillermo Gallego is with TU Berlin, the Einstein Center Digital Future and the SCIoI Excellence Cluster, Berlin, Germany.