Unsupervised Joint Learning of Optical Flow and Intensity with Event Cameras (ICCV 2025 Highlight)

E2FAI: Events to Flow And Intensity
Project page: https://github.com/tub-rip/E2FAI
PDF: https://arxiv.org/pdf/2503.17262

Event cameras rely on motion to obtain information about scene appearance. This means that appearance and motion are inherently linked: either both are present and recorded in the event data, or neither is captured. Previous works treat the recovery of these two visual quantities as separate tasks, which does not fit with the above-mentioned nature of event cameras and overlooks the inherent relations between them. We propose an unsupervised learning framework that jointly estimates optical flow (motion) and image intensity (appearance) using a single network. From the data generation model, we newly derive the event-based photometric error as a function of optical flow and image intensity. This error is further combined with the contrast maximization framework to form a comprehensive loss function that provides proper constraints for both flow and intensity estimation. Exhaustive experiments show our method's state-of- the-art performance: in optical flow estimation, it reduces EPE by 20% and AE by 25% compared to unsupervised approaches, while delivering competitive intensity estimation results, particularly in high dynamic range scenarios. Our method also achieves shorter inference time than all other optical flow methods and many of the image reconstruction methods, while they output only one quantity.

Reference:
Shuang Guo, Friedhelm Hamann, Guillermo Gallego,
Unsupervised Joint Learning of Optical Flow and Intensity with Event Cameras,
IEEE/CVF International Conference on Computer Vision (ICCV), 2025.

Affiliations:
Technical University of Berlin (Berlin, Germany),
Robotics Institute Germany (RIG), https://www.robotics-institute-german...
Science of Intelligence Excellence Cluster (Berlin, Germany), https://www.scienceofintelligence.de/
Einstein Center Digital Future (Berlin, Germany), https://www.digital-future.berlin/en/

Event-based Vision:
Research: https://sites.google.com/view/guiller...
Code: https://github.com/tub-rip/event-visi...
Survey paper: https://arxiv.org/abs/1904.08405
Course at TU Berlin: https://sites.google.com/view/guiller...