TUM AI Lecture Series - Reflections on Image-Based Rendering (Richard Szeliski)

Abstract: Image-based rendering, which combines computer vision and computer graphics techniques to create photorealistic novel views of complex real-world scenes, has been an active area of research for over twenty five years. In this talk, I give a high-level retrospective of this field and make connections to the nascent field of neural rendering, which has the same goals but uses deep networks as its primary tool. Starting with the basics of geometry-driven view interpolation and the theory of light fields and Lumigraphs, I review representational issues such as layers and opacity, view-dependent appearance, and modeling reflections and transparency. I also discuss some applications such as 360 and 3D photos and videos. I close by connecting classic techniques to their modern neural rendering counterparts and discuss two of our recent papers on 3D scene rendering and stochastic animation from still images.


Bio: Richard Szeliski is an Affiliate Professor at the University of Washington and is a Member of the National Academy of Engineering and a Fellow of the ACM and IEEE. Dr. Szeliski has done pioneering research in the fields of Bayesian methods for computer vision, image-based modeling, image-based rendering, and computational photography, which lie at the intersection of computer vision and computer graphics. His research on Photo Tourism, Photosynth, and Hyperlapse are exciting examples of the promise of large-scale image and video-based rendering.

Dr. Szeliski received his Ph.D. degree in Computer Science from Carnegie Mellon University in 1988. He joined Facebook as the founding Director of the Computational Photography group in 2015 and retired in 2020. Prior to Facebook, he worked at Microsoft Research for twenty years as well as several other industrial research labs. He has published over 180 research papers in computer vision, computer graphics, neural networks, and numerical analysis, as well as the books Computer Vision: Algorithms and Applications and Bayesian Modeling of Uncertainty in Low-Level Vision. He was a Program Chair for CVPR'2013 and ICCV'2003, served as an Associate Editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence and on the Editorial Board of the International Journal of Computer Vision, and was a Founding Editor of Foundations and Trends in Computer Graphics and Vision.


TUM AI Lecture Series previous lectures:
Pushing Factor Graphs beyond SLAM (Frank Dellaert):    • TUM AI Lecture Series - Pushing Factor Gra...  
Explainability and Compositionality for Visual Recognition (Zeynep Akata):    • TUM AI Lecture Series - Explainability and...