Matthew Tancik: Neural Radiance Fields for View Synthesis

Talk @ Tübingen seminar series of the Autonomous Vision Group
https://uni-tuebingen.de/en/faculties...

Neural Radiance Fields for View Synthesis
Matthew Tancik (UC Berkeley)

Abstract: In this talk I will present our recent work on Neural Radiance Fields (NeRFs) for view synthesis. We are able to achieve state-of-the-art results for synthesizing novel views of scenes with complex geometry and view dependent effects from a sparse set of input views by optimizing an underlying continuous volumetric scene function parameterized as a fully-connected deep network. In this work we combine the recent advances in coordinate based neural representations with classic methods for volumetric rendering. In order to recover high frequency content in the scene, we find that it is necessary to map the input coordinates to a higher dimensional space using Fourier features before feeding them through the network. In our followup work we use Neural Tangent Kernel analysis to show that this is equivalent to transforming our network into a stationary kernel with tunable bandwidth.

Bio: Matthew studied Computer Science and Physics at Massachusetts Institute of Technology where he received his bachelor degree in 2016 and master degree in 2017. During his masters degree he worked under the supervision of Ramesh Raskar and Fredo Durand and published his thesis on non-line-of-sight imaging using data driven approaches. He began his PhD in 2018 at UC Berkeley under the supervision of Ren Ng. He is currently interested in exploring the intersection of vision and graphics for robotic perception and view synthesis applications. https://www.matthewtancik.com/