3D Scene Reconstruction from a Single Viewport - Maximilian Denninger (ECCV 2020)

This is a two parts talk. It is based on the papers "3D Scene Reconstruction from a Single Viewport" presented at ECCV 2020 and the "BlenderProc" paper. The speaker is the main author of both papers. This is the recording of part 1.
Recording of part 2:    • Blender pipeline to generate images for de...  

References to everything covered in the talk:   / references_from_double_lecture_photorealistic  

00:00 Intro
02:57 Motivation
05:39 3D Representations
13:21 TSDF Compression
20:42 Tree Architecture
32:24 Loss Shaping
43:43 Data Generation
48:41 Qualitative Results
01:05:17 Summary
01:09:56 Discussion

[Chapters were auto-generated using our proprietary software - contact us if you are interested in access to the software]

Lecture abstract:

We present a novel approach to infer volumetric reconstructions from a single viewport, based only on a RGB image and a reconstructed normal image. The main contributions of reconstructing full scenes including the hidden and occluded areas will be discussed and their advantages in contrast to prior works which focused either on shape reconstruction of single objects floating in space or on complete scenes where either a point cloud or at least a depth image were provided. We propose to learn this information from synthetically generated high-resolution data. To do this, we introduce a deep network architecture that is specifically designed for volumetric TSDF data by featuring a specific tree net architecture. Our framework can handle a 3D resolution of 512³ by introducing a dedicated compression technique based on a modified autoencoder. Furthermore, we introduce a novel loss shaping technique for 3D data that guides the learning process towards regions where free and occupied space are close to each other.

git : https://github.com/DLR-RM/SingleViewR...
paper: https://tinyurl.com/singleviewreconst...

Presenter BIO:

Maximilian Denninger is currently pursuing his PhD at the German Aerospace Center (DLR), where he is a full-time researcher. His research goal is to improve the computer vision on mobile robots, where the training data is always scarce. At the DLR he heads the vision part of an exciting project called SMiLE, where the goal is to design and implement robots, which are able to assist people working in elderly homes. This includes a variety of tasks from semantic segmentation to scene reconstruction. As robots need a natural understanding of their environment to fulfill any kind of task. For that he and his colleagues created BlenderProc, which helps in the generation of data for the training of neural networks. He is advised for his PhD by his department head Dr. Rudolph Triebel, which also works for the Technical University of Munich (TUM), where Max also works as a teaching assistant to help teach the course "Maching Learning for Computer Vision".

Linkedin:   / maximilian-denninger  
Twitter:   / denningermax  

-------------------------
Find us at:

Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D
Sub-reddit for discussions ➜   / 2d3dai  
Discord server for, well, discord ➜   / discord  
Blog ➜ https://2d3d.ai

We are the people behind the AI consultancy Abelians ➜ https://abelians.com/