Click “Watch Now” to login or join the NVIDIA Developer Program.

WATCH NOW

3D Deep Learning in Function Space

Michael Niemeyer, MPI-IS and University of Tübingen

GTC 2020

Recent advances in GPU technology and scalable algorithms have led to breakthroughs in deep learning. In particular, convolutional neural networks (CNNs) achieve state-of-the-art results in longstanding vision problems, such as image classification or object detection. However, autonomous agents that navigate and interact in our world need to reason in 3D. Unlike images in the 2D case, it is not clear how to represent 3D geometry and how to make it amenable for deep-learning techniques. We'll introduce our approach to learning 3D representations in function space. First, we'll show how this approach can represent arbitrary topologies without discretization at fixed memory cost. Then we'll extend this framework to learning to predict not only the shape of an object, but also its texture and motion. Finally, we'll show how we can scale our method to real-world scenarios using state-of-the art NVIDIA GPU technology.