Click “Watch Now” to login or join the NVIDIA Developer Program.

WATCH NOW

Panoptic Segmentation DNN for Autonomous Vehicles

Ke Chen , NVIDIA | Ryan Oldja, NVIDIA

GTC 2020

We'll present our NVIDIA DRIVE^? AV's Panoptic Segmentation Deep Neural Network (DNN), which can be used for semantic and instance segmentation of complex scenes for self-driving car scenarios, such as complex urban areas, congested traffic, construction zones with unusual activities, and so on. With Panoptic Segmentation DNN, input images can be accurately parsed for both semantic segmentation (which pixels represent which object class), as well as instance content (which pixels represent which object instance). Planning and control modules can use panoptic segmentation results to better inform autonomous driving decisions. We'll cover our highly accurate GT dataset, DNN architecture, our multi-task training process, and our real-time inference (that includes post-processing steps) on vehicles' AGX compute. Our network achieves state-of-the-art accuracy and runs at 7ms end-to-end on NVIDIA AGX GPUs. We'll show videos of our experiments on a real vehicle in various challenging conditions.