NVIDIA Nsight Deep Learning Designer
Nsight Deep Learning (DL) Designer is an integrated development environment that helps developers efficiently design and optimize deep neural networks for high inference performance. It's built atop the industry standard ONNX model format and popular inference solutions like TensorRT? and ONNX Runtime.
Nsight Deep Learning Designer is still under active development. A Developer Preview build of version 2024.1 is available now by request.
Key Benefits
Nsight DL Designer visualizes a TensorRT model for inspecting and editing.
GUI for ONNX Model Design and Optimization
Nsight DL Designer is a GUI-based tool that makes editing and creating an ONNX model visible and intuitive. Its integration with other tools (including user-defined ones) allows quick and easy whole model transformations.
Built-in Profiler for Performance Evaluations
Nsight DL Designer ships with both a ONNX Runtime profiler and a TensorRT profiler. Developers can quickly evaluate a model’s inference performance using these profilers while they make changes to the model.
Close Integration With TensorRT
Nsight DL Designer can be used as a frontend for NVIDIA’s TensorRT inference engine. Developers can easily perform inference profiling and build a TensorRT engine for a model within the same environment.
Explore Key Features
Efficient Model Design Without Coding
Nsight DL Designer is a full-fledged editor for ONNX models. Its GUI allows developers to open an existing ONNX model, visualize its computation graph, and make changes to the model graph simply by dragging and dropping ONNX operators. DL Designer is currently aligned with ONNX 1.17 and opset 20 and supports the latest features like Local Functions and FP8. Advanced developers can create a model from scratch entirely in DL Designer (no coding in Python needed!).
DL Designer is integrated with popular ONNX tools like GraphSurgeon and Polygraphy to make easy global modifications to a model, like graph sanitization, FP16 conversion, and initializers type conversions. Custom tools are also supported.
DL Designer also supports user-defined tools that allow developers to make quick whole model transformations using tools that they are familiar with.
Inference Performance Profiling With GPU Metrics
For performance-critical applications, frequent and accurate profiling of a model’s inference performance is important for developers to gain insights on where the performance falls short and to guide them to tune the model’s architecture to meet the required performance target.
Nsight DL Designer is tightly coupled with ONNX Runtime and TensorRT so developers can design and profile their models all in a single environment. DL Designer’s ONNX Runtime profiler currently supports both CUDA and DirectML execution providers in addition to the CPU. With DL Designer’s TensorRT profiler, developers can also get detailed profiling data based on GPU metrics like SM utilization, Tensor Core utilization, SM occupancy, and more. This allows developers to further optimize their models for a specific GPU family.
One distinctive feature of DL Designer’s profilers is the ability to visually correlate the profiling data gathered at low-level kernels to the original ONNX operators in a model.
Easy Model Export For Deployment
Nsight DL Designer provides flexible options to support model deployment. A network model edited or created using DL Designer can be easily exported as an ONNX file for platform-independent deployment or as a TensorRT engine for high-performance inferencing with TensorRT.
When exporting a model as a TensorRT engine, the same tactics used for profiling will be used by default. This ensures that developers will see the same inference performance from the model in deployment as they see in profiling.
Tactics and optimization parameters selected for exporting models as TensorRT Engines.
View Other Tools Within the Nsight Suite
Nsight DL Designer is part of the NVIDIA Nsight Developer Tools suite—a collection of powerful tools, libraries, and SDKs that enable developers to build, debug, and profile software utilizing the latest accelerated computing hardware.
Nsight Systems
NVIDIA Nsight Systems is a system-wide performance analysis tool designed to visualize an application’s algorithms, help you identify the largest opportunities to optimize, and tune to scale efficiently across any quantity or size of CPUs and GPUs.
Nsight Compute
Nsight Compute is an interactive kernel profiler for CUDA applications. It provides detailed performance metrics and API debugging via a user interface and command-line tool. It also provides a customizable, data-driven user interface and metric collection that can be extended with analysis scripts for post-processing results.
Nsight Graphics
NVIDIA Nsight Graphics is a standalone developer tool with ray-tracing support that enables you to debug, profile, and export frames built with Direct3D, Vulkan, OpenGL, OpenVR, and the Oculus SDK.
