NVIDIA CUDA Profiling Tools Interface (CUPTI) - CUDA Toolkit

The NVIDIA CUDA Profiling Tools Interface (CUPTI) is a library that enables the creation of profiling and tracing tools that target CUDA applications. CUPTI provides a set of APIs targeted at ISVs creating profilers and other performance optimization tools:

• the Activity API,
• the Callback API,
• the Host Profiling API,
• the Range Profiling API,
• the PC Sampling API,
• the SASS Metric API,
• the PM Sampling API,
• the Checkpoint API,
• the Profiling API,
• the Event API,
• the Metric API, and
• the Python API (available separately)

Using these CUPTI APIs, independent software developers can create profiling tools that provide low and deterministic profiling overhead on the target system, while giving insight into the CPU and GPU behavior of CUDA applications. Normally packaged with the CUDA Toolkit, NVIDIA occasionally uses this page to provide CUPTI improvements and bug fixes between toolkit releases.

Revision History

Key Features

• Trace CUDA API by registering callbacks for API calls of interest
• Full support for entry and exit points in the CUDA C Runtime (CUDART) and CUDA Driver
• GPU workload trace for the activities happening on the GPU, which includes kernel executions, memory operations (e.g., Host-to-Device memory copies) and memset operations.
• CUDA Unified Memory trace for transfers from host to device, device to host, device to device and page faults on CPU and GPU etc.
• Normalized timestamps for CPU and GPU trace
• Profile hardware and software event counters, including:
• Utilization metrics for various hardware units
• Instruction count and throughput
• Memory load/store events and throughput
• Cache hits/misses
• Branches and divergent branches
• Many more
• Enables automated bottleneck identification based on metrics such as instruction throughput, memory throughput, and more
• Range profiling to enable metric collection over concurrent kernel launches within a range
• Metrics attribution at the high-level source code and the executed assembly instructions.
• Device-wide sampling of the program counter (PC). The PC Sampling gives the number of samples for each source and assembly line with various stall reasons.

Updates in CUDA Toolkit 12.9

New Features

• In confidential computing, rotating a key is a high-impact operation that halts all traffic using that key. During the key rotation, channels sharing the key will be drained and will not accept new CUDA work until the rotation completes. However, this process is not directly visible to user applications. CUPTI offers additional feedback to tools, allowing users to account for their impact on application timelines. The struct CUpti_ActivityConfidentialComputeRotation provides timestamps for each stage of the encryption rotation, while the enum CUpti_ConfidentialComputeRotationEventType lists the supported stages.
• For Orin+ mobile chips on the Linux aarch64 platform, added metrics (mcc__*) support for memory controller channel (MC Channel) unit which connects to the DRAM.
• A new attribute CUPTI_ACTIVITY_ATTR_DEVICE_BUFFER_SIZE_DEVICE_GRAPHS is added in the activity attribute enum CUpti_ActivityAttribute to set and get the size for the profiling buffer for device graph operations.

Resolved Issues

• Decoupled activity kinds CUPTI_ACTIVITY_KIND_CUDA_EVENT and CUPTI_ACTIVITY_KIND_SYNCHRONIZATION. Previously, enabling one of these activities would automatically enable the capturing of records for the other.
• Fixed a crash when the client fails to provide a buffer in response to the activity buffer request callback issued by CUPTI. This condition is notified to the client using the CUPTI_CB_DOMAIN_STATE callback.
• The callback CUPTI_CBID_RESOURCE_MODULE_PROFILED is issued regardless of whether kernel tracing is enabled.

Requirements

Supported platforms

• Linux x86_64^[1]
• Windows x86_64^[1]
• Linux aarch64 SBSA^[1]
• DRIVE OS QNX aarch64^[2]
• DRIVE OS Linux aarch64^[2]
[1] available in the CUDA Desktop Toolkit only
[2] available in the Embedded or Drive toolkits only


Supported NVIDIA GPU architectures

• Activity and Callback APIs
• All architectures supported by CUDA Toolkit
• Event and Metric APIs (this API and associated GPUs have been deprecated as of CUDA 12.8, and will be dropped in a future release)
• Volta
• Pascal
• Maxwell
• Profiling and PC Sampling APIs
• Blackwell: B100, GB10x
• Hopper: GH100
• Ada: AD10x
• Ampere: A100 with Multi-Instance GPU, GA10x
• Turing
• Volta: GV100, GV10B (Volta GPUs have been deprecated as of CUDA 12.8, and will be dropped in a future release)

CUDA Toolkit

• CUPTI can be found in the CUDA Toolkit 12.9 production release

Drivers

Please use the following drivers
• 576.02 (Windows)
• 575.51.03 (Linux)
provided with CUDA Toolkit 12.9 production release or a more recent version.

Documentation

• Online Product Documentation

Support

To provide feedback, request additional features, or report issues, please use the Developer Forums.

Installation Overview

When installing CUDA Toolkit 12.9 and specifying options, be sure to select CUDA > Development > Tools > CUPTI.