Click “Watch Now” to login or join the NVIDIA Developer Program.

WATCH NOW

Optuna: An Eager Hyperparameter Optimization Library

Shotaro Sano, Preferred Networks

GTC 2020

We'll present Optuna, an open-source, next-generation hyperparameter optimization framework with three novel design criteria: (1) an eager API that allows users to concisely construct dynamic, nested, or conditional search spaces; (2) efficient implementation of both sampling and early-stopping strategies; and (3) an easy-to-set-up, versatile architecture that can be deployed for various purposes, ranging from simple scaling on distributed GPUs to lightweight experimentation, conducted via interactive interface. Optuna is the first optimization software designed to run in Eager mode. We'll present the basic usage of Optuna, describe the design techniques that we needed to meet the above criteria during development, and share our experiences leveraging Optuna to tune hyperparameters on single and multiple GPUs.