What Is cuDNN?

cuDNN, the CUDA Deep Neural Network library, is NVIDIA’s library of GPU-accelerated building blocks for deep learning. First released in 2014, it provides fast implementations of the operations neural networks are built from, tuned for each NVIDIA GPU architecture.

Frameworks such as PyTorch, TensorFlow and JAX call cuDNN underneath, so they do not have to write and tune those operations themselves for every GPU. You rarely call it directly: you benefit from it through the framework you use. It is one layer of the NVIDIA stack, sitting above CUDA and below the frameworks.

Most of the work in a neural network is a handful of operations repeated millions of times. cuDNN supplies optimized versions of those operations.

It helps to see cuDNN as one layer in a stack, with each layer relying on the one below.

cuDNN uses Tensor Cores automatically when the precision and layer shapes allow, which is where much of its speed comes from on recent GPUs. Our guide to common terms in AI GPU computing explains the rest of the vocabulary.

Early versions of cuDNN offered one function per operation. Since cuDNN 8, a graph API lets a framework describe several operations together, such as a convolution followed by a bias add and an activation, so cuDNN can fuse them into one GPU kernel. Fusion avoids writing intermediate results to memory and back, which often matters more for speed than the maths itself. cuDNN 9, released in 2024, built on this with faster attention kernels for transformer models, and NVIDIA publishes an open-source C++ and Python front end that makes the graph API easier to use.

Each cuDNN release is built for particular CUDA major versions, and each framework build expects particular cuDNN and CUDA versions. A mismatch is one of the most common reasons a GPU-accelerated application fails to start or quietly falls back to the CPU. Pip and conda packages of PyTorch now bundle their own cuDNN, which avoids most conflicts on servers.

On a Jetson, cuDNN comes with JetPack, so its version is set by the JetPack release flashed onto the module, and frameworks have to be built against it. We configure these versions together so the deployed software is consistent from the development system to every system in the field. Our Jetson Orin computers and workstation GPUs both run cuDNN-accelerated frameworks.

At the edge, cuDNN matters most during development and for models that run inside a framework. Teams often prototype on an industrial PC with a discrete GPU, running PyTorch with cuDNN, before converting the model to TensorRT for deployment. Some applications keep the framework in production, for example when models change often or use layers TensorRT does not yet support, and then cuDNN does the work.

Our industrial GPU computers suit that development and on-site training role, with RTX cards whose Tensor Cores cuDNN uses for FP16 and BF16 layers. Matching the cuDNN version on the development system to the one on the deployed system avoids surprises in accuracy and speed.

There are three common routes. On a Jetson, it is installed as part of JetPack. On an x86 server or industrial PC, framework packages from pip or conda usually bring their own copy, and NVIDIA also publishes cuDNN as system packages and archives for developers who build their own software. Use the same route on every system in a fleet, so versions stay identical. To check which version a framework is using, PyTorch reports it with torch.backends.cudnn.version(), and on Jetson the installed packages show it.

We supply GPU and Jetson computers with matched CUDA, cuDNN and framework versions installed and validated as standard, so your models run as they did in development. Tell us your framework and versions, and we will build the system to match.