What Is the ZED SDK?

The ZED SDK is the software Stereolabs provides to run its ZED cameras. A stereo camera captures two images from slightly different positions, and the SDK compares them to work out how far away each pixel is. On top of that depth data it adds motion tracking, 3D mapping and AI detection, so an application receives finished spatial information rather than raw pictures. Our explainer on how a stereo camera measures depth covers the optics behind it.

The SDK is a library plus a set of tools. Developers call it from C++, Python, C# or C. The Python package is called pyzed and is installed by a script shipped with the SDK. The tools include ZED Explorer for checking that a camera streams, ZED Depth Viewer for inspecting depth and point clouds, ZED Sensor Viewer for the IMU and other sensors, and calibration and diagnostic utilities. Its tutorials and code samples are on GitHub under the MIT licence.

Every ZED camera needs it. That includes the USB models such as the ZED 2i and ZED Mini, the GMSL2 ZED X family and the ZED X One cameras, which can be used singly or paired as a custom stereo rig.

The SDK is organised into modules, and an application turns on only the ones it needs, because each one adds GPU load.

Depth is the most demanding module, and the depth mode sets the balance between accuracy and frame rate. Since version 5.0 the SDK uses neural network depth by default. The older computer vision modes (PERFORMANCE, QUALITY and ULTRA) are deprecated but still available in the API.

Stereolabs’ figures show a Jetson AGX Orin holding about 30 fps with four cameras in NEURAL, so the module choice sets how many cameras a system can run. An SVO recording stores the raw images, not depth, so footage recorded with depth turned off can be replayed later in any depth mode. Version 5.5 adds an INT8 option for NEURAL depth, which trades a little accuracy for speed.

The SDK runs its depth and AI models on an NVIDIA GPU through CUDA and TensorRT, so it needs one. Stereolabs’ requirements, in summary:

Stereolabs offers a separate installer for each platform: an executable for Windows and a .run file for Ubuntu and for each supported JetPack release. The installer must match the CUDA version on a PC, or the JetPack version on a Jetson. Our guide to JetPack 6 and its software stack explains those versions. After installing, open ZED Explorer and ZED Depth Viewer first: if the camera streams and shows depth there, any later fault lies in your own code, not the hardware.

From there the SDK connects to most robotics and vision tools. Stereolabs maintains integrations for ROS 2, NVIDIA Isaac ROS and Isaac Sim, Docker, GStreamer, OpenCV, PyTorch, YOLO detectors and the Unity and Unreal game engines. For robots, the ROS 2 node is the usual route, and our white paper on the ZED ROS2 wrapper on NVIDIA Jetson walks through building that stack in the right order.

Stereo depth with built-in tracking suits machines that move or handle objects: autonomous mobile robots, forklifts, robotic arms, agricultural vehicles and inspection systems. Here the main work is version control and GPU budget.

Fix the JetPack, ZED X driver and ZED SDK versions for each product build, because a mismatch is the most common cause of a camera that will not open. Check the GPU load of the depth mode and modules you need at your real camera count before choosing the Jetson module. And record SVO files during commissioning, so problems can be replayed off the machine.

Our stereo cameras include the USB ZED 2i and ZED Mini and the GMSL2 ZED X family. We also list Stereolabs GMSL frame grabbers for Jetson Orin, and the ZED Box Mini sits among our rugged Jetson computers.

We supply ZED cameras with the Jetson computer they run on, and as standard we install JetPack, the ZED X driver and a matched ZED SDK version before the system ships.