What Is NVIDIA Holoscan?

NVIDIA Holoscan is a set of software and hardware for processing streaming sensor data with AI in real time, close to where the data is produced. A Holoscan application takes a continuous feed, such as video from an endoscope, frames from an inspection camera or echoes from an ultrasound probe, and runs it through capture, pre-processing, AI inference and display or control with as little delay as possible.

It began as Clara Holoscan, aimed at medical devices, and NVIDIA has since widened it to robotics, industrial inspection, scientific instruments and broadcast. The common thread is latency. In a surgical display or a reject gate on a production line, a result that arrives a few frames late is not useful, so Holoscan is designed to keep data on the GPU and move it between processing stages without unnecessary copies.

Holoscan is not a camera or a single product. It is a development framework plus reference hardware, and an application written with it can run on several classes of NVIDIA system.

NVIDIA uses the Holoscan name for several related pieces. Most projects use the SDK; the Sensor Bridge and IGX hardware are added when the system needs them.

A Holoscan application is built from operators, each doing one job, connected into a graph. A typical vision pipeline has a source operator that receives frames, a format converter that resizes and normalises them, an inference operator that runs a TensorRT or ONNX model, a post-processing step that turns the output into boxes, masks or measurements, and a visualiser or output operator that draws an overlay or sends a result to a controller.

The SDK handles the plumbing between them. Data stays in GPU memory as tensors where possible, operators run in parallel on a multi-threaded or event-based scheduler, and a large application can be split into fragments running on more than one computer. Built-in data-flow tracking reports the latency of every path, which is often the figure a medical or safety review asks for.

Holoscan works with ordinary capture paths as well as its own Sensor Bridge. The right choice depends on how far the sensors are from the computer and how much latency the application can tolerate.

Our Holoscan Sensor Bridges take HDMI 2.0, 12G-SDI or GMSL2 camera sources and send them over SFP+ fiber to the processing system, so the computer can sit in a rack well away from the cameras. For local capture, SDI capture cards remain a common input, and our guide to SDI capture for edge AI covers how they feed an inference pipeline.

Medical imaging is still the largest use. Endoscopy, surgical video, ultrasound and robotic surgery all need AI overlays with very low delay, along with the traceability and long-term support that medical device makers require, which is why IGX was designed with them in mind. Our case study on AI medical carts with endoscopy video capture shows the hardware side of that kind of pipeline.

Outside medicine, Holoscan suits industrial inspection with several high-speed cameras, robotics that fuse cameras with lidar or radar, and scientific instruments such as radio telescopes and particle detectors. On the hardware side, Jetson AGX Orin and Jetson Thor computers suit compact and mobile systems, while industrial GPU computers, including IGX Thor systems with an optional RTX PRO 6000 Blackwell card, handle the heaviest multi-stream pipelines.

We supply our Jetson, IGX and GPU computers with the capture hardware, drivers and NVIDIA software stack fitted and validated as standard, ready for your Holoscan application.