What Is GPU Edge Computing?

GPU edge computing puts a graphics processor in a computer beside the cameras, sensors and machines that produce data, and uses it for the parallel work a processor alone handles slowly. Almost all of that work is AI inference: running a trained model on live video, lidar or sensor data to detect, classify, measure or decide.

The alternative is to send the data to a cloud or data centre GPU. That suits occasional analysis and model training, but live video at many sites quickly becomes expensive to send, and the round trip adds latency that a robot, vehicle or safety system cannot accept. Our comparison of edge AI vs cloud AI sets out the trade-offs. Processing at the edge also keeps video and process data on site, which helps with data protection.

The workloads are the same as in a data centre, scaled to the site:

Edge GPUs come in five broad forms, from a few watts to several kilowatts. Integrated graphics and NPUs inside the processor handle light AI work in a compact system without a discrete GPU. NVIDIA Jetson modules combine an Arm processor, a GPU and video engines on one board. MXM and embedded GPU modules bring discrete GPUs to compact and fanless computers, as our MXM GPU explainer describes. Standard PCIe cards, from the 72 W L4 to large RTX cards, go into industrial GPU computers. At the top, edge AI servers carry several GPUs for a whole site.

The GPU itself is only part of the choice. An edge GPU has to run on the power available, survive the temperature, shock and dust of its location and keep running for years without a technician. That is why the same GPU ends up in very different computers on a factory line, in a vehicle and in a substation.

A data centre gives a GPU stable mains power, chilled air and a technician down the corridor. An edge site usually offers none of these, so an edge GPU computer is designed around four constraints.

Power. The system may run from a 12 or 24 V vehicle supply, a cabinet’s 24 V rail or a solar battery, so the GPU’s power draw has to fit the budget, including peaks. Heat. Cabinets, vehicles and outdoor enclosures can be hot and dusty, so the cooling is sealed, filtered or fanless, and the GPU must hold its clocks at the top of the rated temperature range. Shock and vibration. Cards are braced or replaced by modules that are screwed down, and storage is solid state. Remote operation. Watchdogs, out-of-band management and long-lifecycle parts let a system run for years and be updated without a site visit.

For many projects the real decision is between a rugged Jetson computer and a rugged computer with a discrete GPU. Both run CUDA and NVIDIA’s AI software, so a model trained in the cloud can usually be deployed to either. The differences are in power, throughput and the software platform.

Choose a rugged Jetson computer when power, space or weight are tight, as on a drone, robot or small vehicle, and when the model fits the module’s memory. Choose a rugged GPU computer when the model is large, the camera count is high, the software needs Windows or x86, or the GPU must be upgradable later. If you are unsure, benchmark the model on both: the frames per second you measure decides it far more reliably than peak TOPS figures.

We fit and configure the GPU, storage and operating system as standard, so the system arrives ready for your models. Our range of edge AI computers covers each of these platforms.