US How edge AI works
Edge AI is artificial intelligence that runs on a computer at the site where the data is produced, rather than on a server in a data centre. A model is trained once, usually on large GPU clusters, and then the trained model is deployed to a smaller computer beside the cameras, sensors or machines it serves. That computer runs inference: it feeds live data through the model and acts on the answer.
A typical edge AI system has three parts. Sensors produce the data, such as camera frames, vibration readings or LiDAR points. An edge computer with an AI processor runs the model on each frame or sample as it arrives. The results, an alert, a count, a reject signal or a steering command, go to the local equipment straight away, and only summaries or events travel onward to a control room or the cloud.
Because the decision is made where the data lives, a round trip to the cloud is no longer in the control loop. That is what makes edge AI fast enough to stop a conveyor, trigger a barrier or steer a robot.
Why run AI at the edge
Cloud AI suits training and large batch jobs. Edge AI suits decisions that have to be made on site, often continuously, from data that is too large or too sensitive to send away.
-
//
Latency. Local inference typically answers in milliseconds, without the delay or jitter of a network round trip.
-
//
Bandwidth. A single 1080p camera stream can use several megabits per second. Sending only events and metadata cuts the uplink to a fraction of that.
-
//
Privacy and data sovereignty. Images of people, vehicles or production lines can stay on site, with only anonymised results leaving the building.
-
//
Resilience. The system keeps working if the internet connection drops, which matters on vehicles, ships and remote sites.
-
//
Running cost. Inference on owned hardware avoids per-request cloud charges for workloads that run around the clock.
Edge AI hardware compared
Edge AI performance is usually quoted in TOPS, trillions of operations per second, at INT8 precision. The figure is a useful guide to scale, but the right processor also depends on the model, the number of streams, the power budget and the software stack.
| Processor type | Typical AI performance | Typical power | Best suited to |
|---|---|---|---|
| CPU with integrated NPU | Around 10 to 50 TOPS (NPU) | Around 15 to 45 W (whole processor) | Light vision, sensor analytics, x86 software |
| M.2 AI accelerator | Around 13 to 40 TOPS | Typically 2.5 to 8 W | Adding AI to an existing x86 or Arm computer |
| NVIDIA Jetson Orin module | Up to 67 (Orin Nano Super) to 275 TOPS (AGX Orin) | Around 7 to 60 W | Multi-camera vision, robotics, CUDA software |
| NVIDIA Jetson Thor module | Up to 2,070 FP4 TFLOPS | Around 40 to 130 W | Generative AI and humanoid robotics at the edge |
| Discrete GPU | Hundreds of TOPS and up | Around 70 to 300 W per card | Many streams, large models, edge servers |
Figures are typical vendor peak values and are not directly comparable across precisions. Our guide to NPUs for edge AI explains where a neural processing unit is enough, and our edge AI computers span NPU, GPU, Hailo and NVIDIA Jetson systems.
Edge AI examples
Edge AI is most useful where cameras or sensors produce a constant stream of data and the response has to be quick and local.
The table gives common edge AI examples by sector, with the data each one uses and the reason the model runs on site.
| Sector | Edge AI examples | Data used | Why it runs at the edge |
|---|---|---|---|
| Manufacturing | Defect inspection, assembly checks, code reading | Area-scan and line-scan cameras | The reject decision must land within the line’s cycle time |
| Logistics and warehousing | Parcel dimensioning, label reading, AMR navigation | Cameras, 3D sensors and LiDAR | Sorters and robots cannot wait for a cloud round trip |
| Retail | Shelf monitoring, footfall counting, self-checkout loss prevention | In-store cameras | Video stays in the store; only counts and alerts leave |
| Transport and rail | Track and overhead line inspection, passenger counting, ALPR | Vehicle and trackside cameras | Vehicles move in and out of network coverage |
| Security and smart cities | Intrusion detection, traffic monitoring, PPE checks | CCTV streams | Dozens of video streams would swamp the uplink |
| Energy and utilities | Predictive maintenance, substation and pipeline monitoring | Vibration, current and thermal camera data | Remote sites often have limited connectivity |
| Agriculture | Crop and weed detection, livestock monitoring | Cameras on machines and drones | Fields rarely have a reliable network |
| Healthcare | Imaging assistance, patient fall detection | Medical images and ward cameras | Patient data stays on site |
The pattern is the same in each case: a continuous stream of camera or sensor data, a decision needed within milliseconds or seconds, and a reason the raw data should stay on site. Our explainers on video analytics and predictive maintenance cover two of the most common applications in more depth.
What it takes to deploy edge AI
A working edge AI system is more than a fast processor. The model has to be optimized for the target hardware, often quantised to INT8 and compiled with a runtime such as TensorRT, OpenVINO or the Hailo toolchain. The computer needs the right inputs, such as PoE, GMSL or USB cameras and serial or CAN for the machine, and enough storage for recording.
Edge sites are also harder on hardware than an office. Industrial edge AI computers are usually fanless or sealed, run from wide-range DC power, cope with wide operating temperatures and vibration, and support remote management so a fleet can be updated without site visits. Our white paper on deploying rugged edge computers for AIoT covers those choices, and our rugged Jetson computers are built for vehicles, outdoor cabinets and factory floors.
Quick answers on edge AI
Edge AI: common questions
What is edge AI in simple terms?
AI that thinks on site. Instead of sending camera or sensor data to the cloud for analysis, a computer beside the equipment runs the AI model itself and acts on the result straight away.
Is edge AI the same as edge computing?
No, it is a subset. Edge computing means processing data near its source. Edge AI is edge computing where the processing is an AI model, which usually needs an NPU, GPU or dedicated accelerator.
What hardware is used for edge AI systems?
A computer with an AI processor. That can be an x86 processor with a built-in NPU, an M.2 accelerator such as Hailo-8, an NVIDIA Jetson module or a discrete GPU, chosen by model size, stream count and power budget.
Are NPUs better than GPUs for edge AI?
For efficiency, often; for flexibility, no. An NPU runs supported models at low power. A GPU runs a wider range of models and frameworks and scales to more streams, at higher power.
Does edge AI need an internet connection?
Not to make decisions. Inference runs locally. A connection is useful for sending results, updating models and remote management, but the system keeps working without one.
What are some examples of edge AI?
Inspection, video analytics, robots and predictive maintenance. Edge AI checks products on production lines, counts people and vehicles in camera feeds, steers AMRs and drones, and spots machine faults from vibration data, all on computers at the site.
We build edge AI computers with the AI processor, cameras, storage and operating system image fitted as standard, and validate the full system before it ships. Tell us your model, camera count and site conditions, and we will specify the hardware to match.