US Why there is no single NPU TOPS figure
NPU TOPS is the peak number of trillion operations per second a neural processing unit can perform, usually at INT8 precision. How many you need depends on the work: a small model on one camera needs a fraction of what several cameras running a large detector need. The rating is a ceiling, not a promise, and real throughput depends on how well a model maps onto the hardware.
Two numbers are often confused. NPU TOPS covers the NPU alone. Platform TOPS adds the CPU, integrated GPU and NPU together, which produces a much larger figure that no single workload will see. Our product data shows the gap: the AMD Ryzen PRO 8640U is listed with a 16 TOPS NPU and 31 TOPS across the chip.
What an NPU TOPS rating measures
A TOPS figure is calculated, not measured: the number of multiply-accumulate (MAC) circuits, multiplied by two operations per MAC, multiplied by the clock speed. It assumes every circuit is busy on every cycle, which never happens in a real model. Some vendors also quote TOPS with sparsity, which doubles the figure for models pruned to a specific pattern. Our explainer on what TOPS means covers the calculation in more detail.
The precision matters too. A rating at INT4 is roughly double the INT8 figure, and an FP16 rating roughly half, so only compare numbers quoted at the same precision.
NPU and accelerator ratings in our range
The table lists AI ratings for processors and accelerators we supply, so you can see where each sits. Figures are the manufacturers’ peak ratings, at INT8 unless noted.
| Processor or module | Type | AI rating | What the figure covers |
|---|---|---|---|
| Intel Core Ultra 5 135U (Meteor Lake) | Integrated NPU | 11 TOPS | NPU only |
| Intel Core Ultra 5 225H (Arrow Lake) | Integrated NPU | 13 TOPS | NPU only |
| AMD Ryzen PRO 8640U | Integrated NPU | 16 TOPS (31 platform) | NPU, then whole chip |
| Intel Core Ultra X7 368H (Panther Lake) | Integrated NPU | 50 TOPS | NPU only |
| AMD Ryzen AI 300 series | Integrated NPU | 50 TOPS | NPU only |
| Hailo-8L | M.2 accelerator | 13 TOPS | Accelerator only |
| Hailo-8 | M.2 accelerator | 26 TOPS | Accelerator only |
| Hailo-10H | M.2 accelerator | 40 TOPS (INT4) | Accelerator only |
Two systems with the same rating can perform very differently on the same model, so treat the table as a shortlist and confirm with a benchmark.
How many TOPS for common workloads
The ranges below are rough starting points for INT8 models at typical industrial resolutions. They assume a well-optimised model and leave some headroom.
-
//
Audio, keyword spotting and sensor models. A few TOPS or less. Almost any current NPU handles these alongside other work.
-
//
One camera, small detector. Around 5 to 15 TOPS for a compact object detection model at 720p or 1080p and 30 frames per second.
-
//
Four to eight cameras. Typically 20 to 50 TOPS, depending on model size and whether every frame is analyzed.
-
//
Segmentation, pose estimation or large detectors. Often 40 TOPS or more, and memory bandwidth starts to limit performance before TOPS does.
-
//
Small language and vision-language models. 40 TOPS or more, plus enough fast memory to hold the model. Microsoft sets 40 NPU TOPS as the minimum for Copilot+ PC features.
Why TOPS is not the whole story
Three things often matter more than the peak figure. Memory bandwidth decides how quickly weights and images reach the NPU, and many vision models are limited by it. Operator support decides whether the whole model runs on the NPU or whether some layers fall back to the CPU, which can halve throughput. Software maturity decides how close real performance gets to the peak, and it improves with driver and toolkit releases.
Precision is the other lever. INT8 quantisation is usually needed to reach the rated figure at all, and some models lose accuracy when quantised, so check accuracy as well as speed.
How to size an NPU for your project
Start with the model you will deploy, not with the hardware. Export it, quantise it and run it on two or three candidate processors, then compare frames per second per stream, latency and power at the operating temperature. If the integrated NPU falls short, an accelerator adds capacity without changing the computer: our Hailo AI computers pair an x86 or Arm processor with a Hailo module, while Intel Core Ultra computers and AMD Ryzen AI 300 systems keep everything on one chip. Allow 30 to 50 percent headroom for model updates and extra cameras.
Measure at the temperature and power limit the system will run at on site. A fanless computer in a warm cabinet may hold lower clocks than the same board on a bench, and the NPU’s sustained throughput falls with it.
Quick answers on NPU TOPS
NPU TOPS: common questions
Is a 40 or 45 TOPS NPU good?
Yes, for most edge inference. It clears Microsoft’s 40 TOPS bar for Copilot+ features and handles several camera streams with compact models. Large generative models still need more memory than the TOPS figure suggests.
What does an NPU with 13 TOPS mean?
It can perform up to 13 trillion operations a second. That is the peak at INT8. It is enough for one or two cameras with a small detector, or for audio and background effects, but not for heavy multi-stream work.
Which processors have a 50 TOPS NPU?
In our range, Panther Lake and Ryzen AI 300. Intel Core Ultra Series 3 (Panther Lake) and AMD Ryzen AI 300-series processors both carry NPUs rated at around 50 TOPS.
Are NPU TOPS and GPU TOPS comparable?
Only loosely. Both are peak figures, but GPUs and NPUs reach different shares of their peak on the same model. Compare measured frames per second instead.
Can I add TOPS to an existing computer?
Often, yes. A spare M.2 slot can take a Hailo accelerator, and a PCIe slot can take a GPU, provided the power supply, cooling and software support allow it.
We configure NPU and accelerator-equipped computers with the drivers and AI runtime installed and validated as standard. Send us your model, cameras and frame rate, and we will recommend the processor or accelerator that meets them.