What Is INT8 Quantization?

INT8 quantization is the process of storing and computing a neural network with 8-bit integers instead of 32-bit floating-point numbers. Models are usually trained in FP32, but inference rarely needs that much precision. Quantization maps each value onto 256 integer levels, using a small calibration set of real data to choose the ranges. Where accuracy drops too far, quantization-aware training builds the reduced precision into training so the model learns to cope with it.

The gains come from three places. An 8-bit value takes a quarter of the memory of a 32-bit one, so the model is smaller and moves through memory four times faster. Integer arithmetic needs far less silicon than floating point, so accelerators fit many more multipliers in the same space. And less data movement means less power, which matters most in fanless and battery-powered systems.

Each tensor of weights or activations gets a scale factor, and sometimes a zero point, that maps its real range onto the integer range of −128 to 127. A value is divided by the scale, rounded and stored as an integer; the hardware multiplies the integers and the result is rescaled at the end. Symmetric quantization centres the range on zero, which is simpler and usual for weights, while asymmetric quantization adds a zero point to fit ranges that are not centred, as activations often are.

The scale can be set for a whole tensor or per channel. Per-channel scales follow the different ranges of each filter in a layer and usually keep more accuracy, so most current toolkits use them for weights.

INT8 is one point on a range of number formats. Hardware support decides which ones a given chip can run quickly.

The precision also sets the performance figure. Most TOPS ratings are quoted at INT8, so a model only reaches the hardware’s headline speed once it is quantized.

Most edge AI hardware is built to run INT8 models efficiently, and each vendor supplies the tools to get a model there. A model is usually exported from PyTorch or TensorFlow to ONNX first, then quantized by the toolkit for the target hardware.

For common vision models, a well-calibrated INT8 version typically loses around a percentage point of accuracy or less, but some models are more sensitive. The usual fixes are a better calibration set that reflects real site conditions, per-channel scales, keeping one or two sensitive layers at FP16, and, if needed, quantization-aware training. Our OpenVINO FAQ and TensorRT FAQ cover the Intel and NVIDIA toolkits that quantize models, and our Hailo range uses accelerators that run models prepared by the Hailo Dataflow Compiler.

INT8 is the default for edge vision, but not for every model. Regression models that output precise measurements, such as depth, pose or dimensions from a camera, can lose more accuracy than a classifier, because small rounding errors show directly in the result. Some transformer layers have outlier values that a single integer scale handles badly, which is one reason FP8 and mixed precision have grown for language models.

In those cases teams often run the model at FP16 on a GPU or Jetson, or quantize most layers to INT8 and keep the sensitive ones at higher precision. The right answer comes from measuring accuracy and speed on the target hardware, not from the format alone.

We supply edge AI computers with the runtime for your target hardware installed and validated as standard. Send us your model and accuracy target, and we will recommend the accelerator and precision to match.