Edge AI

When a program recognizes speech or finds faces in photos, this requires computation. This computational work can happen in two places. Either on large computers in a data center, meaning in remote halls full of servers to which the device connects over the internet. Or directly on the device you hold in your hand. The second case is called Edge AI. “Edge” refers to the edge of the network, the place where the devices and the users are. A phone that converts speech to text without an internet connection is a typical example.

Why this matters

Three reasons favor computing on the device. The first is speed. The trip to the data center and back always costs some time, often tenths of a second up to full seconds. For a car recognizing a pedestrian, that is too long.

The second reason is privacy. What the device itself evaluates never leaves the device. A doorbell video that is analyzed locally does not end up with a provider abroad. This is also legally simpler, because European regulations strongly restrict the transfer of personal data.

The third reason is money. Every request to a data center costs the provider electricity and computing time. If the computation runs on the customer’s device, the customer pays for the electricity. That is why the industry is highly interested in this approach.

How it works

The core problem is space. Large AI models, like those behind well-known chatbots, require memory and power on a scale that no phone has. So smaller models are built, or large ones are shrunk down.

An important technique for this is called quantization. A model consists of billions of numbers. If these numbers are stored more coarsely, meaning with fewer decimal places, the model shrinks significantly. It becomes a little less precise in the process, but often barely noticeably so. You can imagine it like a photo that is converted from high quality to a smaller file size: it is still good enough for display on a phone.

On top of that comes special hardware. Modern phone and laptop chips contain dedicated components just for AI computations, often called NPUs, meaning neural processing units. They compute exactly the same recurring multiplications that an AI model needs, while consuming little battery.

In practice, both locations are often combined. The device handles simple tasks, and forwards difficult ones to the data center. The user usually notices nothing of this division.

Where you encounter the term

In everyday life, Edge AI is found in many devices. The facial recognition used to unlock a phone runs locally. So does the automatic image enhancement of the camera, the sorting of the photo gallery by people, and many translation features that work without a network. Driver assistance systems in cars and voice commands in TVs also belong to this category.

In business news, the term comes up especially in connection with chipmakers. Companies like Qualcomm, Apple, Arm, or Nvidia advertise that their processors run AI directly on the device. Industrial companies also use the term, for instance when cameras in a factory detect defective parts themselves. Anyone reading market reports will often find Edge AI framed as a counterpart to the cloud, meaning computing in remote data centers. Both approaches are growing in parallel; they do not replace each other.