FP4

A computer stores every number as a sequence of zeros and ones. A single one of these digits is called a bit. The more bits a number gets, the more precisely it can be captured. FP4 is a number format that manages with just four bits per number. This allows only 16 different values to be represented. For a long time, 16 or 32 bits per number were standard in AI systems. FP4 is thus a very coarse grid — deliberately chosen to save space and computing time.

Why it matters

Large language models consist of billions of numbers, known as parameters. These numbers are what the model has absorbed during training. They all have to fit into the memory of the graphics cards for the model to be able to respond. This memory is expensive and scarce.

Halving the bits per number also halves the memory requirement. A model that still needs two graphics cards at 8 bits fits onto a single one in FP4. In addition, smaller numbers can be computed faster. For companies answering millions of requests per day, this translates directly into saved electricity costs. That is precisely why FP4 has become a selling point for new computing chips.

How it works

The trick is called scaling. Four bits alone are only enough for 16 levels, roughly from minus six to plus six. But the actual values in the model lie in entirely different orders of magnitude. That’s why numbers are not converted individually, but always in small groups of about 16 or 32 numbers together. For each group, a conversion factor is stored alongside it, the scaling factor.

You can picture this like a map. The map itself is drawn coarsely, but a scale at the edge indicates how the figures should be read. The scale only applies to a small section of the map at a time. This way, even very large and very small values remain usably representable.

The price for this is inaccuracy. Every number is rounded to the nearest of the 16 levels. These small errors accumulate over many computational steps. If a model is converted too coarsely, it starts to mix up facts or fail at math. That’s why sensitive parts of the model often remain in a more precise format. This is called mixed precision. The process of converting a model into a coarser format is called quantization.

Where you encounter the term

Most commonly in chip manufacturers' announcements. Nvidia explicitly advertises its Blackwell generation as supporting FP4 directly in hardware. The record-breaking performance figures mentioned in this context usually refer to FP4 — with a coarser format, higher numbers are easier to achieve. Anyone comparing such figures should therefore always pay attention to the format. AMD and Google are pursuing similar approaches with their own chips.

FP4 variants also show up among freely available models. Anyone wanting to run a language model on an ordinary gaming PC often reaches for a heavily quantized version. It is noticeably weaker than the original, but it runs at all. In file names, you can recognize this by abbreviations like 4bit, NVFP4, or MXFP4. And in data center operators' quarterly reports, FP4 appears indirectly: as the reason why cost per request is falling.