Compute

Every computer performs tasks by executing computational steps. Compute is the English word for exactly this computational work, and the term has become established in German as well. What is meant is the amount of computational steps a task consumes, and the equipment used to carry it out. Compute can therefore be read in two directions: as the demand of a task and as the reserve of a company. A comparison helps: compute relates to a computational task the way electricity relates to a factory. It can be measured, bought, consumed – and it can become scarce.

Compute as the AI industry’s most expensive resource

Modern AI systems require an absurd amount of computational work. A large language model like the technology behind ChatGPT learns from enormous amounts of text. This learning process, the training, runs for weeks on thousands of specialized chips simultaneously. The bill for a single top-tier model runs into several hundred million dollars. This is the reason why there are only a handful of companies that can build such models at all.

That’s why compute is one of the industry’s most important metrics for investors. Whoever possesses a lot of it can train larger models and serve more customers. Companies report in their quarterly earnings how many billions they are pouring into data centers. The chipmaker Nvidia became one of the world’s most valuable companies precisely for this reason: it sells the graphics processors, i.e. the specialized chips, on which this computing takes place.

Scarcity plays a major role here. There are waiting lists for chips, disputes over power connections, and political export bans on high-performance chips to China. Compute is thus not just a technical quantity, but a matter of economic policy.

How computational work is measured and categorized

Computational work is usually counted in computational operations. A common unit is FLOP, which stands for a single floating-point operation. Training large models involves numbers with more than twenty zeros. Such figures sound abstract, but they are practical: they let you compare two models without knowing the providers' prices. The EU’s AI regulations also use such a threshold to identify particularly powerful models.

How much compute a task requires roughly depends on two things. First, the size of the model, meaning the number of its adjustable parameters. Second, the amount of data it learns from. Doubling both means needing roughly four times the computational work. This is exactly where the limit to growth lies: models don’t grow arbitrarily larger because compute isn’t arbitrarily available.

A common misconception is confusing compute with storage space. Storage indicates how much data fits. Compute indicates how fast and how often calculations can be performed. A hard drive full of text does not replace a powerful processor.

Where compute shows up in news and products

In everyday life, you notice compute through limits. Free AI services cap the number of requests per day or respond more slowly during peak times. Paid versions get more compute time and are allowed to use the more powerful models. Even the phrase that a model is now allowed to “think longer” is a statement about compute: it performs more steps per answer.

In business news, you encounter the term in investment plans. Reports about new data centers, chip orders, or power contracts with power plants all revolve around compute. Cloud providers such as Amazon, Microsoft, and Google rent out computing power by the hour, allowing even small companies to participate.

If you want to compute something yourself, by the way, that’s possible for free. Services like Google Colab provide a graphics processor in the cloud for a few hours a day. That’s enough to train a small model – a top-tier model, of course, not.