Schematischer Aufbau eines Nvidia-Grafikchips: Der Chip enthält ein Raster aus vielen Streaming-Multiprozessoren (SMs). Ein einzelner SM ist vergrößert dargestellt und zeigt seine internen Bestandteile: mehrere Gruppen von CUDA-Cores, Tensor-Cores und einen gemeinsamen Shared Memory.

Streaming Multiprocessor

A streaming multiprocessor is the fundamental computing unit in an Nvidia graphics chip that executes hundreds of computing operations simultaneously. The more streaming multiprocessors a chip has, the more parallel computations it can handle at once — crucial for AI training and graphics rendering.

A streaming multiprocessor — often abbreviated as SM — is the fundamental building block of an Nvidia graphics chip. You can picture it as a compact factory floor: inside, hundreds of computing units work simultaneously, each on its own small task. The chip itself contains not just one, but dozens to hundreds of such factory floors side by side. Together, they make it possible to process millions of individual calculations per second in parallel — that is, simultaneously rather than one after another. This fundamentally distinguishes a graphics chip from a computer’s normal main CPU, which has only a few cores, albeit very versatile ones.

Why the SM is the heart of modern AI chips

AI training consists for the most part of matrix multiplication — the mass multiplication and addition of long strings of numbers. That is exactly what streaming multiprocessors are built for. The more SMs a chip has, the more of these computing operations run simultaneously, and the faster a model learns.

A modern Nvidia H100 chip, which currently counts as the industry standard for AI training, contains 132 streaming multiprocessors. Nvidia states a computing performance of nearly 4,000 teraflops for this chip — that’s four quadrillion computing operations per second. This figure would simply be unattainable without the massive parallelization enabled by the SMs. That is why the number of streaming multiprocessors is one of the first metrics researchers and developers consult when comparing AI chips.

The same principle applies to graphics computations: every pixel of a 3D-rendered frame can be calculated independently of its neighboring pixels. Many parallel SMs handle this task in a fraction of a millisecond — fast enough for smooth moving images.

What’s inside a streaming multiprocessor

Each SM contains several groups of so-called CUDA cores — these are the individual computing units that perform simple addition and multiplication operations. Alongside these are specialized tensor cores, which are specifically optimized for the matrix operations used in AI training and deliver far more computing work per clock cycle than an ordinary CUDA core.

All the computing units of an SM share a fast, small intermediate memory known as shared memory. This is physically located right next to the computing units on the chip. Data that is needed frequently stays there instead of having to access the slower main memory every time. This is a decisive speed advantage, because transporting data to main memory often takes more time than the actual calculation itself.

Tasks are divided into so-called warps — groups of 32 threads each, which the SM always executes together and in sync. If a warp briefly has to wait for data, the SM immediately switches to another warp that is already ready. This way, the computing units are almost never idle.

Streaming multiprocessors in products and headlines

Anyone reading tech news about Nvidia announcing a new chip generation with “more SMs and new tensor cores” is directly encountering the subject of this entry. The number of streaming multiprocessors largely determines how expensive a chip is and how it is positioned in the market. Nvidia’s chip series — from the consumer GeForce line for gamers to the datacenter line H100 or B200 for data centers — differ, among other things, in how many SMs are unlocked.

While gaming, the SM is already present without you noticing: every time a game engine requests light reflections or shadow calculations, the driver distributes this work across the available SMs. A gaming GPU like the GeForce RTX 4090 has 128 SMs — almost as many as the professional H100, albeit with less memory and without some AI-specific features.

The SM also appears indirectly in the discussion about AI sovereignty and export restrictions: the US government has set upper limits on computing performance, above which chips may no longer be exported without approval. These limits are ultimately based on metrics to which the number and performance of streaming multiprocessors also contribute.

Related Products

Latest News

Subscribe free. Unsubscribe the second it sucks.

High-signal news across AI, business, UX, and tech. Every morning.