Observability

A large computer program doesn’t run on a single machine but is distributed across many. From the outside, at first you only see the result: the page loads, or it doesn’t. Observability is the property of such a system that makes its internal state visible from the outside. To achieve this, the system continuously records data about itself: how long a request takes, which steps it went through, where an error occurred. The term originates from control theory, where it refers to whether the state of a machine can be inferred from its measurements. Translated, observability simply means the ability to be observed.

Why outages otherwise turn into guesswork

In the past, an application often consisted of a single large program. If something went wrong, you looked in one place. Today, an online shop consists of dozens of small services that call one another. A slow checkout could be caused by the database, the payment service, or the network in between. Without recorded data, all that remains is guessing.

The economic pressure is considerable. Large providers calculate outages in terms of revenue per minute. What matters, therefore, is not just that an error is noticed, but how quickly the cause is found. In the industry, this timespan is called Mean Time to Resolution, i.e. the average duration until a fix is made. Good observability reduces it from hours to minutes.

A common misconception: observability is the same as monitoring, i.e. classic surveillance. Monitoring answers questions that have been asked in advance, such as whether the hard drive is full. Observability is meant to also answer questions that nobody thought of beforehand. It is therefore the more demanding task.

Metrics, logs, and traces

Usually, three types of data are distinguished. Metrics are numbers over time, such as 4,200 requests per minute or 63 percent memory utilization. They are sparse and well suited for charts and alerts. Logs are lines of text that a program writes when something noteworthy happens. They contain details but quickly grow into enormous volumes of data.

The third type is called tracing. Here, each request is assigned a unique number when it enters the system, which is passed along to all services involved. In the end, the complete path of a single order can be reconstructed, including the duration of each individual step. This works like package tracking: you don’t just see that it arrived late, but at which distribution center it got stuck.

So that different programs speak the same language, there is the open standard OpenTelemetry. It specifies the format in which such data is generated and transmitted. It is then collected in specialized systems that keep billions of entries searchable. The cost factor is real: at some companies, storing observability data makes up a noticeable part of the IT bill.

From cloud bills to AI applications

In business news, observability usually appears via company names. Datadog, Splunk, Grafana, New Relic, and Dynatrace sell exactly these kinds of tools. Splunk was acquired by Cisco in 2024 for around 28 billion dollars, which shows how large this market has become. The major cloud providers also offer their own services.

A new branch is LLM observability. A language model doesn’t always give the same answer to the same question, and an error is rarely a clear-cut crash. Therefore, things like response time, cost per request, and whether the model fabricated sources are observed. Without such records, it is hardly possible to prove whether an AI product works reliably.

Anyone who wants to see this in practice doesn’t need a data center. The developer tools in every browser show, under the Network tab, which file of a web page loaded and how long it took. This is observability on a small scale, based on the same basic idea: don’t guess, check.