Human Evaluation

When a program writes a text, someone has to decide whether that text is good. For a math problem this happens automatically: the result is right or wrong. For a summary, a translation, or a piece of advice, there is no single correct solution. Human Evaluation is the approach of letting real people judge in such cases. They read the outputs and assign scores, or they compare two versions and say which one is better. The result is a number that developers can use to compare their systems.

Why test questions with a reference answer are not enough

The usual way of measuring AI systems is so-called benchmarks. These are fixed collections of tasks with stored reference answers, often multiple-choice questions. Such tests run through in minutes and cost almost nothing. Their problem: they only measure what can be clearly checked off.

A system can score highly on such tests and still be unpleasant in everyday use. It answers in a roundabout way, ignores the question, or invents details. Yet these are exactly the traits that determine whether people actually want to use a product. Only a human reliably notices that an answer, while formally correct, misses the point.

A second reason is contamination of test data. Because benchmarks are publicly available on the internet, their questions can end up in a model’s training data. In that case the model has already seen the answers before, and the test measures memory rather than ability. Fresh judgments from humans cannot be gamed in this way.

From rating sheet to score

At the start there is a set of instructions, often called a guideline. It specifies what the raters should pay attention to: Is the answer factually correct? Does it answer the question? Is it phrased clearly? Without such clear criteria, everyone rates by gut feeling, and the results are useless.

Particularly common is the direct comparison of two answers. The rater sees the same question answered twice, but doesn’t know which system produced which answer, and picks the better one. Humans can make such pairwise comparisons far more reliably than absolute scores from one to five. Many individual duels then combine into a ranking, similar to chess ratings.

For a result to be robust, several people usually rate the same answer. If they agree, that’s a sign of clear criteria. If they constantly disagree, the task was poorly framed. This degree of agreement is measured separately and published alongside the results. The price for this rigor is high: thousands of ratings from paid experts can easily cost five-figure sums and take weeks.

Leaderboards, thumbs icons, and the special case of AI as judge

The principle is most visibly encountered in public comparison arenas. There, visitors ask a question, receive two anonymous answers, and vote. Millions of such votes produce leaderboards that are regularly cited in news coverage of new models. When a company claims that its model is preferred by users, such an evaluation is almost always behind that claim.

Chat programs also involve constant rating. The small thumbs-up and thumbs-down icons under an answer are a simplified form of Human Evaluation. This feedback later flows into training and shifts the system’s behavior in the preferred direction.

Because human evaluation is so expensive, developers increasingly let a powerful AI model judge the answers of other models. This method is called LLM-as-a-Judge and is considerably cheaper. But it has blind spots: such judges favor long, confident-sounding texts and overlook mistakes they would make themselves. Human judgments therefore remain the standard against which automatic judges are first measured.