Machine Learning
Topic 13 · Model quality · ⏱ about 5 h

Evaluation Metrics & Cross-Validation

Measuring performance honestly — accuracy is often not enough.

Show:
In this topic
Beginner

The confusion matrix

For a classification model, the starting point for every evaluation is the confusion matrix, a small table that counts the four possible outcomes of a yes/no prediction. Suppose the "positive" class is the thing we are trying to detect, such as fraud or disease.

PredictedpositivenegativeActualpositivenegativeTPcaughtFNmissedFPfalse alarmTNcorrectly ignoredRecall =TP / (TP + FN)Precision = TP / (TP + FP)
Recall reads along the "actually positive" row; precision reads down the "predicted positive" column.
Predicted positivePredicted negative
Actually positiveTrue Positive (TP): correctly detectedFalse Negative (FN): missed
Actually negativeFalse Positive (FP): false alarmTrue Negative (TN): correctly left alone

The simplest summary is accuracy, the fraction of all predictions that were correct, (TP+TN)/total(TP + TN)/\text{total}. Accuracy is easy to understand but can be badly misleading when the classes are imbalanced. If 99% of transactions are legitimate, a "model" that labels every transaction legitimate achieves 99% accuracy while catching no fraud whatsoever. We need measures that focus on the positive class.

Precision answers the question: "of all the cases the model flagged as positive, what fraction really were positive?"

Precision=TPTP+FP. \text{Precision} = \frac{TP}{TP + FP}.

High precision means few false alarms. It matters most when false alarms are costly, for example a spam filter that hides genuine emails.

Recall (also called sensitivity, or the true-positive rate) answers a different question: "of all the cases that really were positive, what fraction did the model catch?"

Recall=TPTP+FN. \text{Recall} = \frac{TP}{TP + FN}.

High recall means few misses. It matters most when missing a positive is costly, for example in cancer screening or fraud detection, where an undetected case can be very serious.

There is usually a tension between the two: flagging more cases catches more true positives (higher recall) but also raises more false alarms (lower precision). The F1 score combines them into a single number using the harmonic mean, which is only high when both are high:

Interactive: this part runs in your browser.

Breakdown. Each metric uses different cells of the confusion matrix. Pick a metric to see which.
F1=2⋅Precision⋅RecallPrecision+Recall. F_1 = \frac{2\cdot\text{Precision}\cdot\text{Recall}}{\text{Precision} + \text{Recall}}.
Beginner

Watch it: Cross-validation and time-series splits

A short narrated animation. Press ▶ Play, or step through it with ⏮ ⏭. Tick 🔊 narrate to hear each step.

Interactive: this part runs in your browser.

Try it

Try it: move the threshold

A model outputs a score for each example. Everything right of the threshold is predicted positive. Watch precision and recall trade off as you slide.

Interactive: this part runs in your browser.

Beginner

ROC, AUC & regression metrics

Most classifiers don't output a hard yes or no; they output a score or probability, and a threshold turns it into a decision. As the demo above showed, every threshold gives a different balance of precision and recall. Rather than judging a model at one arbitrary threshold, it is often more informative to see how it performs across all thresholds.

perfect: AUC = 1good: AUC ≈ 0.8weak: AUC ≈ 0.63random guessing: 0.5false-positive ratetrue-positive rate (recall)
ROC curves: each point is one threshold. Better models bow towards the top-left corner; the area under the curve (AUC) summarises them.

The ROC curve (receiver operating characteristic, a name inherited from radar engineering) does this by plotting the true-positive rate (recall) against the false-positive rate, FP/(FP+TN)FP/(FP + TN), as the threshold sweeps from high to low. A perfect classifier jumps straight to the top-left corner, catching every positive before raising any false alarm, while a classifier that guesses at random follows the diagonal. The area under the ROC curve (AUC) condenses the curve into one number: 1.0 for a perfect classifier and 0.5 for random guessing. Because it considers every threshold, AUC measures how well the model ranks positives above negatives, independently of where you eventually set the threshold. When the positive class is very rare, the precision–recall curve is often more revealing than the ROC curve, because the ROC curve's false-positive rate can look reassuringly small simply because the number of negatives is huge.

For regression problems, where the target is a number, three measures are standard. The mean absolute error (MAE) averages the absolute size of the errors; it is easy to interpret and not dominated by a few large errors. The mean squared error (MSE) averages the squared errors, so it penalises large errors heavily; its square root, the RMSE, is in the same units as the target, which makes it easier to interpret. And R2R^2 reports the fraction of the variation in the target that the model explains, where 1 is perfect and 0 means no better than predicting the average.

Beginner

k-fold cross-validation

A single train/validation split gives a single number, and that number depends partly on luck: which examples happened to land in the validation set. With small or medium datasets, a lucky or unlucky split can easily make one model look better than another when it isn't. K-fold cross-validation reduces this luck by using every example for validation exactly once.

The procedure is as follows. First, divide the training data into kk equal parts, called folds; k=5k = 5 or k=10k = 10 is typical. Then train the model kk times. Each time, one fold is held out as the validation set and the model is trained on the other k−1k - 1 folds. This gives kk validation scores, which we average to get the cross-validation estimate. The diagram shows the five rounds of 5-fold cross-validation, with the orange block being the validation fold each time.

Round 1Round 2Round 3Round 4Round 5

Besides a more reliable average, cross-validation tells you how much the score varies from fold to fold, which is a useful warning sign: if the fold scores range from 70% to 90%, a reported average of 80% hides a lot of uncertainty.

The way the folds are formed should match the structure of the data. For classification, use stratified k-fold, which keeps the class proportions the same in every fold. For time series, use a time-series split, which always trains on earlier data and validates on later data. When one person or entity contributes several rows, use group k-fold, which keeps all of an individual's rows in the same fold, so the model can't score well simply by recognising people it has already seen.

Cross-validation is also the standard tool for tuning hyperparameters. You choose a set of candidate settings, compute the cross-validation score for each, and pick the best. Grid search tries every combination from a list of values. Random search tries random combinations, which is usually more efficient, because typically only a few hyperparameters really matter and random search explores more distinct values of each. Bayesian optimisation goes further by using the results so far to decide which settings to try next.

One rule still applies: even with cross-validation, keep a final test set that is never used for any decision, and evaluate on it once at the end. If you need an unbiased estimate of how well your whole tuning procedure performs, use nested cross-validation, which is explained in the Advanced section.

Interactive: this part runs in your browser.

Interactive. Step through k-fold cross-validation one round at a time.
Intermediate · Maths

Metric formulas and what AUC really means

📐 Maths you'll need: m-prob
💡 The idea in plain words

Each metric answers a different question. Precision: "when I raise an alarm, am I usually right?" Recall: "do I catch most of the real cases?" AUC: "do I rank positives above negatives?" And the best threshold isn't always 0.5: it depends on how much each kind of mistake costs.

🔤 Symbols in this section
TP,FP,TN,FNTP, FP, TN, FNtrue/false positives and negatives
FβF_\betaF-score weighting recall β times as much as precision
s(x)s(x)model score
x+, x−x^+,\ x^-a random positive and a random negative example
cFP, cFNc_{FP},\ c_{FN}cost of a false positive / false negative
1[⋅]\mathbb 1[\cdot]indicator: 1 if true, 0 if false

This section gathers the precise definitions of the common metrics, explains a surprising interpretation of AUC, and shows how to choose a decision threshold from the costs of mistakes.

The four basic rates are all ratios taken from the confusion matrix:

Precision=TPTP+FP,Recall (TPR)=TPTP+FN \text{Precision} = \frac{TP}{TP+FP}, \qquad \text{Recall (TPR)} = \frac{TP}{TP+FN}
FPR=FPFP+TN,Specificity=1−FPR \text{FPR} = \frac{FP}{FP+TN}, \qquad \text{Specificity} = 1 - \text{FPR}

The F1 score weights precision and recall equally. When one matters more than the other, the F-beta score lets you choose the balance:

Fβ=(1+β2) P⋅Rβ2P+R. F_\beta = (1+\beta^2)\,\frac{P\cdot R}{\beta^2 P + R}.

A value of β>1\beta > 1 gives recall more importance, so the F2F_2 score is popular when missing positives is costly, while β<1\beta < 1 favours precision.

For imbalanced data, a good single-number summary is the Matthews correlation coefficient, which uses all four cells of the confusion matrix and is essentially a correlation between predictions and truth:

MCC=TP⋅TN−FP⋅FN(TP+FP)(TP+FN)(TN+FP)(TN+FN). \text{MCC} = \frac{TP\cdot TN - FP\cdot FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}.

It ranges from −1 (always wrong) through 0 (no better than chance) to +1 (perfect), and unlike accuracy it cannot be inflated simply by predicting the majority class.

Some metrics judge the predicted probabilities rather than the decisions. Log-loss, −1n∑[ylog⁡p+(1−y)log⁡(1−p)]-\frac1n\sum[y\log p + (1-y)\log(1-p)], heavily punishes confident mistakes. The Brier score, 1n∑(pi−yi)2\frac1n\sum(p_i - y_i)^2, is the mean squared error of the probabilities. Both reward probabilities that are honest as well as accurate.

What AUC really measures. There is an elegant interpretation of the area under the ROC curve: it equals the probability that a randomly chosen positive example receives a higher score than a randomly chosen negative example.

AUC=P(s(x+)>s(x−))=1n+n−∑i∈+∑j∈−1[si>sj]. \text{AUC} = P\big(s(x^+) > s(x^-)\big) = \frac{1}{n_+ n_-}\sum_{i\in +}\sum_{j\in -}\mathbb 1[s_i > s_j].

The formula on the right simply checks every positive–negative pair and counts the fraction in which the positive is ranked higher (the indicator 1[⋅]\mathbb 1[\cdot] equals 1 when its condition is true). Statisticians will recognise this as the normalised Mann–Whitney U statistic. The interpretation makes clear that AUC is purely about ranking. It is unaffected by whether the scores are well-calibrated probabilities, and it says nothing about which threshold to use.

Choosing the threshold from costs. The default threshold of 0.5 is only sensible if the two kinds of mistake are equally costly, which is rarely true. Suppose a false positive costs cFPc_{FP} and a false negative costs cFNc_{FN}, and the model's probability pp is well calibrated. Predicting "positive" has an expected cost of (1−p) cFP(1-p)\,c_{FP} (we are wrong if the case is negative), while predicting "negative" has an expected cost of p cFNp\,c_{FN}. Predicting positive is the better choice when (1−p)cFP<p cFN(1-p)c_{FP} < p\,c_{FN}, which rearranges to

p>cFPcFP+cFN. p > \frac{c_{FP}}{c_{FP} + c_{FN}}.

For example, if missing a cancer is judged nine times worse than a false alarm, the threshold should be 1/(1+9)=0.11/(1 + 9) = 0.1, not 0.5: any patient with more than a 10% estimated risk should be referred.

For problems with more than two classes, per-class metrics are combined in one of three ways. A macro average computes the metric for each class and averages them, treating every class as equally important. A micro average pools all the true and false positives across classes first, treating every example as equally important. A weighted average is a macro average weighted by the size of each class.

Key takeaway. Pick the metric that matches the real-world question, remember AUC measures ranking only, and set the threshold from the costs: predict positive when p > c_FP / (c_FP + c_FN).
Advanced

Cross-validation theory, nested CV and comparing models

📐 Maths you'll need: m-prob · m-stats
💡 The idea in plain words

Cross-validation gives an estimate of performance, and every estimate has uncertainty. If you tune hyperparameters using the same folds you report, you're quietly picking the luckiest result. Nested CV keeps an honest outer test, and proper statistical tests tell you whether one model is really better than another.

🔤 Symbols in this section
kknumber of folds
f^(−j)\hat f^{(-j)}model trained without fold j
b, cb,\ ccases where only model A / only model B is right
χ2\chi^2chi-squared test statistic
ECE\text{ECE}expected calibration error
accb, confb\text{acc}_b,\ \text{conf}_baccuracy and mean confidence in bin b

Cross-validation produces an estimate of performance, and like every estimate it has a bias and a variance. Understanding these helps you avoid reading too much into small differences, and helps you avoid one particularly common trap.

The k-fold estimate is the average of the kk fold errors, Err^CV=1k∑j=1kErr(f^(−j),foldj)\widehat{\text{Err}}_{CV} = \frac1k\sum_{j=1}^k \text{Err}(\hat f^{(-j)}, \text{fold}_j), where f^(−j)\hat f^{(-j)} is the model trained without fold jj. Its bias comes from the fact that each model is trained on only a fraction k−1k\frac{k-1}{k} of the data, so it is slightly worse than a model trained on everything; cross-validation is therefore slightly pessimistic, and using more folds reduces this. Its variance is more subtle. The kk fold scores are not independent, because their training sets overlap heavily, so the usual standard-error formula s/ks/\sqrt k underestimates the real uncertainty. Leave-one-out cross-validation (k=nk = n) is almost unbiased, but its nn models are nearly identical, which can make its estimate highly variable. Five or ten folds is a good compromise, and repeating k-fold several times with different random splits further reduces the variance.

The tuning trap. Suppose you try fifty hyperparameter settings and report the cross-validation score of the best one. That score is optimistically biased, because by picking the best of fifty noisy estimates you have partly selected the setting that got lucky on these particular folds. The remedy is nested cross-validation. An outer loop splits the data into folds as usual. Inside each outer training set, an inner cross-validation loop performs the entire tuning procedure and picks the best setting. That chosen model is then evaluated on the outer fold, which played no part in the tuning. The outer scores therefore measure how well the whole procedure, tuning included, performs on genuinely unseen data.

Is model A really better than model B? To compare two classifiers on the same test set, McNemar's test looks only at the examples where they disagree. Let bb be the number of examples that A gets right and B gets wrong, and cc the number that B gets right and A gets wrong. If the models were equally good, bb and cc should be similar, and the statistic χ2=(∣b−c∣−1)2b+c\chi^2 = \frac{(|b - c| - 1)^2}{b + c} measures how surprising the observed imbalance is (it is compared with a chi-squared distribution with one degree of freedom). To compare models across cross-validation folds, the corrected resampled t-test of Nadeau and Bengio adjusts the ordinary t-test for the overlap between training sets, by inflating the variance by a factor (1k+ntestntrain)\big(\frac1k + \frac{n_{\text{test}}}{n_{\text{train}}}\big). Without such corrections, small differences look far more significant than they are.

Checking calibration. A classifier is calibrated if its probabilities match reality: among all the cases where it predicts 70%, about 70% should turn out positive. To check this, group the predictions into bins by their predicted probability and, for each bin, plot the average predicted probability against the fraction that were actually positive. This plot is called a reliability diagram, and a calibrated model lies along the diagonal. The expected calibration error summarises the plot as a weighted average gap, ECE=∑bnbn ∣accb−confb∣\text{ECE} = \sum_b \frac{n_b}{n}\,|\text{acc}_b - \text{conf}_b|, where confb\text{conf}_b is the average predicted probability in bin bb and accb\text{acc}_b the observed fraction of positives.

Key takeaway. CV estimates are noisy and tuning makes them optimistic; use nested CV for honest numbers, McNemar or corrected t-tests to compare models, and reliability diagrams to check calibration.
Code

Code: metrics, ROC/PR, tuning and nested CV

Everything from this topic in scikit-learn, including a check that AUC equals the pairwise ranking probability.

Python · scikit-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import (train_test_split, StratifiedKFold,
                                     GridSearchCV, cross_val_score)
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import (confusion_matrix, precision_score, recall_score,
                             f1_score, matthews_corrcoef, roc_auc_score,
                             average_precision_score, brier_score_loss)

X, y = make_classification(n_samples=2000, weights=[0.9, 0.1], random_state=0)
Xtr, Xte, ytr, yte = train_test_split(X, y, stratify=y, test_size=0.3, random_state=0)
model = RandomForestClassifier(n_estimators=200, random_state=0).fit(Xtr, ytr)
p = model.predict_proba(Xte)[:, 1]

for thr in [0.5, 0.3, 0.1]:                         # moving the threshold
    pred = (p >= thr).astype(int)
    print(f"thr={thr}: P={precision_score(yte, pred):.2f} R={recall_score(yte, pred):.2f} "
          f"F1={f1_score(yte, pred):.2f} MCC={matthews_corrcoef(yte, pred):.2f}")
print(confusion_matrix(yte, (p >= 0.5).astype(int)))
print("ROC-AUC:", round(roc_auc_score(yte, p), 3),
      " PR-AUC:", round(average_precision_score(yte, p), 3),
      " Brier:", round(brier_score_loss(yte, p), 4))

# AUC == P(score of random positive > score of random negative)
pos, neg = p[yte == 1], p[yte == 0]
pairwise = (pos[:, None] > neg[None, :]).mean() + 0.5 * (pos[:, None] == neg[None, :]).mean()
print("pairwise ranking probability:", pairwise.round(3))

# Nested CV: inner loop tunes, outer loop estimates performance honestly
inner = StratifiedKFold(3, shuffle=True, random_state=1)
outer = StratifiedKFold(5, shuffle=True, random_state=2)
search = GridSearchCV(RandomForestClassifier(random_state=0),
                      {"max_depth": [3, 6, None], "min_samples_leaf": [1, 5]},
                      cv=inner, scoring="roc_auc")
nested = cross_val_score(search, X, y, cv=outer, scoring="roc_auc")
print("nested CV AUC: %.3f ± %.3f" % (nested.mean(), nested.std()))
Open in playground ↗
Check yourself

Quiz

1. For cancer screening, which metric matters most?

2. A model that always predicts "not fraud" on data with 1% fraud has accuracy…

3. An AUC of 0.5 means…

4. ROC-AUC equals the probability that…

5. A false negative costs 9× a false positive. With calibrated probabilities, the best threshold is…

6. Why use nested cross-validation?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready