Cross-Validation

noun

say it: “KRAWS val-ih-DAY-shun” /ˈkrɔs ˌvæləˈdeɪʃən/

A way of testing a model by training and scoring it several times, each time holding back a different part of the data, and averaging the results.

What is cross-validation?

Cross-validation scores a model several times, each time training on most of the data and testing on a different held-back part, then averages the scores. One lucky or unlucky split can't decide the result, and every row gets tested once. scikit-learn's cross_val_score() does it.

from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LinearRegression
temps = [[60], [64], [68], [72], [76], [80], [84], [88], [92], [96]]
cups = [20, 27, 37, 43, 52, 61, 66, 77, 83, 92]
scores = cross_val_score(LinearRegression(), temps, cups, cv=5, scoring="neg_mean_absolute_error")
print(len(scores), round(-scores.mean(), 2))   # 5 0.95: five scores, off by 0.95 cups on average

In a sentence

  • “Cross-validation gave five scores instead of one lucky one.”
  • “She compared the two models by their cross-validation averages.”

Level: Journeyman · More AI Basics terms at this level

Updated