Validation Set
noun
say it: “val-ih-DAY-shun SET” /ˌvæləˈdeɪʃən ˈsɛt/
A part of the data held back from training and used to compare models or choose settings, separate from the final test data.
What is a validation set?
A validation set is rows held back to compare models and choose settings, such as a tree's depth. Because you keep whatever does best on it, its score ends up a little flattering, so a separate test set is saved for one final, honest score. Cross-validation can stand in for it.
from sklearn.model_selection import train_test_split
rows = list(range(100))
rest, test = train_test_split(rows, test_size=0.2, random_state=0)
train, valid = train_test_split(rest, test_size=0.25, random_state=0)
print(len(train), len(valid), len(test)) # 60 20 20
In a sentence
- “The test set stayed untouched until the validation set had picked the model.”
- “She tried depths 2 to 10 on the validation set and kept 4.”
Level: Journeyman · More AI Basics terms at this level
Updated