Data Leakage
noun
say it: “DAY-tuh LEE-kij” /ˈdeɪtə ˈlikɪdʒ/
In machine learning, when information that wouldn't be available in real use gets into a model's training or testing, making its score look better than it really is.
What is data leakage?
Data leakage is when information the model won't have in real use sneaks into training or testing, so its score looks better than it will turn out: a feature that's only known after the answer, a scaler fitted on the test rows too, or one customer's rows on both sides. The fix is to split first and keep the test rows out of every step.
In a sentence
- “Data leakage explained the 99% test score: the refund column was only filled in after a fraud was confirmed.”
- “Fitting the scaler before the split was a quiet form of data leakage.”
Level: Journeyman · More AI Basics terms at this level
Updated