Imputation

noun

say it: “im-pyoo-TAY-shun” /ˌɪmpjuˈteɪʃən/

In statistics, filling in missing values with reasonable estimates worked out from the rest of the data.

What is imputation?

Imputation is filling in missing values with stand-ins worked out from the data itself, such as the median of the values that are there, or each group's median. It keeps rows you'd otherwise lose, but the filled values are guesses, so they're flagged rather than passed off as measured.

import pandas as pd
intake = pd.read_csv("intake.csv", na_values=["unknown", "-"])
intake["age_filled"] = intake["age_years"].isna()      # flag first
intake["age_years"] = intake["age_years"].fillna(intake.groupby("species")["age_years"].transform("median"))

In a sentence

  • “The report noted that 52 ages came from imputation, not from the vet.”
  • “Using each species' median for imputation was fairer than one median for all.”

Level: Journeyman · More Python terms at this level

Updated