Feature Importance

noun

say it: “FEE-cher im-POR-tunss” /ˈfitʃər ɪmˈpɔrtəns/

A score that says how much a model relies on each of its input features.

What is feature importance?

Feature importance is a score for how much a model relies on each feature. Tree models offer a built-in one, feature_importances_, which can overrate features with many different values; permutation importance, measured on test data, is the more trustworthy check. Neither says whether a feature causes anything.

from sklearn.tree import DecisionTreeRegressor
homes = [[900, 7], [1200, 2], [1500, 9], [1800, 1], [2000, 8], [2400, 3]]   # sqft, house number
bills = [60, 80, 95, 110, 125, 150]   # monthly energy bill, dollars
tree = DecisionTreeRegressor(random_state=0).fit(homes, bills)
print(tree.feature_importances_.round(2))   # [1. 0.]: all on square feet

In a sentence

  • “A high feature importance doesn't mean the feature causes the bill to rise.”
  • “The tree's feature importance put square footage far ahead of the year built.”

Level: Journeyman · More AI Basics terms at this level

Updated