Looking Inside a Model
Looking Inside a Model: Reading Coefficients and Trees, Permutation Importance, How a Guess Changes with One Feature.
Updated
- Reading Coefficients and Trees (Journeyman): A linear regression trained on standardized features gives coefficients you can compare, each the change in the guess for one standard deviation more of its feature with the others held the same, but features that move together share and swap credit, a small tree's top splits show the questions it asks first, and neither a coefficient nor a split is a cause.
- Permutation Importance (Freelancer): To see which features any fitted model leans on, shuffle one column of held-out data so its link to the label is broken, score the model again, and take the difference, which scikit-learn's permutation_importance repeats n_repeats times with a random_state, giving a fairer answer than a tree's built-in feature_importances_, though features that move together can each look less important than they are.
- How a Guess Changes with One Feature (Freelancer): Changing one feature of a home while keeping the rest the same, and watching the model's guess, gives that home's individual line, and averaging every home's line gives the partial dependence, which PartialDependenceDisplay.from_estimator draws with kind='both', but it shows how the model behaves, not cause and effect, and it can't be trusted where the data is thin.