Preparing Data in a Pipeline

Models want numbers on similar scales, with no gaps. Scale numeric features, turn categories into columns a model can use, and fill in missing values, each learned from the training data only. Then put every step and the model into one pipeline, with a column transformer that treats numbers and categories differently, so the same steps run every time. The running example is a pet adoption center.

Updated

Numbers and Categories

  1. Scaling Features (Journeyman): Features measured in different units (pounds, years, a 0 or 1) pull unevenly on any model that measures distance or penalizes large weights, so standardizing rescales each one with the training data's mean and standard deviation, which StandardScaler learns with fit on the training part and applies with transform to both parts.
  2. Turning Categories into Numbers (Journeyman): Numbering categories such as dog, cat and rabbit 1, 2 and 3 invents an order and distances that aren't there, so one-hot encoding gives each category a column of its own, which OneHotEncoder learns from the training data, with handle_unknown='ignore' for a category it never saw, while OrdinalEncoder fits categories that really have an order, given in that order.
  3. Missing Values in a Model (Journeyman): Most scikit-learn models stop with an error when a feature holds NaN, and dropping those rows throws away examples and leaves a new pet with a blank unscorable, so SimpleImputer fills each blank with a value learned from the training data (the median for numbers, the most frequent value for categories), and add_indicator=True keeps a column saying which values were missing.

Pipelines

  1. Pipelines: Every Step, Every Time (Journeyman): A pipeline chains the preparation steps and the model into one object, so make_pipeline(SimpleImputer(), StandardScaler(), LogisticRegression()) fits every step on the data it's given with one fit, repeats the same steps with one predict, names each step for named_steps, and refits every step inside each fold when it's handed to cross_val_score.
  2. Different Steps for Different Columns (Freelancer): A ColumnTransformer sends each list of columns through its own steps, so the number columns are filled with the median and standardized while the category columns are filled with the most frequent value and one-hot encoded, columns it isn't given are dropped by default, and as the first step of a pipeline it lets one fit and one predict handle a whole table, with get_feature_names_out listing the columns the model gets.

Exercise: Adoption Pipeline

  1. Exercise: Adoption Pipeline (Freelancer): Write three functions that build a ColumnTransformer to fill, scale and one-hot encode Happy Tails' 800 pets, put it in a pipeline with logistic regression, and train it on the training pets only, and let pytest check the steps, their settings, what each step learned, and a guess for a pet with a blank weight and a new color.