Up Next: Predicting Categories

Look back at what Module 2 taught, fitting lines, splitting data, measuring error against a baseline, curves, trees and fair comparisons, try a short harvest guide for the orchard with all of it, and see what Module 3 adds, models that guess a category, such as spam or not, and ways to count which mistakes they make.

Journeyman · AI Basics · Python · Updated

By

After this lesson you can

  • Describe what classification will let you predict, and why counting each kind of mistake matters more than one score

Key takeaways

  • After Module 2 you can train models that guess a number, score them honestly on rows they never saw, and choose between them fairly.
  • A number's guess is close or far off; a category's guess is right or wrong, and which wrong answer it gives can matter more than how often.
  • Module 3 guesses categories with logistic regression, decision trees and nearest neighbors, and counts each kind of mistake with the confusion matrix, precision and recall.

When this module started, a model was an idea from Module 1. Now you've trained real ones. You can fit a line to home prices, hold back test homes, say in minutes how far off a delivery estimate runs, bend a model to follow an orchard's harvest, and tell a model that learned the pattern from one that memorized its examples. That's the daily work of anyone who builds models that predict amounts: prices, times, sales, harvests.

This lesson looks back at what you can do, gives you something to try with all of it, and shows what Module 3 adds: guesses that are categories instead of numbers.

What you can do now

After Module 2, you can:

  • fit a straight line by least squares, by hand and with LinearRegression, and read its slope and intercept in the label's units;
  • split before anything learns, with train_test_split, a test size and a fixed random_state;
  • read coefficients carefully, one feature at a time with the others held the same, never as a cause;
  • measure error in the label's own units with the MAE, weigh big misses with the RMSE, and compare with always guessing the average with R squared;
  • beat a baseline, scored on the same test rows, and read a residual plot for what a model is missing;
  • follow a curve with PolynomialFeatures, fit on the training rows only, and keep the degree low;
  • grow a regression tree with max_depth and random_state, and know why its guesses are steps;
  • compare models fairly: one split, one metric, a baseline, training error beside test error, and the simplest model that's good enough.

Try it: a harvest guide

A model is only useful when someone can act on it. Try writing a short harvest guide for Hilltop Orchard's manager, with the files from Exercise: Orchard Harvest Models and only what you know now:

  • The model you'd use, and its test MAE in pounds, from python report.py.
  • Where it misses most. In the notebook, compare the test trees' errors for dry trees (under 15 inches of rain) and soaked ones (over 45) with the rest.
  • One thing it can't tell the manager, such as what a 60-inch season would bring, and why.
  • One experiment: add avg_summer_temp_f to the features and compare again. Did the test error get better, worse or stay about the same? seed_data.py says what to expect.

Write it as a Markdown cell at the end of the notebook. There's no test for this one: when the orchard's manager could read it and know how far to trust each guess, it works.

What Module 3 unlocks

Plenty of questions aren't "how much?" but "which one?": is this email spam, is this fruit an apple, a pear or a plum, is this card payment fraud? Guessing a category is called classification, and Module 3 is about it:

  • Classification with logistic regression: a model that gives each answer a probability, and a line at 0.5 that turns it into a yes or a no.
  • Trees and neighbors: the trees you know, asking questions that end in a category instead of an average, and a model that guesses from the most similar examples it has seen.
  • Measuring a classifier: a guess is right or wrong, and the kind of wrong matters. Calling a real email spam (a false alarm) and letting a fraudulent payment through are different mistakes with different costs, so you'll count each kind with the confusion matrix, precision and recall. And one score can fool you when the answer is rare: if 2 payments in 100 are fraud, a model that never says fraud is right 98 times in 100 and catches none.

Here's a sneak peek, on eight made-up emails: the number of links in each, whether it mentions "free", and whether it was spam (1) or not (0):

import pandas as pd
from sklearn.linear_model import LogisticRegression

emails = pd.DataFrame({
    "num_links": [0, 1, 0, 2, 7, 9, 6, 8],
    "mentions_free": [0, 0, 0, 0, 1, 1, 0, 1],
    "is_spam": [0, 0, 0, 0, 1, 1, 1, 1],
})
model = LogisticRegression()
model.fit(emails[["num_links", "mentions_free"]], emails["is_spam"])

new_emails = pd.DataFrame({"num_links": [1, 8], "mentions_free": [0, 1]})
print(model.predict(new_emails))
[0 1]

The same three steps you know, make, fit and predict, and the guesses are categories: the first new email isn't spam, the second is. You don't need to understand every part of it yet: that's what Module 3 is for.

Check yourself

2 questions to try on the lesson page.

So what

Module 2 taught you to predict a number and to measure the guess honestly. Module 3 predicts categories, where the question becomes not only how often a model is wrong, but which way.

Sources

  • scikit-learn 1.9.1, LogisticRegression, API reference: a logistic regression classifier; predict gives a class label for each sample and predict_proba gives probability estimates.
  • scikit-learn, Tuning the decision threshold for class prediction, User Guide: in binary classification, predict predicts the positive class when the probability from predict_proba is greater than 0.5, a cut-off that isn't ideal for most uses.

Key terms

Related