What Classification Is

Classification guesses a category instead of a number, such as spam or not spam, with two classes (binary) or more (multiclass) and often one class you're looking for (the positive class), and a scikit-learn classifier lists its classes in classes_ and gives back one of them for each row from predict.

Apprentice · AI Basics · Python · Updated

By

After this lesson you can

  • Tell a classification problem from a regression problem, and name its classes and its positive class
  • Train a classifier with fit, read its classes_, and get one class for each new row with predict

Key takeaways

  • Classification guesses a category, one of a fixed set of answers, instead of a number. Each possible answer is a class.
  • Two classes is binary classification, and more than two is multiclass. The positive class is the one you're looking for, here spam.
  • A fitted classifier lists its classes in classes_, in sorted order, and predict gives back one of those classes for each row.
  • A classifier's score is the share of its guesses that are right. When one class is much more common, a guess that never says spam scores well too.

Inbox Guard, a small email service, wants a spam filter: every new email should land in the inbox or in the spam folder. Nobody can sort thousands of emails a day by hand, but the service has 1,000 emails people already marked. Sorting things into groups from examples like these is one of the most common jobs in machine learning: spam or not, will a member renew or not, which fruit is on the conveyor belt. You'll use this lesson's idea every time the answer you need is a category.

The example for the whole chapter is Inbox Guard's emails.csv, our own made-up data, which comes with Exercise: Spam or Not. Each code block below is one notebook cell, and the cells run in order: the first one loads the data, and the others use what it made.

"How much?" or "which one?"

Module 2's models guessed numbers: a price, a delivery time, a harvest. A guess of $307,366 for a home that sold for $310,000 is close, and closeness is what MAE and RMSE measured.

A spam filter asks a different question. An email is spam or it isn't; there's no "a little bit spam" folder. Guessing a category, one of a fixed set of answers, is classification, and a model that does it is a classifier. Kinds of Machine Learning drew the line: regression when the answer is a number, classification when it's a category. A classifier's guess is either the right category or the wrong one, so "how close?" stops being the question.

Classes, and the one you're looking for

Each possible answer is a class. Inbox Guard's label, the column is_spam, holds 1 for spam and 0 for not spam:

import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

emails = pd.read_csv("emails.csv")
print(emails["is_spam"].value_counts())
is_spam
0    701
1    299
Name: count, dtype: int64

701 emails are fine and 299 are spam, about 30%. Two classes make binary classification. More than two, such as apple, pear or plum, make multiclass classification, which the next chapter's fruit sorter does.

In a binary problem, one class is usually the thing you're hunting for: spam, fraud, a member about to leave. That's the positive class. "Positive" doesn't mean good; a spam email is the positive case here because it's what the filter looks for. scikit-learn's own rule, unless you tell it otherwise, is that the class with the larger label is positive: of 0 and 1, it's 1.

A first classifier

The make, fit, predict steps from Your First scikit-learn Model: fit and predict work the same way for a classifier. This one is LogisticRegression, which the next lesson opens up; for now it's a black box that learns from five columns about each email:

FEATURES = ["num_links", "num_exclamations", "mentions_free", "sender_in_contacts", "subject_all_caps"]
X = emails[FEATURES]
y = emails["is_spam"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LogisticRegression()
model.fit(X_train, y_train)
print("Classes:", model.classes_)
print("Guesses for eight test emails:", model.predict(X_test.head(8)))
print("What they really were:", y_test.head(8).tolist())
Classes: [0 1]
Guesses for eight test emails: [1 0 0 0 0 0 0 1]
What they really were: [0, 0, 1, 1, 0, 0, 0, 1]

classes_ (read "classes underscore") lists the classes the model learned, in sorted order. predict gives back one of them for each email, never a number in between. Five of these eight guesses are right: the first email was called spam and wasn't, and two spam emails got through.

The classes can be words

The label doesn't have to be numbers. Give fit words, and classes_ and predict use the words:

words = y_train.map({0: "not spam", 1: "spam"})
word_model = LogisticRegression()
word_model.fit(X_train, words)
print("Classes:", word_model.classes_)
print(word_model.predict(X_test.head(3)))
Classes: ['not spam' 'spam']
['spam' 'not spam' 'not spam']

The classes are sorted alphabetically, and "spam", which sorts last, is the positive class by scikit-learn's rule.

How often is it right?

A classifier's score is the share of its guesses that are right, called accuracy:

print("Share of test emails guessed right:", model.score(X_test, y_test))
print("Share of test emails that aren't spam:", (y_test == 0).mean())
Share of test emails guessed right: 0.765
Share of test emails that aren't spam: 0.695

76.5% sounds good until you see the second line. A filter that never calls anything spam is right about every email that isn't spam: 69.5% of these test emails. That's the "most common answer" baseline from The Simplest Guess: Baselines, and when one class is much more common than the other, it scores well while catching nothing. The model beats it, but by less than 76.5% suggests. Accuracy and When the Answer Is Rare show how to measure a classifier when the classes are uneven; for now, always put a classifier's score next to that baseline.

Check yourself

2 questions to try on the lesson page.

So what

When the answer is a category, the job is classification: name the classes and the one you're looking for, and remember that a score means little until you compare it with always guessing the most common class.

Sources

  • scikit-learn, Glossary of Common Terms and API Elements: a classifier is a predictor with a finite set of discrete possible output values and stores a classes_ attribute after fitting; binary is a classification problem of two classes and multiclass of more than two; in binary problems the class label with the greater value is considered the positive class unless told otherwise (of [0, 1], 1; of ['no', 'yes'], 'yes'); classes_ is sorted, numerically or lexicographically for strings.
  • scikit-learn 1.9.1, LogisticRegression, API reference: classes_, a list of class labels known to the classifier; predict gives a class label for each sample; score returns the accuracy on the given test data and labels.

Key terms

Related