Your First scikit-learn Model: fit and predict
Every scikit-learn model follows the same three steps, make it, fit(X, y) to learn from examples, and predict(X) to guess for new ones, with the features X as a table even when there's only one, and a linear regression's learned slope and intercept kept in coef_ and intercept_.
Journeyman · AI Basics · Python · Updated
After this lesson you can
- Train a LinearRegression with fit(X, y), passing the features as a table and the label as one column
- Read a fitted linear regression's coef_ and intercept_, and guess the label for new rows with predict
Key takeaways
- Every scikit-learn model follows one pattern. Make it, fit it on the features and the label, then predict for new rows.
- The features X are a table (two-dimensional) even with one feature, written homes[["size_sqft"]] with two pairs of brackets. The label y is one column.
- fit stores what the model learned in names that end in an underscore. A linear regression keeps its slopes in coef_ and its starting point in intercept_.
- predict needs a table with the same column names the model learned from, and it gives back one guess for each row.
Fitting a Line found the best line through Maple Grove's home sales with a formula from Python's statistics module. That works for one feature and one kind of model. scikit-learn, the machine learning package this course uses, does the same job with three steps that every one of its models shares, from this straight line to the decision trees and nearest neighbors later in the course. Learn the three steps once, and every new model is a new name in the same code.
The example is Maple Grove Homes' 400 sales in home_sales.csv, which comes with Exercise: Home Price Guesser. Each code block below is one notebook cell, and the cells run in order: the first one loads the data, and the others use what it made.
Make, fit, predict
Think of a new real estate agent in town. First you hire them (they know nothing about Maple Grove yet). Then they study the past sales. Then you ask them about a home, and they guess a price. scikit-learn's three steps are the same:
- Make the model:
LinearRegression()is a linear regression that hasn't learned anything yet. - Fit it:
model.fit(X, y)learns from examples with their answers. - Predict:
model.predict(X)guesses the answer for new examples.
The rule a model learns is the model. scikit-learn's documentation calls anything with a fit method an estimator: you'll meet the word there.
X is a table, y is a column
scikit-learn's names for the two parts are X (read "capital X, the features") and y (read "y, the label"). The capital letter is a reminder that X is a table, with a row for each example and a column for each feature. y is one column: the answer for each row.
import pandas as pd
from sklearn.linear_model import LinearRegression
homes = pd.read_csv("home_sales.csv")
X = homes[["size_sqft"]]
y = homes["sale_price"]
print("X:", X.shape)
print("y:", y.shape)
X: (400, 1)
y: (400,)
X is 400 rows by 1 column: a table that happens to have one column. Two pairs of brackets, homes[["size_sqft"]], ask pandas for a table of the columns listed inside. One pair, homes["sale_price"], gives a single column, which is what the label is. Give fit the size with one pair, and it stops:
model = LinearRegression()
model.fit(homes["size_sqft"], y)
ValueError: Expected a 2-dimensional container but got <class 'pandas.Series'> instead. Pass a DataFrame containing a single row (i.e. single sample) or a single column (i.e. single feature) instead.
A Series is pandas' name for one column. scikit-learn wants the features as a table even with one feature, because most models look at many.
fit: learning from the examples
model = LinearRegression()
model.fit(X, y)
print("coef_:", model.coef_)
print("intercept_:", model.intercept_)
coef_: [136.66990174]
intercept_: 34026.55382217621
fit found the least squares line, the same one statistics.linear_regression found in Fitting a Line: $136.67 a square foot from a starting point of $34,026.55. The names end in an underscore, read "coef underscore" and "intercept underscore", and the underscore marks a value the model learned in fit. coef_ (short for coefficient) is a list of numbers, one slope for each feature, so with one feature it holds one number and model.coef_[0] is the size's slope. intercept_ is the starting point.
predict: guessing for homes it hasn't seen
New homes go in a table with the same column name the model learned from, and predict gives back one guess for each row:
new_homes = pd.DataFrame({"size_sqft": [1500, 2000, 2350]})
guesses = model.predict(new_homes)
for size, guess in zip(new_homes["size_sqft"], guesses):
print(size, "square feet:", round(guess))
1500 square feet: 239031
2000 square feet: 307366
2350 square feet: 355201
Each guess is the intercept plus the slope times the size: the family's 2,000 square feet gets $307,366 again, and the 2,350-square-foot home, which had no sale to look up, gets a guess too. predict gives back a NumPy array, a list of numbers that zip and round work with like a list.
The column's name matters. Ask about a column named size, and predict stops:
model.predict(pd.DataFrame({"size": [2000]}))
ValueError: The feature names should match those that were passed during fit.
Feature names unseen at fit time:
- size
Feature names seen at fit time, yet now missing:
- size_sqft
And a model that hasn't been fit has nothing to guess with:
untrained = LinearRegression()
untrained.predict(new_homes)
NotFittedError: This LinearRegression instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.
This lesson fit the model on all 400 homes to see the pattern. Before trusting how good its guesses are, some homes have to be kept away from fit: Splitting Data with train_test_split shows how.
Check yourself
2 questions to try on the lesson page.
So what
Make, fit, predict is the whole shape of a scikit-learn model: the features as a table, the label as a column, what was learned in names that end in an underscore, and one guess for every new row.
Sources
- scikit-learn 1.9.1, LinearRegression, API reference:
fit(X, y)withXof shape (n_samples, n_features) andyof shape (n_samples,),coef_(one coefficient per feature for one target),intercept_,feature_names_in_, andpredict(X). - scikit-learn, Linear Models: Ordinary Least Squares, User Guide:
LinearRegressionstores the coefficients of the least squares line incoef_andintercept_. - scikit-learn, Glossary of Common Terms and API Elements: estimator (an object with a
fitmethod), fit, predict, X and y, and attributes that end in an underscore when they are set infit. - scikit-learn 1.9.1, Getting Started: every estimator is fitted with
fit, and a fitted estimator predicts targets for new data withpredict.