Features and Labels
Before a model can learn, its table is split in two, X (capital X, the features, the columns it looks at) and y (the label, the one column it learns to guess), with ID columns left out, and with rows that are enough of the right kind.
Apprentice · AI Basics · Python · Updated
After this lesson you can
- Split a table into X, the feature columns, and y, the label column, leaving out the ID column
- Judge whether a column makes a useful feature, known at the time of the guess, possibly related to the label, and not an ID
Key takeaways
- Each row is one example. The features are the columns the model looks at, and the label is the one column it learns to guess.
- X, read "capital X", is the table of features, and y, read "y", is the label column. scikit-learn calls the label the target.
- A useful feature is known when you'd make the guess, may be related to the label, and isn't an ID that only names the row.
- A model needs enough rows, and rows like the cases it will guess for. A handful of rows gives averages that jump around.
A new apartment is about to go up for rent in the town of Riverside, and its owner wants to know what a fair monthly rent would be. A model can learn that from past listings, but only once you've told it which column is the answer and which columns it may look at. You'll make that choice at the start of every model you build, in this course and after it, and a wrong choice here spoils everything that follows.
The example for the whole chapter is Riverside Apartments: 300 listings in listings.csv, our own made-up data. It comes with Exercise: Know Your Listings, beside the seed script that made it, so you can run this lesson's code in that folder.
Which column is the answer?
Here are the first three listings, as they are in the file:
listing_id,size_sqft,bedrooms,bathrooms,minutes_to_downtown,has_parking,floor,monthly_rent
RA-1001,1100,2,2,8,1,7,1870
RA-1002,560,1,1,16,1,2,960
RA-1003,1000,2,2,27,1,4,1605
Each row is one apartment: one example. The question is "what rent?", so monthly_rent is the label, the column a model learns to guess. The columns it's allowed to look at to make that guess are its features: the size, the bedrooms, the minutes to downtown and the rest. (has_parking is 1 for an apartment with a parking space and 0 without.)
Nothing in the file says which column is the label. That's your decision, from the question you're asking. Ask "how big is an apartment that rents for $1,600?" and size_sqft becomes the label instead.
X and y
scikit-learn, the library this course uses for its models from Module 2 on, wants the two parts separately. By habit they're named X, read "capital X, the features", and y, read "y, the label". The capital letter is a math habit: a capital for a whole table, lowercase for a single column. scikit-learn's documentation calls the label the target; it's the same thing.
This notebook cell loads the file with pandas and makes both:
import pandas as pd
listings = pd.read_csv("listings.csv")
X = listings.drop(columns=["listing_id", "monthly_rent"])
y = listings["monthly_rent"]
print("X:", X.shape)
print("y:", y.shape)
print(list(X.columns))
X: (300, 6)
y: (300,)
['size_sqft', 'bedrooms', 'bathrooms', 'minutes_to_downtown', 'has_parking', 'floor']
X is 300 rows by 6 columns; y is one column of 300 rents, a row for each row of X. drop made a new table without the two columns it was given, and left listings as it was.
What makes a useful feature
The label went out of X for a clear reason: a model that can see the answer learns nothing. listing_id went too. Three questions decide whether a column belongs in X:
- Will you know it when you make the guess? The owner knows the size and the floor before listing the apartment. A column such as "days until it was rented" only exists afterward, so a model can't have it when it guesses. The same goes for anything that is the label in disguise, such as a yearly rent that's twelve times the monthly one.
- Could it be related to the label? Size probably is. The floor might be, or might not. You don't have to guess: How Two Columns Move Together measures it.
- Is it more than a name tag?
RA-1001says which apartment a row is, nothing about the apartment. A model given an ID can only memorize rows it has seen, which helps no apartment it hasn't.
Enough rows, and the right rows
The rows matter as much as the columns. They should look like the apartments you'll guess for: Riverside's listings, from about now. Rents from another city, or from twenty years ago, teach a different pattern.
And there should be enough of them. Here is the average rent of three groups of five listings, then of all 300:
rents = listings["monthly_rent"]
for start in [0, 5, 10]:
print("Listings", start + 1, "to", start + 5, "average:", rents.iloc[start:start + 5].mean())
print("All 300 average:", round(rents.mean(), 2))
Listings 1 to 5 average: 1424.0
Listings 6 to 10 average: 1163.0
Listings 11 to 15 average: 1210.0
All 300 average: 1373.97
.iloc[start:start + 5] takes five rows by position. Five listings give an average anywhere from $1,163 to $1,424, depending on which five; 300 settle it. No row count is right for every problem, but a model learning from a handful of examples learns their accidents. When Machine Learning Is the Wrong Tool has more on too few examples.
Do it: split listings into features and labels
Riverside's listings arrive as a list of dictionaries, one per apartment, and a model needs them as features and labels.
The inputs:
listings: a list of dictionaries, one per apartment. Every dictionary has a"listing_id"key and the same other keys.label: the name of the key to guess, such as"monthly_rent".
For example:
listings = [
{"listing_id": "RA-1001", "size_sqft": 1100, "floor": 7, "monthly_rent": 1870},
{"listing_id": "RA-1002", "size_sqft": 560, "floor": 2, "monthly_rent": 960},
]
features_and_labels(listings, "monthly_rent") returns this pair:
([{"size_sqft": 1100, "floor": 7}, {"size_sqft": 560, "floor": 2}], [1870, 960])
Your job: write features_and_labels(listings, label).
- It returns a pair:
features, thenlabels. featuresis a list with one new dictionary per listing, in the same order, holding every key except"listing_id"and the label.labelsis a list of each listing's label value, in the same order.- It doesn't change the dictionaries it was given.
- An empty
listingsreturns([], []).
Practiced live on the lesson page: write the Python right there and check it.
Check yourself
2 questions to try on the lesson page.
So what
A model learns only from the columns you hand it as X, toward the one you hand it as y, so choosing them, and the rows, is the first decision in every model, and the one that decides what it can learn.
Sources
- scikit-learn, Glossary of Common Terms and API Elements: X, data observed at training and prediction time, uppercase because it's ordinarily a matrix; y, the target of prediction, observed at training time but unavailable at prediction time; feature, a column of the data matrix; sample, one row; target, the dependent variable passed as
ytofit, also known as the label. - pandas 3.1.1, DataFrame.drop, API reference: removes the named columns given with
columns=, and returns a copy, a new DataFrame, by default (inplace=False).