Exercise: Orchard Harvest Models

Write three functions that make curve features with PolynomialFeatures, build a regression tree with a depth limit and a seed, and compare models fairly on Hilltop Orchard's 200 harvests, each fit on the same training trees with its training and test errors side by side, and let pytest check them.

Journeyman · AI Basics · Python · Updated

By

Hilltop Orchard wants to guess how many pounds of apples each tree will give, from the season's rain and the tree's age, so it can plan pickers and crates. Its harvests rise with rain up to about 30 inches and then fall, so a straight line won't do. You'll write the three functions behind a fair comparison of models that can bend: one that makes curve features, one that builds a regression tree, and one that scores any set of models on the same training and test trees. Tests call each function and check what it gives back, and a notebook comes with the exercise for looking at the data while you work.

Get the exercise code

Open this lesson's GitHub repository, github.com/ask-codey/exercise-ml-orchard-harvest-models, also linked from the Exercise box beside the lesson. Clone it into your python-course folder. In VS Code's terminal:

cd ~/python-course
git clone https://github.com/ask-codey/exercise-ml-orchard-harvest-models.git

Git makes a folder, exercise-ml-orchard-harvest-models, with the exercise code inside. Open that folder in VS Code (File > Open Folder), open its terminal with Terminal > New Terminal, and make the exercise's virtual environment. On Windows (PowerShell):

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt

On a Mac (zsh):

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The prompt starts with (.venv) once the virtual environment is on. The exercise's README says what each command does, and what to do if PowerShell won't run the second one.

The folder holds:

  • orchard.csv: the 200 harvests, our own made-up data, the same file the chapter's lessons use.
  • seed_data.py: the script that made the CSV, with the pattern it planted. You don't need to run it: the committed CSV is what keeps everyone's numbers the same.
  • orchard_models.py: the three functions, for you to finish, with FEATURES (the columns the models look at: rainfall_inches and tree_age_years) and LABEL (apples_lbs), ready to use.
  • explore.ipynb: a notebook for looking at the harvests, with a scatter plot of rain against apples, and for trying your code. Choose the .venv kernel with Select Kernel (top right) before you run it. The tests don't read it.
  • report.py: splits the harvests, makes the curve features and the trees, and compares five models with your three functions. Ready to use.
  • tests/: the tests, which call each function and check what comes back, and conftest.py, which gives each test a fresh copy of the harvests and the same 160 / 40 split report.py uses. You don't need to change them.
  • requirements.txt and pytest.ini: the packages the exercise needs, at exact versions, and where pytest finds the tests and your code.

What to do

Every function in orchard_models.py is already declared with its name and parameters, and for now it stops with a NotImplementedError that says Not built yet. The docstring under each one says what it should do, with examples. In each function, replace the raise NotImplementedError("Not built yet.") line with your own code:

  1. curve_features(X_train, X_test, degree) makes a PolynomialFeatures(degree=degree, include_bias=False), fits it on the training rows with fit_transform(X_train), changes the test rows with transform(X_test), and returns both: curve_train, curve_test. With one feature and degree 2, the training values 2 and 3 become [[2, 4], [3, 9]] and the test value 10 becomes [[10, 100]]. With the two FEATURES and degree 2, a row of 2 and 5 becomes [2, 5, 4, 10, 25]: each feature, the first squared, the two multiplied, the second squared. On the orchard's 160 training trees and 40 test trees, degree 2 gives shapes (160, 5) and (40, 5), and degree 3 gives 9 columns.
  2. build_tree(max_depth, seed) returns a new DecisionTreeRegressor with max_depth=max_depth and random_state=seed, not fit yet. build_tree(3, 0) is at most 3 questions deep with random_state 0; build_tree(None, 7) has no depth limit and random_state 7.
  3. compare_models(models, X_train, y_train, X_test, y_test) takes a dictionary of names to models that haven't been fit, such as {"straight line": LinearRegression()}. For each one, in order, it fits the model on X_train and y_train, then works out its training MAE (mean_absolute_error of y_train and its guesses for X_train) and its test MAE (the same for y_test and X_test). It returns a dictionary of each name to {"train_mae": ..., "test_mae": ...}, not rounded.

Here is compare_models worked through by hand. The model is DummyRegressor(strategy="mean"), named "average". The training labels are 10, 20 and 30, so it always guesses their average, 20. Its training misses are 10, 0 and 10, so its training MAE is 20 / 3 = 6.666…. The test labels are 25 and 5, so its test misses are 5 and 15, and its test MAE is 20 / 2 = 10.0. It returns {"average": {"train_mae": 6.666666666666667, "test_mae": 10.0}}. That example is one of the tests.

Every model is fit on the training rows only, and PolynomialFeatures is fit only on the training rows too: that's what keeps the test trees ones the models never saw.

Check your work

Run the tests from the exercise folder, with the virtual environment on:

python -m pytest

At first all 16 tests fail. That's expected: each function you finish turns its tests green. A failing test's message says what it called, what it expected and what came back, for example build_tree(3, 0).max_depth should return 3, but it returned None. The tests check:

  • curve_features: the two worked examples above, the row 2 and 5, the shapes for degree 2 and 3 on the orchard, and that degree 1 gives back the two features with no column of 1s;
  • build_tree: that it's a DecisionTreeRegressor, its max_depth and random_state for (3, 0) and (None, 7), and that it hasn't been fit;
  • compare_models: the worked example, a straight line through exact points (both errors 0), the names in order with train_mae and test_mae, that the scores equal mean_absolute_error on the 160 and 40 trees, that the average model guesses the training trees' average, and that it works with the curve features.

No test checks how accurate a model is: that depends on each harvest's random wobble.

To see your functions at work on the whole file, run python report.py.

Hints

  • Look before you write. Run the notebook's first cells: the scatter plot shows the hump that a straight line can't follow.
  • Fit on one part, transform the other. Curves with Polynomial Features has the two calls, fit_transform and transform, in that order.
  • Settings go in the parentheses. A tree's max_depth and random_state are set when you make it, as Regression Trees shows; fit comes later, in compare_models.
  • One loop, one new dictionary. Start with an empty dictionary, loop over models.items(), and add one entry for each name. Comparing Regression Models has the fit-then-score steps.
  • Don't round. The tests compare your scores with mean_absolute_error's own, to many decimal places.

When you're done

Every test is green: pytest's last line says 16 passed. And python report.py prints:

1) training trees: 160
2) test trees: 40
3) curve features, training: (160, 5)
4) curve features, test: (40, 5)
5) tree settings: max_depth 3 random_state 0
6) average (baseline): training MAE 51.6, test MAE 54.3
7) straight line: training MAE 48.5, test MAE 54.5
8) tree, depth 3: training MAE 21.0, test MAE 26.4
9) tree, no limit: training MAE 0.0, test MAE 24.6
10) curve, degree 2: training MAE 16.6, test MAE 14.9

The curve gets its own compare_models call because it learns from its own columns; Pipelines: Every Step, Every Time, in Module 4, joins the two steps into one model. Now try a few changes in the notebook: a tree of depth 4 or 5, and a curve of degree 3. Which one would you keep, and what is each one's gap between training and test error?

Submit your exercise

The exercise is how you finish this lesson. When your work is done, put it on GitHub in a repo of your own, then submit that repo's address here. Hand In Your Work with Git walks through each step.

  1. On GitHub, click +, then New repository. Name it (for example exercise-ml-orchard-harvest-models), choose Public, and click Create repository. Leave out the README: your folder already has one.

  2. In VS Code's terminal, in your exercise folder, send your work to your new repo. Point your copy at your own repo, then commit and push:

    git remote set-url origin https://github.com/your-name/exercise-ml-orchard-harvest-models.git
    git add .
    git commit -m "My solution"
    git push -u origin main
    
  3. Paste your repo's address, such as https://github.com/your-name/exercise-ml-orchard-harvest-models, into Submit your exercise on this page. The site checks that the repo is public and holds your commits, and then the lesson is complete.

Changed something after you submitted? Push it, and submit again: your latest submission is the one that counts.

So what

Curve features fit on the training rows, trees with their settings written down, and every model scored the same way on the same rows: that's how you choose between models honestly, and each piece is small enough to test.

Key terms

Related