Exercise: Café Regulars

Write four functions that run one round of k-means by hand in plain Python, group the Cozy Corner Café's 600 loyalty members with a StandardScaler and KMeans pipeline, and describe each group with groupby, and let pytest check the hand round, the pipeline, the labels and the summaries.

Journeyman · AI Basics · Python · Updated

By

The Cozy Corner Café wants to know what kinds of regulars it has, so it can plan a deal for each, and its 600 loyalty members have no column that says. You'll write the four functions behind the answer: two that run one round of k-means by hand, one that groups the members with scikit-learn, and one that turns the group numbers into a description the owner can read. Tests call each function and check what it gives back, and a notebook comes with the exercise for looking at the data while you work.

Get the exercise code

Open this lesson's GitHub repository, github.com/ask-codey/exercise-ml-cafe-regulars, also linked from the Exercise box beside the lesson. Clone it into your python-course folder. In VS Code's terminal:

cd ~/python-course
git clone https://github.com/ask-codey/exercise-ml-cafe-regulars.git

Git makes a folder, exercise-ml-cafe-regulars, with the exercise code inside. Open that folder in VS Code (File > Open Folder), open its terminal with Terminal > New Terminal, and make the exercise's virtual environment. On Windows (PowerShell):

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt

On a Mac (zsh):

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The prompt starts with (.venv) once the virtual environment is on. The exercise's README says what each command does, and what to do if PowerShell won't run the second one.

The folder holds:

  • cafe_members.csv: the 600 members, our own made-up data, the same file the chapter's lessons use.
  • seed_data.py: the script that made the CSV, with the four groups it planted (they aren't in the file). You don't need to run it: the committed CSV is what keeps everyone's numbers the same.
  • cafe_groups.py: the four functions, for you to finish, with FEATURES (the columns k-means groups by: visits_per_month, avg_spend, share_before_9am and share_with_food), ready to use.
  • explore.ipynb: a notebook for looking at the members and trying your code. Choose the .venv kernel with Select Kernel (top right) before you run it. The tests don't read it.
  • report.py: runs a round by hand, groups the members and describes the groups with your four functions, and prints what each step gave. Ready to use.
  • tests/: the tests, which call each function and check what comes back, and conftest.py, which gives each test a fresh copy of the members. You don't need to change them.
  • requirements.txt and pytest.ini: the packages the exercise needs, at exact versions, and where pytest finds the tests and your code.

What to do

Every function in cafe_groups.py is already declared with its name and parameters, and for now it stops with a NotImplementedError that says Not built yet. The docstring under each one says what it should do, with examples:

def move_centers(points, assignments, k):
    """Return the k new centers: center number n is the mean of the points assigned to n.

    assignments[i] is the number of the center points[i] belongs to, from 0 to k - 1.
    Each new center is a tuple with one mean per feature, in the points' order of
    features, and the list holds center 0 first. Every number from 0 to k - 1 has at
    least one point.

    points [(18, 5), (20, 4), (4, 20), (2, 22)], assignments [0, 0, 1, 1] and k = 2
    -> [(19.0, 4.5), (3.0, 21.0)]
    """
    raise NotImplementedError("Not built yet.")

In each function, replace the raise NotImplementedError("Not built yet.") line with your own code:

  1. assign_to_nearest(points, centers), in plain Python, returns a list with the number of each point's nearest center (0 for the first center), by the straight-line distance. A point equally near two centers gets the one that comes first. With the six members of How k-means Works, [(18, 5), (16, 6), (20, 4), (4, 20), (2, 22), (6, 18)], and the centers [(18, 5), (16, 6)], it returns [0, 1, 0, 1, 1, 1].
  2. move_centers(points, assignments, k), in plain Python, returns the k new centers: center number n is a tuple of the means, feature by feature, of the points assigned to n. With those six members and [0, 1, 0, 1, 1, 1], it returns [(19.0, 4.5), (7.0, 16.5)]. Points can have any number of features.
  3. cluster_members(members, k, seed) makes a pipeline with make_pipeline(StandardScaler(), KMeans(n_clusters=k, n_init=10, random_state=seed)), fits it on the FEATURES columns only, and returns the fitted pipeline. Its KMeans step then holds one label per member in labels_: 600 labels, from 0 to 3, for cluster_members(members, 4, 0).
  4. describe_groups(members, labels) returns a table with one row per group, the group numbers as its index from smallest to largest, each feature's mean in the FEATURES columns, and a last column, members, with each group's size.

The first two are the round you did by hand in How k-means Works; the last two are k-means with scikit-learn and Describing Each Group.

Check your work

Run the tests from the exercise folder, with the virtual environment on:

python -m pytest

At first all 18 tests fail. That's expected: each function you finish turns its tests green. A failing test's message says what it called, what it expected and what came back, for example assign_to_nearest(SIX, [(18, 5), (16, 6)]) should return [0, 1, 0, 1, 1, 1], but it returned [0, 0, 0, 1, 1, 1]. No test checks which members k-means puts together in the real data, which depends on its random starts: the tests check the hand round exactly, the pipeline's steps and settings, one label per member, that the same seed gives the same groups, that two groups far apart are split the same way whatever the seed, and the summaries' arithmetic.

To see your functions at work on the whole file, run python report.py.

Hints

  • Look before you write. Run the notebook's first cells: members.head(10) and members.describe() show the four features and their very different ranges.
  • Nearest, by position. math.dist(point, center) gives a distance; make a list of them, one per center, and distances.index(min(distances)) gives the position of the first smallest, which keeps ties with the first center.
  • A mean per feature. Collect the points assigned to each center number in a list. zip(*mine) turns that list of points into one tuple per feature, and sum(values) / len(mine) is each feature's mean. tuple(...) around a comprehension in round brackets, such as tuple(x * 2 for x in (1, 2)), which gives (2, 4), makes the new center a tuple.
  • Two steps, one object. k-means with scikit-learn builds the same pipeline; cluster_members fits it on members[FEATURES] with fit, then hands it back.
  • groupby on the labels. members[FEATURES].groupby(labels).mean() gives the means, sorted by group number, and members.groupby(labels).size() gives the sizes, as Describing Each Group shows.

When you're done

Every test is green: pytest's last line says 18 passed. And python report.py prints:

1) six members, nearest of two starting centers: [0, 1, 0, 1, 1, 1]
2) the centers moved to: [(19.0, 4.5), (7.0, 16.5)]
3) members: 600
4) labels: 600
5) the four groups:
   visits_per_month  avg_spend  share_before_9am  share_with_food  members
0              2.62       9.85              0.47             0.40      146
1              4.16      21.19              0.10             0.84      122
2             17.46       5.50              0.85             0.15      180
3              9.89       7.18              0.10             0.35      152

Now compare the four groups with the ones seed_data.py planted: how close did k-means come to each group's size and numbers? In the notebook, try cluster_members(members, 3, 0) and cluster_members(members, 5, 0), describe each, and write one sentence for the café's owner about which you'd hand her, and why.

Submit your exercise

The exercise is how you finish this lesson. When your work is done, put it on GitHub in a repo of your own, then submit that repo's address here. Hand In Your Work with Git walks through each step.

  1. On GitHub, click +, then New repository. Name it (for example exercise-ml-cafe-regulars), choose Public, and click Create repository. Leave out the README: your folder already has one.

  2. In VS Code's terminal, in your exercise folder, send your work to your new repo. Point your copy at your own repo, then commit and push:

    git remote set-url origin https://github.com/your-name/exercise-ml-cafe-regulars.git
    git add .
    git commit -m "My solution"
    git push -u origin main
    
  3. Paste your repo's address, such as https://github.com/your-name/exercise-ml-cafe-regulars, into Submit your exercise on this page. The site checks that the repo is public and holds your commits, and then the lesson is complete.

Changed something after you submitted? Push it, and submit again: your latest submission is the one that counts.

So what

One round by hand, a scaled pipeline with a fixed seed, and a plain summary of each group: that's a clustering you can explain, repeat and test, function by function.

Key terms

Related