What the Data Can and Can't Show

Two things moving together doesn't show one causes the other, a rate from a few sessions swings far more than one from many, and the period covered and the rows left out limit every finding, so a report ends with a short What this data can't tell us section that names them.

Journeyman · Python · Updated

By

After this lesson you can

  • Explain why two things moving together doesn't show one causes the other, and why a rate from a few sessions is less trustworthy
  • Write a What this data can't tell us section that names the period covered, the small groups and the rows left out

Key takeaways

  • Two things that move together, such as free classes and no-shows, don't show that one causes the other. Something else may explain both.
  • A rate from a few sessions can swing with one busy or quiet session, so say how many sessions each rate comes from.
  • Say which period the data covers, so nobody reads one year as a pattern for every year.
  • Say which rows the cleaning or the calculation left out, and why.
  • A What this data can't tell us section lists these limits in plain words, so a reader knows how far to trust each finding.

People act on reports. Denise, who runs Pine Valley Community Center, will set next year's schedule and prices from what yours says, so a finding that reaches further than the data does can cost the center money or a class people loved. Before a number becomes a decision, the report says how far it can be trusted. You'll write that part for every analysis you hand someone, from a class project to a report at work.

The example is the center's classes.csv: 588 class sessions, with each session's class_name, category, month, capacity, signed_up, attended and fee.

Moving together isn't causing

Here is a finding that looks like a decision waiting to happen:

import pandas as pd

classes = pd.read_csv("classes.csv")
recorded = classes.dropna(subset=["attended"]).copy()
recorded["free"] = recorded["fee"] == 0
by_fee = recorded.groupby("free")[["signed_up", "attended"]].sum()
by_fee["no_show_rate"] = ((by_fee["signed_up"] - by_fee["attended"]) / by_fee["signed_up"] * 100).round(1)
by_fee
       signed_up  attended  no_show_rate
free
False       5490    5045.0           8.1
True        1655    1273.0          23.1

At free sessions, 23.1% of the people who signed up didn't come; at paid ones, 8.1%. So charging a fee makes people show up? Not so fast. Ice cream sales and sunburns rise together every summer, and ice cream doesn't cause sunburns: hot, sunny days cause both. Two things that move together have a correlation. One making the other happen is causation, and the data can show the first without the second. Statistics classes put it as "correlation isn't causation".

The free classes here are Senior Strength, Tai Chi, Knitting Circle, Junior Chess, Smartphone Basics and Computer Basics. Maybe people sign up "just in case" when it costs nothing, or these groups' plans change more often. The data can't tell those apart from the fee. Finding out would take a test: the same class, free some months and paid in others, compared.

Small groups swing

The categories' fill rates, with how many sessions each comes from:

seats = classes.groupby("category").agg(
    sessions=("class_id", "count"),
    signed_up=("signed_up", "sum"),
    capacity=("capacity", "sum"),
)
seats["fill_rate"] = (seats["signed_up"] / seats["capacity"] * 100).round(1)
seats.sort_values("fill_rate", ascending=False)
            sessions  signed_up  capacity  fill_rate
category
technology        18        168       180       93.3
kids             138       1806      2088       86.5
fitness          276       4051      5400       75.0
arts             156       1276      1824       70.0

Technology looks fullest, from 18 sessions. Imagine one quiet session, with six fewer sign-ups: technology drops to 162 of 180 seats, 90.0%. The same six fewer in kids classes moves them to 1800 of 2088, 86.2%. How many observations a number rests on is its sample size, and a small one lets a few rows move the answer a long way. Show the count beside every rate, and warn about any group below a number you choose and state, such as 30 sessions.

The period covered and the rows left out

Two more limits come straight from the data:

print("first month:", classes["month"].min())
print("last month:", classes["month"].max())
print("months:", classes["month"].nunique())
print("sessions:", len(classes))
print("attendance not recorded:", classes["attended"].isna().sum())
first month: 2025-11
last month: 2026-10
months: 12
sessions: 588
attendance not recorded: 12

One year of data has one January. If it was the busiest month, that's a fact about that January, not a promise about the next one. And 12 sessions have no attendance recorded, so the no-show rates come from the other 576. A reader deserves to know about any rows your cleaning or your calculation left out.

Writing "What this data can't tell us"

The limits go in their own section at the end of the story, as a short list in plain words, each with the number that makes it concrete:

## What this data can't tell us

- **One year only.** The data covers November 2025 to October 2026, so January may not be the busiest month every year.
- **Small groups.** Technology's 93.3% comes from 18 sessions; one quiet session would move it by about 3 points.
- **Why people don't come.** Free classes had more no-shows (23.1% against 8.1%), but the data can't say the fee is the reason.
- **Rows left out.** 12 sessions had no attendance recorded, so the no-show rates come from the other 576.

Write it plainly, without apologizing: every dataset has limits, and naming them is what makes the rest of the report believable.

Check yourself

2 questions to try on the lesson page.

So what

A finding is only as useful as its limits are clear: say what moved together without claiming why, how many rows each number rests on, which period it covers, and what was left out.

Key terms

Related