Testing Edge Cases and Errors
Bugs hide at the edges of what a function accepts, so good tests try zero and empty values, the smallest and biggest values allowed and each side of a boundary, make sure bad values are refused with pytest.raises, and run many examples through one test with parametrize.
Journeyman · Python · Updated
After this lesson you can
- Choose edge cases to test, such as zero, an empty list, the biggest value allowed and both sides of a boundary
- Test that code raises an error with pytest.raises, and run one test on many examples with pytest.mark.parametrize
Key takeaways
- An edge case is a value at the limit of what code handles, such as zero, an empty list, the biggest value allowed, or exactly the amount where a rule changes.
- Test both sides of a boundary, the last value allowed and the first one that isn't, because that's where a > written in place of >= hides.
- with pytest.raises(ValueError) makes sure the code inside it raises that error, and the test fails when it doesn't.
- The parametrize mark runs one test once for each example in its list, and pytest reports every example on its own.
A function that works for the example you thought of can still break for the one a customer types: a cart with nothing in it, a parcel right at the weight limit, an order of exactly $50. This lesson shows which examples to test, so those bugs turn up in your test run instead of in a customer's email.
The example is Crafty Corner's postage: its carrier charges $4.50 for the first pound and $1.25 for each pound after it, for parcels of 1 to 70 whole pounds.
One example isn't enough
Here is the postage rule in shipping.py, with a bug in it:
def parcel_cost(pounds):
"""Postage: $4.50 for the first pound and $1.25 for each pound after it.
The carrier takes parcels of 1 to 70 whole pounds."""
if pounds < 1 or pounds >= 70:
raise ValueError("A parcel must weigh 1 to 70 pounds.")
return 4.50 + (pounds - 1) * 1.25
And here is its test, in test_shipping.py, with a typical parcel: 3 pounds cost $4.50 plus 2 × $1.25, which is $7.00.
from shipping import parcel_cost
def test_3_pounds_costs_7():
assert parcel_cost(3) == 7.00
python -m pytest ends with 1 passed. The test is right and it passes, and yet a 70-pound parcel, which the carrier takes, is refused: >= (read "greater than or equal to") turns 70 away along with 71. No test asked about 70, so nothing noticed.
Where bugs hide: the edges
At an amusement park, the sign by the ride says You must be this tall. Nobody argues about the tall kids or the tiny ones; the argument is always about the kid whose head is right at the line. Code is the same. An edge case is a value at the limit of what the code handles, and a boundary is the place where a rule changes. Test these:
- Zero or empty: 0 pounds, a cart with no items (
[]), a blank name (""). - The smallest and the biggest allowed: 1 pound and 70 pounds.
- Both sides of each boundary: the last value allowed and the first one that isn't: 70 and 71. For free shipping from $50, that's $49.99 and $50.00.

Add the smallest and biggest parcels to test_shipping.py, each worked out by hand: 1 pound is $4.50, and 70 pounds is $4.50 plus 69 × $1.25, which is $90.75.
def test_1_pound_costs_4_50():
assert parcel_cost(1) == 4.50
def test_70_pounds_costs_90_75():
assert parcel_cost(70) == 90.75
Now the run catches the bug. This is the end of what pytest prints:
> raise ValueError("A parcel must weigh 1 to 70 pounds.")
E ValueError: A parcel must weigh 1 to 70 pounds.
shipping.py:5: ValueError
=========================== short test summary info ============================
FAILED test_shipping.py::test_70_pounds_costs_90_75 - ValueError: A parcel mu...
========================= 1 failed, 2 passed in 2.03s ==========================
The fix is pounds > 70 in place of pounds >= 70, and then all three tests pass.
Making sure an error happens: pytest.raises
Refusing 0 pounds and 71 pounds is part of the rule too, so it needs tests. assert can't check it: the call raises a ValueError and never hands back a value to compare. pytest's raises can. Put the call inside with pytest.raises(ValueError): (read "with pytest raises ValueError"), and the test passes only if that code raises a ValueError. Add match= to make sure the error's message contains certain words, too:
import pytest
from shipping import parcel_cost
def test_71_pounds_is_refused():
with pytest.raises(ValueError):
parcel_cost(71)
def test_0_pounds_is_refused_with_a_message():
with pytest.raises(ValueError, match="1 to 70 pounds"):
parcel_cost(0)
Both pass. Had someone deleted the 70-pound limit, the first would fail like this:
> with pytest.raises(ValueError):
^^^^^^^^^^^^^^^^^^^^^^^^^
E Failed: DID NOT RAISE ValueError
Raising Your Own Exceptions shows how a function raises an error like this one.
One test, many examples: parametrize
The tests for 1, 3 and 70 pounds are the same test with different numbers. @pytest.mark.parametrize (read "at pytest mark parametrize"; a line starting with @ above a function, called a decorator, adds something to it) runs one test once for each example in a list. Its first argument names the test's parameters, and each pair in the list fills them in:
import pytest
from shipping import parcel_cost
@pytest.mark.parametrize(
"pounds, expected",
[
(1, 4.50),
(3, 7.00),
(70, 90.75),
],
)
def test_parcel_cost(pounds, expected):
assert parcel_cost(pounds) == expected
@pytest.mark.parametrize("pounds", [0, 71])
def test_out_of_range_is_refused(pounds):
with pytest.raises(ValueError):
parcel_cost(pounds)
Run it with python -m pytest -v (for verbose), and pytest lists each example as a test of its own, its values in square brackets:
test_shipping.py::test_parcel_cost[1-4.5] PASSED [ 20%]
test_shipping.py::test_parcel_cost[3-7.0] PASSED [ 40%]
test_shipping.py::test_parcel_cost[70-90.75] PASSED [ 60%]
test_shipping.py::test_out_of_range_is_refused[0] PASSED [ 80%]
test_shipping.py::test_out_of_range_is_refused[71] PASSED [100%]
============================== 5 passed in 0.13s ===============================
A new edge case is now one more line in the list.
Do it: test the bead sale at its edges
Crafty Corner's bead sale takes 10% off a cart that holds 10 or more items. The function sale_total already works. Write the edge-case tests that would catch someone breaking it.
sale_total(prices): in the code box, ready to use.pricesis a list with each item's price; it returns the cart's total after the sale, rounded to cents. Don't change it.empty_cart,nine_itemsandten_items: three carts, ready to use: no items, nine $2.00 items, and ten $2.50 items.
Your job: below the carts, write three test functions, each with one assert:
test_empty_cart_is_0: makes sure thatsale_total(empty_cart)is equal to0.test_9_items_pay_full_price: makes sure thatsale_total(nine_items)is equal to18.00, the full 9 × $2.00.test_10_items_get_10_percent_off: makes sure thatsale_total(ten_items)is equal to22.50.
Worked through: ten $2.50 items make $25.00. That's 10 items, so the sale applies: 10% of $25.00 is $2.50, and $25.00 − $2.50 = $22.50. The test's line is assert sale_total(ten_items) == 22.50.
The box runs your three tests on the working sale_total, where all three must pass, and then on three broken copies, where at least one must fail on each: a sale that starts only above 10 items, a sale that starts at 9 items, and a sale_total that crashes on an empty cart.
Practiced live on the lesson page: write the Python right there and check it.
Check yourself
2 questions to try on the lesson page.
So what
The bugs that reach customers are rarely in the middle of a rule; they're at its edges, so a few tests at zero, the limits and each side of a boundary catch more than a dozen tests of everyday values.