FreeCampus Python

Write Tests That Protect Behavior

Turn public behavior into independent pytest assertions, expose false confidence and brittle checks, and build a focused protection map for a weather-risk rule.
python-foundations testing-python-programs assertions contracts boundaries
Open in Colab
  • Level: Python Foundations
  • Estimated time: 3–4 hours
  • You will learn: Translate a behavioral contract into independent assertions, recognize tests that pass without protecting the right behavior, and keep checks stable across safe refactoring.
  • Practice in: A local pytest project, with optional notebook experiments for direct function calls

Open this lesson in Google Colab

Meteor Watch receives a wind speed and returns one of three labels. The first version appears reasonable:

def risk_level(wind_kph):
    if wind_kph < 40:
        return "low"
    if wind_kph < 70:
        return "medium"
    return "high"

Running the function with 25 displays "low". That is an example, but it is not yet a useful protection net. Which threshold should be included in medium? What should happen for a negative speed? Would changing 70 to 71 be caught? A test becomes meaningful when its expected result comes from an explicit contract and the failure would identify a behavior worth preserving.

As you work, answer these questions with code and failure evidence:

1. Start with a contract, not the current implementation

Use this contract throughout the lesson:

Input Required behavior
a real number below 0 raise ValueError with wind must be non-negative
0 <= wind < 40 return "low"
40 <= wind < 70 return "medium"
wind >= 70 return "high"

The table supplies expected values independently of the function body. A test for 70 should expect "high" because the published interval begins at 70, not because the current code uses a particular comparison.

A minimal pytest test is an ordinary function whose name begins with test_:

from meteor_watch.risk import risk_level


def test_seventy_kph_begins_high_risk():
    observed = risk_level(70)
    assert observed == "high"

The lines have three jobs:

  1. Arrange: choose 70 because the contract makes it a boundary.
  2. Act: call risk_level(70) and retain the observed result.
  3. Assert: compare the observation with the contract’s "high".

The test name records the scenario and promised result. If it fails in a list of hundreds, test_seventy_kph_begins_high_risk is more useful than test_risk_2.

Arrange–Act–Assert describes a reading pattern. It does not require three comments in every five-line test. Blank lines or well-chosen names can make the stages clear:

def test_calm_wind_is_low_risk():
    observed = risk_level(12)

    assert observed == "low"

Predict a deliberate boundary defect

Change the production comparison to if wind_kph <= 70:. Predict the value for 70, then run only the boundary test. It should fail with "medium" versus "high". Restore < 70 and run again. Keep both outputs: the red run shows that the test can notice the defect; the green run shows that this selected case now matches the contract.

The contract supplies the expected result. Copying a decision from the implementation back into the test would make both sides share the same mistake.

flowchart LR
  A["Published wind intervals"] --> B["Expected high at 70"]
  C["risk_level implementation"] --> D["Observed result at 70"]
  B --> E{"Compare"}
  D --> E
  E --> F["Pass or useful failure"]

2. State one observable promise per test

A focused test may contain several assertions when they describe one result. For example, a parsed observation promises a station, wind value, and note as one returned record:

def parse_observation(line):
    station, wind_text, note = line.split("|", maxsplit=2)
    return {
        "station": station.strip(),
        "wind_kph": float(wind_text),
        "note": note.strip(),
    }


def test_parse_observation_returns_the_three_public_fields():
    observed = parse_observation(" ridge-7 | 42.5 | crosswind ")

    assert observed["station"] == "ridge-7"
    assert observed["wind_kph"] == 42.5
    assert observed["note"] == "crosswind"

All three assertions explain one public record. By contrast, placing parsing, risk classification, file writing, and notification in one test gives a failure too many possible causes. Test names and boundaries should make the broken promise easy to locate.

Compare direct and vague assertions

This assertion technically passes for any valid risk label:

def test_risk_returns_text():
    assert isinstance(risk_level(70), str)

It does not protect the important result. An implementation that always returns "low" still passes. The direct equality assertion is stronger:

def test_seventy_kph_begins_high_risk():
    assert risk_level(70) == "high"

Prefer the clearest assertion that expresses the contract. Pytest can explain plain assert expressions, so a custom if followed by raise AssertionError usually hides useful details.

Check exceptions as behavior

Invalid input is not protected by a test that merely calls the function and lets any exception end the test. State the promised type and stable part of the message:

import pytest


def test_negative_wind_is_rejected():
    with pytest.raises(ValueError, match="wind must be non-negative"):
        risk_level(-0.1)

pytest.raises fails if no exception occurs or if the type differs. The match expression checks user-relevant context without coupling to an entire traceback. Do not catch Exception in the production function merely to make this test pass; the contract names ValueError because the value is outside the accepted domain.

Compare measurements approximately

Binary floating-point operations can produce a nearby value instead of an exact decimal. If Meteor Watch converts 10 metres per second to 36 kilometres per hour, the contract is numeric closeness rather than a formatted string:

import pytest


def metres_per_second_to_kph(speed):
    return speed * 3.6


def test_speed_conversion_is_numerically_close():
    assert metres_per_second_to_kph(10 / 3) == pytest.approx(12.0)

Use exact equality when exactness is the behavior: category labels, integer counts, dictionary keys, and normalized text should not be made fuzzy.

Checkpoint: assertions that describe behavior

3. Separate examples, checks, and test suites

An interactive example answers, “What happened in this run?”

print(risk_level(25))

An assertion inside a notebook stops when a claim is false:

assert risk_level(25) == "low"

A pytest test adds discovery, a stable name, independent execution, rich failure reporting, selection, and inclusion in a repeatable suite:

def test_twenty_five_kph_is_low_risk():
    assert risk_level(25) == "low"

All three can help, but they provide different evidence. A copied screenshot of one green run does not guarantee that another machine can discover the same test. A suite becomes dependable when the project declares its dependencies, tests run from a clean state, and each test controls the resources it needs.

4. Recognize false confidence

A false positive in everyday testing language is a test that passes while the behavior you intended to protect is wrong. Consider four examples.

A test with no assertion

def test_risk_example_runs():
    risk_level(70)

This detects an unexpected exception for 70, but it cannot distinguish "high" from "medium". That may be a valid smoke check only if “does not raise” is the written promise. Its current name does not say so.

An assertion on the arranged value

def test_high_wind():
    wind = 70
    observed = risk_level(wind)

    assert wind == 70

The test never checks observed. It proves that the test assigned 70 to wind, not that the system classified it.

Expected logic copied from production

def test_risk_matches_expected_formula():
    wind = 70
    if wind < 40:
        expected = "low"
    elif wind < 70:
        expected = "medium"
    else:
        expected = "high"

    assert risk_level(wind) == expected

This case currently works, but duplicating every branch makes the test capable of sharing the same typo as the implementation. Prefer expected values from a small contract table. A separate simpler model can be a valid oracle later, but it must actually be independent and easier to trust.

An assertion too broad to notice the defect

def test_risk_is_a_known_label():
    assert risk_level(70) in {"low", "medium", "high"}

This protects a useful invariant—no unknown label—but not the boundary. Keep it only if that invariant has value alongside exact examples, not instead of them.

WarningMake the test red before trusting a new check

Temporarily introduce the defect the test is meant to catch, or write the test before the missing behavior. If the test stays green, inspect whether it calls the right code and asserts the right observation. Restore production code and rerun the complete suite afterward.

5. Avoid tests that reject safe refactoring

A false negative is a test that fails even though the public behavior is still acceptable. These tests slow improvement because ordinary refactoring looks like a defect.

Suppose the first implementation has a private helper:

def _upper_boundary_for(label):
    boundaries = {"low": 40, "medium": 70}
    return boundaries[label]


def risk_level(wind_kph):
    if wind_kph < _upper_boundary_for("low"):
        return "low"
    if wind_kph < _upper_boundary_for("medium"):
        return "medium"
    return "high"

This test couples to that private shape:

def test_medium_private_boundary():
    assert _upper_boundary_for("medium") == 70

Refactoring to a tuple of ranges removes the helper and breaks the test even if every promised result remains correct. The public-boundary test survives:

def test_seventy_kph_begins_high_risk():
    assert risk_level(70) == "high"

Private helpers can be tested directly when they contain complicated, independently meaningful behavior. The warning is not “never test a helper.” It is “do not make incidental structure part of the contract by accident.”

The same caution applies to call counts. If the contract says “send one public alert,” an exactly-once interaction may matter. If the function is free to cache, batch, or reorganize an internal helper, an exact call count is trivia.

Checkpoint: identify misleading green and red results

6. Choose examples by input class and boundary

You cannot test every real number. Instead, partition the contract into groups that should behave alike:

Class Representative values Why they matter
invalid negative -0.1 closest invalid value below zero
low 0, 12, 39.9 lower boundary, ordinary case, upper edge
medium 40, 55, 69.9 lower boundary, ordinary case, upper edge
high 70, 120 lower boundary and ordinary high value

Start with one ordinary case in each valid class, then test both sides of every boundary. If input arrives as text, that parser adds separate empty, malformed, and non-finite cases. Do not silently mix those parser requirements into the numeric classifier’s tests.

A compact suite can still be explicit:

import pytest

from meteor_watch.risk import risk_level


def test_zero_kph_is_low_risk():
    assert risk_level(0) == "low"


def test_just_below_forty_is_low_risk():
    assert risk_level(39.9) == "low"


def test_forty_begins_medium_risk():
    assert risk_level(40) == "medium"


def test_just_below_seventy_is_medium_risk():
    assert risk_level(69.9) == "medium"


def test_seventy_begins_high_risk():
    assert risk_level(70) == "high"


def test_negative_wind_is_rejected():
    with pytest.raises(ValueError, match="wind must be non-negative"):
        risk_level(-0.1)

Lesson 3 will turn related cases into a parametrized table. First learn to justify each row. Parametrization makes a design concise; it does not decide which cases belong in the design.

7. Keep tests independent and deterministic

Each test should be able to run alone and in any order. This pair is not independent:

observed_labels = []


def test_record_low_label():
    observed_labels.append(risk_level(10))
    assert observed_labels == ["low"]


def test_record_high_label():
    observed_labels.append(risk_level(90))
    assert observed_labels == ["low", "high"]

The second test passes only if the first test has already changed global state. Run it alone and it fails. Reverse the order and both may fail. Use fresh local state when shared history is not the behavior:

def test_records_low_label():
    observed_labels = []
    observed_labels.append(risk_level(10))
    assert observed_labels == ["low"]


def test_records_high_label():
    observed_labels = []
    observed_labels.append(risk_level(90))
    assert observed_labels == ["high"]

Current time, unseeded randomness, current directory, environment variables, network services, and leftover files create similar hidden dependencies. Later lessons provide fixtures and seams for them. For now, ask of every test: “What must already be true outside this function?” If the answer is not visible in the arrangement, isolation is at risk.

8. Explain the limit of every green result

After the six wind tests pass, accurate statements include:

  • the selected boundary and ordinary values matched the stated intervals;
  • the selected negative value raised the stated exception in this environment;
  • the test suite did not observe shared state or real external services; and
  • the deliberate <= 70 defect was noticed by the 70-kph test.

Inaccurate statements include:

  • risk_level has no bugs;
  • every possible float behaves correctly;
  • parsing, report writing, the CLI, and installation work;
  • the suite will never be flaky; or
  • high coverage has proven the algorithm.

Precise limits do not make tests weak. They make evidence honest and show what to investigate next.

Checkpoint: select a protection map

9. Build a protection map for Meteor Watch

Create src/meteor_watch/risk.py with this deliberately incomplete function:

def risk_level(wind_kph):
    """Return low, medium, or high for a non-negative wind speed."""
    if wind_kph < 40:
        return "low"
    if wind_kph < 70:
        return "medium"
    return "high"

Create tests/test_risk.py. Your finished lab should:

  1. write one ordinary case for each valid interval;
  2. protect 0, 40, and 70 plus the values just below the latter two boundaries;
  3. require ValueError("wind must be non-negative") for -0.1;
  4. temporarily introduce one comparison defect and capture the focused red report;
  5. repair the implementation rather than changing a correct expected value;
  6. run the focused node and then the complete file from a clean process; and
  7. write two sentences: what the suite supports and one risk it does not cover.

The negative rule requires this production guard:

def risk_level(wind_kph):
    """Return low, medium, or high for a non-negative wind speed."""
    if wind_kph < 0:
        raise ValueError("wind must be non-negative")
    if wind_kph < 40:
        return "low"
    if wind_kph < 70:
        return "medium"
    return "high"

Run:

python -m pytest -q tests/test_risk.py

Your observable result should name every case and finish green. Do not add parser, file, or CLI checks; they are different contracts.

Hint 1: turn the interval table into names Start with test_forty_begins_medium_risk and test_just_below_forty_is_low_risk. Repeat the pattern around 70. The name should tell you which expected value belongs in the assertion.
Hint 2: prove the negative path fails for the right reason Use with pytest.raises(ValueError, match="non-negative"): around one call. Run that node before adding the production guard; it should fail because no exception was raised.
Hint 3: find accidental coupling Search the tests for globals, current time, random values, files, or one test calling another. This lab needs none of those dependencies; every case can call the public function with one explicit number.
Show one complete protection map 
import pytest

from meteor_watch.risk import risk_level


def test_zero_kph_is_low_risk():
    assert risk_level(0) == "low"


def test_ordinary_low_wind_is_low_risk():
    assert risk_level(18) == "low"


def test_just_below_forty_is_low_risk():
    assert risk_level(39.9) == "low"


def test_forty_begins_medium_risk():
    assert risk_level(40) == "medium"


def test_ordinary_medium_wind_is_medium_risk():
    assert risk_level(55) == "medium"


def test_just_below_seventy_is_medium_risk():
    assert risk_level(69.9) == "medium"


def test_seventy_begins_high_risk():
    assert risk_level(70) == "high"


def test_ordinary_high_wind_is_high_risk():
    assert risk_level(110) == "high"


def test_negative_wind_is_rejected():
    with pytest.raises(ValueError, match="wind must be non-negative"):
        risk_level(-0.1)
One honest conclusion is: “These selected examples protect both sides of the documented numeric thresholds and one negative value. They do not test parsing, non-finite numbers, files, notifications, or the process boundary.”

Key points

  • Derive expected values from a behavioral contract, not from the current implementation.
  • A focused test arranges visible state, performs one meaningful action, and asserts an observable promise.
  • A green test supports only the case and environment it actually exercised.
  • Broad, vacuous, wrong-value, and copied-logic assertions can pass without protecting the intended behavior.
  • Tests of incidental helpers or call sequences can reject safe refactoring.
  • Choose ordinary, boundary, invalid, and regression examples deliberately.
  • Run a new test red for the expected reason before trusting its green result.
  • Keep tests independent, deterministic, and able to run alone.

Continue learning

Back to top