Meteor Watch should include a timestamp in each alert and send a notification for red risk. A direct implementation can reach the system clock, environment, random generator, and real notification service. The behavior may work in production while remaining slow, destructive, or impossible to reproduce in a test.
The answer is not “mock everything.” First make important dependencies visible. Then choose the simplest controlled replacement that provides the evidence you need. A returned result may need only a stub value. A stateful in-memory repository may deserve a fake. A public send-once obligation may justify a mock interaction assertion.
Use these questions throughout:
Which hidden dependency makes the result nondeterministic or destructive?
Can an explicit parameter or small collaborator create a natural seam?
Is the contract about returned state, visible output, or an interaction?
Where does the system under test look up the name that must be patched?
Would this assertion still pass after a safe internal refactor?
1. Expose the clock instead of racing it
This function hides the current time:
from datetime import datetime, timezone
def build_alert(station, risk):
created_at = datetime.now(timezone.utc)
return {
"station" : station,
"risk" : risk,
"created_at" : created_at.isoformat(),
}
A test cannot know the exact microsecond in advance. Asserting “the timestamp is near now” adds timing tolerance and can still fail on a busy machine. Sleeping does not make the dependency deterministic.
Inject a callable with the behavior the function needs:
from datetime import datetime, timezone
def utc_now():
return datetime.now(timezone.utc)
def build_alert(station, risk, clock= utc_now):
created_at = clock()
return {
"station" : station,
"risk" : risk,
"created_at" : created_at.isoformat(),
}
The ordinary caller uses the real default. The test supplies a deterministic clock:
from datetime import datetime, timezone
def fixed_clock():
return datetime(2035 , 6 , 1 , 12 , 30 , tzinfo= timezone.utc)
def test_build_alert_uses_supplied_clock():
observed = build_alert("ridge-7" , "red" , clock= fixed_clock)
assert observed == {
"station" : "ridge-7" ,
"risk" : "red" ,
"created_at" : "2035-06-01T12:30:00+00:00" ,
}
The injected dependency is small: “a callable returning an aware datetime.” It does not expose an entire framework or require the test to alter process-global time.
Control randomness through the operation you need
Suppose equal-risk alerts are assigned a rotating radio channel:
import random
def choose_channel(channels, chooser= random.choice):
if not channels:
raise ValueError ("at least one channel is required" )
return chooser(channels)
A deterministic chooser can return the last item:
def choose_last(values):
return values[- 1 ]
def test_choose_channel_uses_supplied_chooser():
assert choose_channel(["alpha" , "beta" ], chooser= choose_last) == "beta"
This proves delegation and result behavior without globally seeding randomness. Use a seeded real generator only when the sequence produced by that generator is itself the subject. Tests should not depend on CPython’s exact random sequence unless that is an intentional compatibility contract.
The seam lets production and test callers supply different collaborators while the core behavior remains the same.
flowchart LR
A["Production caller"] --> B["Real clock or chooser"]
C["Test caller"] --> D["Fixed clock or chooser"]
B --> E["Explicit dependency seam"]
D --> E
E --> F["Alert behavior"]
F --> G["Observable result"]
2. Choose a double by the evidence it supplies
Test-double vocabulary is useful when it communicates behavior:
dummy
required shape but no meaningful behavior
an unused context argument
stub
a prepared answer
fixed clock returning one datetime
fake
working lightweight implementation
in-memory notification outbox
spy
records calls for later inspection
sender that retains delivered alerts
mock
preconfigured interaction expectations
spec-constrained sender checked for one public call
Tools and teams use these words with small variations. The design question is more important: what controlled behavior and observation does the test need?
An in-memory fake can be ordinary Python:
class MemorySender:
def __init__ (self ):
self .sent = []
def send(self , alert):
self .sent.append(alert)
def notify_if_red(alert, sender):
if alert["risk" ] == "red" :
sender.send(alert)
return True
return False
def test_red_alert_is_delivered_to_memory_sender():
sender = MemorySender()
alert = {"station" : "ridge-7" , "risk" : "red" }
delivered = notify_if_red(alert, sender)
assert delivered is True
assert sender.sent == [alert]
This asserts public result and final observable state. The fake has no network, sleep, credentials, or retry policy. It is easy to read and can support several tests without a mock API.
Checkpoint: choose the smallest controllable seam
{
const scripts = Array . from (
document . querySelectorAll ("script.fcpython-ojs-quiz-config" )
);
const script = scripts. find (
(node) => node. dataset . fcpythonRendered !== "true"
);
if (! script) {
return html `<div class="fcpython-quiz fcpython-quiz-warning">
Quiz configuration was not found.
</div>` ;
}
script. dataset . fcpythonRendered = "true" ;
const quiz = JSON . parse (script. textContent );
const container = html `<div class="fcpython-quiz"></div>` ;
const title = document . createElement ("h3" );
title. textContent = quiz. title ;
container. appendChild (title);
const instructions = document . createElement ("p" );
instructions. textContent = quiz. instructions ;
container. appendChild (instructions);
const progress = document . createElement ("div" );
progress. className = "fcpython-quiz-progress" ;
const counter = document . createElement ("p" );
counter. className = "fcpython-quiz-counter" ;
counter. setAttribute ("aria-live" , "polite" );
progress. appendChild (counter);
const tabs = document . createElement ("div" );
tabs. className = "fcpython-quiz-steps" ;
tabs. setAttribute ("role" , "tablist" );
tabs. setAttribute ("aria-label" , "Quiz questions" );
progress. appendChild (tabs);
container. appendChild (progress);
const questions = document . createElement ("div" );
questions. className = "fcpython-quiz-questions" ;
const feedbackNodes = [];
const questionPanels = [];
const stepButtons = [];
let currentQuestion = 0 ;
function setStepStatus (questionIndex, status) {
const step = stepButtons[questionIndex];
const question = quiz. questions [questionIndex];
const statuses = {
answered : "answered" ,
correct : "correct" ,
incorrect : "incorrect" ,
unanswered : "not answered" ,
};
step. classList . remove (
"is-answered" ,
"is-correct" ,
"is-incorrect" ,
"is-unanswered"
);
if (status) {
step. classList . add (`is- ${ status} ` );
}
const statusLabel = status ? `, ${ statuses[status]} ` : "" ;
step. setAttribute (
"aria-label" ,
`Question ${ questionIndex + 1 } : ${ question. prompt }${ statusLabel} `
);
}
function showQuestion (questionIndex, focusPanel = false ) {
currentQuestion = Math . max (
0 ,
Math . min (questionIndex, quiz. questions . length - 1 )
);
questionPanels. forEach ((panel, index) => {
panel. hidden = index !== currentQuestion;
});
stepButtons. forEach ((step, index) => {
const isCurrent = index === currentQuestion;
step. classList . toggle ("is-current" , isCurrent);
step. setAttribute ("aria-selected" , String (isCurrent));
step. tabIndex = isCurrent ? 0 : - 1 ;
});
counter. textContent = `Question ${ currentQuestion + 1 } of ${ quiz. questions . length } ` ;
const selected = questionPanels[currentQuestion]. querySelector (
"input[type='radio']:checked"
);
const isLastQuestion = currentQuestion === quiz. questions . length - 1 ;
previous. hidden = currentQuestion === 0 ;
next. hidden = isLastQuestion || ! selected;
check. hidden = ! isLastQuestion;
if (focusPanel) {
questionPanels[currentQuestion]. focus ();
}
}
quiz. questions . forEach ((question, questionIndex) => {
const tabId = ` ${ quiz. id } -question-tab- ${ questionIndex + 1 } ` ;
const panelId = ` ${ quiz. id } -question-panel- ${ questionIndex + 1 } ` ;
const step = document . createElement ("button" );
step. type = "button" ;
step. className = "fcpython-quiz-step" ;
step. id = tabId;
step. textContent = String (questionIndex + 1 );
step. setAttribute ("role" , "tab" );
step. setAttribute ("aria-controls" , panelId);
step. setAttribute ("aria-selected" , "false" );
step. tabIndex = - 1 ;
step. addEventListener ("click" , () => showQuestion (questionIndex));
step. addEventListener ("keydown" , (event ) => {
let destination = null ;
if (event . key === "ArrowRight" ) {
destination = (questionIndex + 1 ) % quiz. questions . length ;
} else if (event . key === "ArrowLeft" ) {
destination =
(questionIndex - 1 + quiz. questions . length ) % quiz. questions . length ;
} else if (event . key === "Home" ) {
destination = 0 ;
} else if (event . key === "End" ) {
destination = quiz. questions . length - 1 ;
}
if (destination !== null ) {
event . preventDefault ();
showQuestion (destination);
stepButtons[destination]. focus ();
}
});
stepButtons. push (step);
tabs. appendChild (step);
setStepStatus (questionIndex, "" );
const panel = document . createElement ("div" );
panel. className = "fcpython-quiz-panel" ;
panel. id = panelId;
panel. setAttribute ("role" , "tabpanel" );
panel. setAttribute ("aria-labelledby" , tabId);
panel. tabIndex = - 1 ;
const fieldset = document . createElement ("fieldset" );
fieldset. className = "fcpython-quiz-question" ;
const legend = document . createElement ("legend" );
legend. textContent = question. prompt ;
fieldset. appendChild (legend);
question. options . forEach ((option, optionIndex) => {
const label = document . createElement ("label" );
label. className = "fcpython-quiz-option" ;
const input = document . createElement ("input" );
input. type = "radio" ;
input. name = ` ${ quiz. id } - ${ question. id } ` ;
input. value = String (optionIndex);
input. addEventListener ("change" , () => {
setStepStatus (questionIndex, "answered" );
feedbackNodes[questionIndex]. textContent = "" ;
feedbackNodes[questionIndex]. className = "fcpython-quiz-feedback" ;
score. textContent = "" ;
showQuestion (questionIndex);
});
const text = document . createElement ("span" );
text. textContent = option;
label. appendChild (input);
label. appendChild (text);
fieldset. appendChild (label);
});
const feedback = document . createElement ("p" );
feedback. className = "fcpython-quiz-feedback" ;
feedback. setAttribute ("aria-live" , "polite" );
feedbackNodes. push (feedback);
fieldset. appendChild (feedback);
panel. appendChild (fieldset);
questionPanels. push (panel);
questions. appendChild (panel);
});
container. appendChild (questions);
const actions = document . createElement ("div" );
actions. className = "fcpython-quiz-actions" ;
const previous = document . createElement ("button" );
previous. type = "button" ;
previous. className = "fcpython-quiz-secondary" ;
previous. textContent = "Previous" ;
previous. addEventListener ("click" , () => {
showQuestion (currentQuestion - 1 , true );
});
const next = document . createElement ("button" );
next. type = "button" ;
next. textContent = "Next question" ;
next. addEventListener ("click" , () => {
showQuestion (currentQuestion + 1 , true );
});
const check = document . createElement ("button" );
check. type = "button" ;
check. textContent = "Check answers" ;
const reset = document . createElement ("button" );
reset. type = "button" ;
reset. className = "fcpython-quiz-secondary fcpython-quiz-reset" ;
reset. textContent = "Reset" ;
const score = document . createElement ("p" );
score. className = "fcpython-quiz-score" ;
score. setAttribute ("aria-live" , "polite" );
check. addEventListener ("click" , () => {
let correctCount = 0 ;
let firstQuestionToReview = null ;
quiz. questions . forEach ((question, questionIndex) => {
const selected = container. querySelector (
`input[name=" ${ quiz. id } - ${ question. id } "]:checked`
);
const feedback = feedbackNodes[questionIndex];
if (! selected) {
feedback. textContent = "Choose an answer before checking." ;
feedback. className = "fcpython-quiz-feedback" ;
setStepStatus (questionIndex, "unanswered" );
if (firstQuestionToReview === null ) {
firstQuestionToReview = questionIndex;
}
return ;
}
const selectedIndex = Number (selected. value );
if (selectedIndex === question. answer_index ) {
correctCount += 1 ;
feedback. textContent = `✅ Correct. ${ question. explanation } ` ;
feedback. className = "fcpython-quiz-feedback is-correct" ;
setStepStatus (questionIndex, "correct" );
} else {
const answer = question. options [question. answer_index ];
feedback. textContent = `❌ Not yet. Correct answer: ${ answer} . ${ question. explanation } ` ;
feedback. className = "fcpython-quiz-feedback is-incorrect" ;
setStepStatus (questionIndex, "incorrect" );
if (firstQuestionToReview === null ) {
firstQuestionToReview = questionIndex;
}
}
});
score. textContent = `Score: ${ correctCount} / ${ quiz. questions . length } ` ;
if (firstQuestionToReview !== null ) {
showQuestion (firstQuestionToReview, true );
}
});
reset. addEventListener ("click" , () => {
container. querySelectorAll ("input[type='radio']" ). forEach ((input) => {
input. checked = false ;
});
feedbackNodes. forEach ((feedback) => {
feedback. textContent = "" ;
feedback. className = "fcpython-quiz-feedback" ;
});
stepButtons. forEach ((_, questionIndex) => {
setStepStatus (questionIndex, "" );
});
score. textContent = "" ;
showQuestion (0 , true );
});
actions. appendChild (previous);
actions. appendChild (next);
actions. appendChild (check);
actions. appendChild (reset);
container. appendChild (actions);
container. appendChild (score);
showQuestion (0 );
return container;
}
3. Patch process state temporarily with monkeypatch
Some application boundaries intentionally read environment variables or the current directory. Redesigning every read as a parameter can make a thin adapter awkward. Pytest’s monkeypatch fixture changes state for one test and restores it afterward.
Example production setting:
import os
def alert_region():
return os.environ.get("METEOR_REGION" , "local" ).strip().casefold()
Test an explicit value:
def test_alert_region_reads_environment(monkeypatch):
monkeypatch.setenv("METEOR_REGION" , " NORTH " )
assert alert_region() == "north"
Test absence separately:
def test_alert_region_defaults_when_variable_is_absent(monkeypatch):
monkeypatch.delenv("METEOR_REGION" , raising= False )
assert alert_region() == "local"
After each test, pytest restores the previous environment. Do not manually put a guessed old value back; the variable may originally have been absent or held a user-specific value.
monkeypatch can also:
setitem or delitem on mappings;
setattr or delattr on objects/modules;
prepend to sys.path for specific import tests;
chdir temporarily; and
create a limited context() for a smaller patch lifetime.
Use these operations at the boundary that deliberately owns process state. A pure classifier should accept a value rather than read the environment in every call.
Change directories without leaking the process state
from pathlib import Path
def current_station_file():
return Path("stations.json" )
def test_station_file_is_relative_to_selected_workspace(
tmp_path,
monkeypatch,
):
monkeypatch.chdir(tmp_path)
assert current_station_file().resolve() == tmp_path / "stations.json"
The test makes the current-directory dependency explicit and restored. An even more reusable design would accept a base path, but a CLI adapter whose contract uses the current directory may intentionally be tested this way.
4. Patch the name the module actually looks up
Patch failures often come from changing the name where a function was defined instead of where the system under test imported it.
Suppose meteor_watch/alerts.py contains:
from meteor_watch.delivery import send_alert
def publish(alert):
send_alert(alert)
At import time, alerts binds its own name send_alert. Patching meteor_watch.delivery.send_alert later does not replace the already-bound name used by publish. Patch meteor_watch.alerts.send_alert:
import meteor_watch.alerts
def test_publish_uses_alert_module_sender(monkeypatch):
sent = []
def fake_send(alert):
sent.append(alert)
monkeypatch.setattr (meteor_watch.alerts, "send_alert" , fake_send)
alert = {"station" : "ridge-7" , "risk" : "red" }
meteor_watch.alerts.publish(alert)
assert sent == [alert]
If production instead used import meteor_watch.delivery and called meteor_watch.delivery.send_alert(...), that is the name path it looks up and the patch location changes accordingly.
The caller follows its local binding. A patch must replace that lookup path, not merely another module’s original definition.
flowchart LR
A["delivery.send_alert definition"] --> B["alerts.send_alert binding"]
B --> C["alerts.publish lookup"]
D["Patch delivery.send_alert only"] -. "does not replace bound name" .-> A
E["Patch alerts.send_alert"] --> B
C --> F["Controlled fake call"]
Reproduce the wrong-namespace symptom
Patch the definition module on purpose and run the focused test. If the real sender raises RuntimeError("network disabled"), seeing that exception proves the fake was not used. Inspect the import statement in the system under test, patch its lookup name, and rerun. This is stronger than adding another patch at random.
5. Use Mock when the interaction is the behavior
Python’s unittest.mock can configure return values, raise side effects, and record calls. Constrain the mock to a known collaborator shape:
from unittest.mock import Mock
class Sender:
def send(self , alert):
raise NotImplementedError
def test_red_alert_is_sent_once():
sender = Mock(spec= Sender)
alert = {"station" : "ridge-7" , "risk" : "red" }
delivered = notify_if_red(alert, sender)
assert delivered is True
sender.send.assert_called_once_with(alert)
spec=Sender prevents a test from configuring imaginary attributes that the real interface lacks. create_autospec(Sender, instance=True) can also enforce method signatures. A spec does not prove the real service works; it catches some test-double drift.
Use return_value when production consumes a collaborator’s result:
from unittest.mock import Mock
def next_sequence(counter):
return counter.next_value()
def test_next_sequence_returns_counter_value():
counter = Mock()
counter.next_value.return_value = 17
assert next_sequence(counter) == 17
Use side_effect to model an anticipated collaborator failure:
from unittest.mock import Mock
def safe_send(alert, sender):
try :
sender.send(alert)
except ConnectionError :
return False
return True
def test_safe_send_reports_connection_failure():
sender = Mock(spec= Sender)
sender.send.side_effect = ConnectionError ("radio offline" )
assert safe_send({"risk" : "red" }, sender) is False
Do not use a broad side effect merely to make every failure path convenient. The production contract should name which collaborator failures it handles.
Checkpoint: patch and specify the real seam
{
const scripts = Array . from (
document . querySelectorAll ("script.fcpython-ojs-quiz-config" )
);
const script = scripts. find (
(node) => node. dataset . fcpythonRendered !== "true"
);
if (! script) {
return html `<div class="fcpython-quiz fcpython-quiz-warning">
Quiz configuration was not found.
</div>` ;
}
script. dataset . fcpythonRendered = "true" ;
const quiz = JSON . parse (script. textContent );
const container = html `<div class="fcpython-quiz"></div>` ;
const title = document . createElement ("h3" );
title. textContent = quiz. title ;
container. appendChild (title);
const instructions = document . createElement ("p" );
instructions. textContent = quiz. instructions ;
container. appendChild (instructions);
const progress = document . createElement ("div" );
progress. className = "fcpython-quiz-progress" ;
const counter = document . createElement ("p" );
counter. className = "fcpython-quiz-counter" ;
counter. setAttribute ("aria-live" , "polite" );
progress. appendChild (counter);
const tabs = document . createElement ("div" );
tabs. className = "fcpython-quiz-steps" ;
tabs. setAttribute ("role" , "tablist" );
tabs. setAttribute ("aria-label" , "Quiz questions" );
progress. appendChild (tabs);
container. appendChild (progress);
const questions = document . createElement ("div" );
questions. className = "fcpython-quiz-questions" ;
const feedbackNodes = [];
const questionPanels = [];
const stepButtons = [];
let currentQuestion = 0 ;
function setStepStatus (questionIndex, status) {
const step = stepButtons[questionIndex];
const question = quiz. questions [questionIndex];
const statuses = {
answered : "answered" ,
correct : "correct" ,
incorrect : "incorrect" ,
unanswered : "not answered" ,
};
step. classList . remove (
"is-answered" ,
"is-correct" ,
"is-incorrect" ,
"is-unanswered"
);
if (status) {
step. classList . add (`is- ${ status} ` );
}
const statusLabel = status ? `, ${ statuses[status]} ` : "" ;
step. setAttribute (
"aria-label" ,
`Question ${ questionIndex + 1 } : ${ question. prompt }${ statusLabel} `
);
}
function showQuestion (questionIndex, focusPanel = false ) {
currentQuestion = Math . max (
0 ,
Math . min (questionIndex, quiz. questions . length - 1 )
);
questionPanels. forEach ((panel, index) => {
panel. hidden = index !== currentQuestion;
});
stepButtons. forEach ((step, index) => {
const isCurrent = index === currentQuestion;
step. classList . toggle ("is-current" , isCurrent);
step. setAttribute ("aria-selected" , String (isCurrent));
step. tabIndex = isCurrent ? 0 : - 1 ;
});
counter. textContent = `Question ${ currentQuestion + 1 } of ${ quiz. questions . length } ` ;
const selected = questionPanels[currentQuestion]. querySelector (
"input[type='radio']:checked"
);
const isLastQuestion = currentQuestion === quiz. questions . length - 1 ;
previous. hidden = currentQuestion === 0 ;
next. hidden = isLastQuestion || ! selected;
check. hidden = ! isLastQuestion;
if (focusPanel) {
questionPanels[currentQuestion]. focus ();
}
}
quiz. questions . forEach ((question, questionIndex) => {
const tabId = ` ${ quiz. id } -question-tab- ${ questionIndex + 1 } ` ;
const panelId = ` ${ quiz. id } -question-panel- ${ questionIndex + 1 } ` ;
const step = document . createElement ("button" );
step. type = "button" ;
step. className = "fcpython-quiz-step" ;
step. id = tabId;
step. textContent = String (questionIndex + 1 );
step. setAttribute ("role" , "tab" );
step. setAttribute ("aria-controls" , panelId);
step. setAttribute ("aria-selected" , "false" );
step. tabIndex = - 1 ;
step. addEventListener ("click" , () => showQuestion (questionIndex));
step. addEventListener ("keydown" , (event ) => {
let destination = null ;
if (event . key === "ArrowRight" ) {
destination = (questionIndex + 1 ) % quiz. questions . length ;
} else if (event . key === "ArrowLeft" ) {
destination =
(questionIndex - 1 + quiz. questions . length ) % quiz. questions . length ;
} else if (event . key === "Home" ) {
destination = 0 ;
} else if (event . key === "End" ) {
destination = quiz. questions . length - 1 ;
}
if (destination !== null ) {
event . preventDefault ();
showQuestion (destination);
stepButtons[destination]. focus ();
}
});
stepButtons. push (step);
tabs. appendChild (step);
setStepStatus (questionIndex, "" );
const panel = document . createElement ("div" );
panel. className = "fcpython-quiz-panel" ;
panel. id = panelId;
panel. setAttribute ("role" , "tabpanel" );
panel. setAttribute ("aria-labelledby" , tabId);
panel. tabIndex = - 1 ;
const fieldset = document . createElement ("fieldset" );
fieldset. className = "fcpython-quiz-question" ;
const legend = document . createElement ("legend" );
legend. textContent = question. prompt ;
fieldset. appendChild (legend);
question. options . forEach ((option, optionIndex) => {
const label = document . createElement ("label" );
label. className = "fcpython-quiz-option" ;
const input = document . createElement ("input" );
input. type = "radio" ;
input. name = ` ${ quiz. id } - ${ question. id } ` ;
input. value = String (optionIndex);
input. addEventListener ("change" , () => {
setStepStatus (questionIndex, "answered" );
feedbackNodes[questionIndex]. textContent = "" ;
feedbackNodes[questionIndex]. className = "fcpython-quiz-feedback" ;
score. textContent = "" ;
showQuestion (questionIndex);
});
const text = document . createElement ("span" );
text. textContent = option;
label. appendChild (input);
label. appendChild (text);
fieldset. appendChild (label);
});
const feedback = document . createElement ("p" );
feedback. className = "fcpython-quiz-feedback" ;
feedback. setAttribute ("aria-live" , "polite" );
feedbackNodes. push (feedback);
fieldset. appendChild (feedback);
panel. appendChild (fieldset);
questionPanels. push (panel);
questions. appendChild (panel);
});
container. appendChild (questions);
const actions = document . createElement ("div" );
actions. className = "fcpython-quiz-actions" ;
const previous = document . createElement ("button" );
previous. type = "button" ;
previous. className = "fcpython-quiz-secondary" ;
previous. textContent = "Previous" ;
previous. addEventListener ("click" , () => {
showQuestion (currentQuestion - 1 , true );
});
const next = document . createElement ("button" );
next. type = "button" ;
next. textContent = "Next question" ;
next. addEventListener ("click" , () => {
showQuestion (currentQuestion + 1 , true );
});
const check = document . createElement ("button" );
check. type = "button" ;
check. textContent = "Check answers" ;
const reset = document . createElement ("button" );
reset. type = "button" ;
reset. className = "fcpython-quiz-secondary fcpython-quiz-reset" ;
reset. textContent = "Reset" ;
const score = document . createElement ("p" );
score. className = "fcpython-quiz-score" ;
score. setAttribute ("aria-live" , "polite" );
check. addEventListener ("click" , () => {
let correctCount = 0 ;
let firstQuestionToReview = null ;
quiz. questions . forEach ((question, questionIndex) => {
const selected = container. querySelector (
`input[name=" ${ quiz. id } - ${ question. id } "]:checked`
);
const feedback = feedbackNodes[questionIndex];
if (! selected) {
feedback. textContent = "Choose an answer before checking." ;
feedback. className = "fcpython-quiz-feedback" ;
setStepStatus (questionIndex, "unanswered" );
if (firstQuestionToReview === null ) {
firstQuestionToReview = questionIndex;
}
return ;
}
const selectedIndex = Number (selected. value );
if (selectedIndex === question. answer_index ) {
correctCount += 1 ;
feedback. textContent = `✅ Correct. ${ question. explanation } ` ;
feedback. className = "fcpython-quiz-feedback is-correct" ;
setStepStatus (questionIndex, "correct" );
} else {
const answer = question. options [question. answer_index ];
feedback. textContent = `❌ Not yet. Correct answer: ${ answer} . ${ question. explanation } ` ;
feedback. className = "fcpython-quiz-feedback is-incorrect" ;
setStepStatus (questionIndex, "incorrect" );
if (firstQuestionToReview === null ) {
firstQuestionToReview = questionIndex;
}
}
});
score. textContent = `Score: ${ correctCount} / ${ quiz. questions . length } ` ;
if (firstQuestionToReview !== null ) {
showQuestion (firstQuestionToReview, true );
}
});
reset. addEventListener ("click" , () => {
container. querySelectorAll ("input[type='radio']" ). forEach ((input) => {
input. checked = false ;
});
feedbackNodes. forEach ((feedback) => {
feedback. textContent = "" ;
feedback. className = "fcpython-quiz-feedback" ;
});
stepButtons. forEach ((_, questionIndex) => {
setStepStatus (questionIndex, "" );
});
score. textContent = "" ;
showQuestion (0 , true );
});
actions. appendChild (previous);
actions. appendChild (next);
actions. appendChild (check);
actions. appendChild (reset);
container. appendChild (actions);
container. appendChild (score);
showQuestion (0 );
return container;
}
6. Prefer observable state unless interaction is a public obligation
This test is brittle:
def test_report_builder_calls_three_helpers(mocker):
parse = mocker.patch("meteor_watch.reports.parse" )
classify = mocker.patch("meteor_watch.reports.classify" )
render = mocker.patch("meteor_watch.reports.render" )
build_report("ridge-7|70" )
parse.assert_called_once()
classify.assert_called_once()
render.assert_called_once()
It also requires the third-party pytest-mock plugin, which this course does not install. More importantly, the assertions freeze three private steps. A refactor that combines parsing and classification breaks the test even if the returned report is identical.
Protect public output instead:
def test_build_report_returns_classified_station():
assert build_report("ridge-7|70" ) == "ridge-7|red"
An interaction assertion is appropriate when interaction is the promise:
send a red alert exactly once;
do not send a green alert;
commit a transaction only after all records validate; or
release a resource acquired by the function.
Even then, assert the narrow obligation, not every internal call:
from unittest.mock import Mock
def test_green_alert_is_not_sent():
sender = Mock(spec= Sender)
alert = {"station" : "lake-2" , "risk" : "green" }
delivered = notify_if_red(alert, sender)
assert delivered is False
sender.send.assert_not_called()
7. Avoid mock chains and imaginary worlds
This arrangement is hard to relate to a real interface:
client.session.return_value.channel.return_value.send.return_value = {
"ok" : True
}
Deep chains reproduce implementation navigation and allow a test-only world that no real client supports. Hide a complex vendor client behind a small application-owned adapter such as Sender.send(alert). Test core code against that small interface, and give the adapter a few integration tests at its real boundary when appropriate.
Do not patch Python builtins such as open across the whole process when tmp_path can exercise a real file. Do not patch time.sleep as a substitute for a design with injectable retry policy. Do not send to a real production service from a normal unit test.
8. Compare dependency injection with patching
Both techniques can be valid:
core behavior already accepts a collaborator
pass a stub, fake, or mock explicitly
a new design naturally benefits from visible dependencies
dependency injection
thin adapter intentionally reads environment/current directory
monkeypatch that state
legacy module bound an imported function
patch the lookup namespace temporarily
real local filesystem is fast and deterministic
tmp_path, not a fake file API
interaction is not public behavior
assert returned state/output instead of calls
Injection improves design visibility. Patching changes an existing lookup for a limited test lifetime. Patching every dependency can conceal a tightly coupled design; forcing every process adapter to accept dozens of parameters can also reduce clarity. Choose the smallest seam at the layer that owns the boundary.
Checkpoint: reject fragile interaction tests
{
const scripts = Array . from (
document . querySelectorAll ("script.fcpython-ojs-quiz-config" )
);
const script = scripts. find (
(node) => node. dataset . fcpythonRendered !== "true"
);
if (! script) {
return html `<div class="fcpython-quiz fcpython-quiz-warning">
Quiz configuration was not found.
</div>` ;
}
script. dataset . fcpythonRendered = "true" ;
const quiz = JSON . parse (script. textContent );
const container = html `<div class="fcpython-quiz"></div>` ;
const title = document . createElement ("h3" );
title. textContent = quiz. title ;
container. appendChild (title);
const instructions = document . createElement ("p" );
instructions. textContent = quiz. instructions ;
container. appendChild (instructions);
const progress = document . createElement ("div" );
progress. className = "fcpython-quiz-progress" ;
const counter = document . createElement ("p" );
counter. className = "fcpython-quiz-counter" ;
counter. setAttribute ("aria-live" , "polite" );
progress. appendChild (counter);
const tabs = document . createElement ("div" );
tabs. className = "fcpython-quiz-steps" ;
tabs. setAttribute ("role" , "tablist" );
tabs. setAttribute ("aria-label" , "Quiz questions" );
progress. appendChild (tabs);
container. appendChild (progress);
const questions = document . createElement ("div" );
questions. className = "fcpython-quiz-questions" ;
const feedbackNodes = [];
const questionPanels = [];
const stepButtons = [];
let currentQuestion = 0 ;
function setStepStatus (questionIndex, status) {
const step = stepButtons[questionIndex];
const question = quiz. questions [questionIndex];
const statuses = {
answered : "answered" ,
correct : "correct" ,
incorrect : "incorrect" ,
unanswered : "not answered" ,
};
step. classList . remove (
"is-answered" ,
"is-correct" ,
"is-incorrect" ,
"is-unanswered"
);
if (status) {
step. classList . add (`is- ${ status} ` );
}
const statusLabel = status ? `, ${ statuses[status]} ` : "" ;
step. setAttribute (
"aria-label" ,
`Question ${ questionIndex + 1 } : ${ question. prompt }${ statusLabel} `
);
}
function showQuestion (questionIndex, focusPanel = false ) {
currentQuestion = Math . max (
0 ,
Math . min (questionIndex, quiz. questions . length - 1 )
);
questionPanels. forEach ((panel, index) => {
panel. hidden = index !== currentQuestion;
});
stepButtons. forEach ((step, index) => {
const isCurrent = index === currentQuestion;
step. classList . toggle ("is-current" , isCurrent);
step. setAttribute ("aria-selected" , String (isCurrent));
step. tabIndex = isCurrent ? 0 : - 1 ;
});
counter. textContent = `Question ${ currentQuestion + 1 } of ${ quiz. questions . length } ` ;
const selected = questionPanels[currentQuestion]. querySelector (
"input[type='radio']:checked"
);
const isLastQuestion = currentQuestion === quiz. questions . length - 1 ;
previous. hidden = currentQuestion === 0 ;
next. hidden = isLastQuestion || ! selected;
check. hidden = ! isLastQuestion;
if (focusPanel) {
questionPanels[currentQuestion]. focus ();
}
}
quiz. questions . forEach ((question, questionIndex) => {
const tabId = ` ${ quiz. id } -question-tab- ${ questionIndex + 1 } ` ;
const panelId = ` ${ quiz. id } -question-panel- ${ questionIndex + 1 } ` ;
const step = document . createElement ("button" );
step. type = "button" ;
step. className = "fcpython-quiz-step" ;
step. id = tabId;
step. textContent = String (questionIndex + 1 );
step. setAttribute ("role" , "tab" );
step. setAttribute ("aria-controls" , panelId);
step. setAttribute ("aria-selected" , "false" );
step. tabIndex = - 1 ;
step. addEventListener ("click" , () => showQuestion (questionIndex));
step. addEventListener ("keydown" , (event ) => {
let destination = null ;
if (event . key === "ArrowRight" ) {
destination = (questionIndex + 1 ) % quiz. questions . length ;
} else if (event . key === "ArrowLeft" ) {
destination =
(questionIndex - 1 + quiz. questions . length ) % quiz. questions . length ;
} else if (event . key === "Home" ) {
destination = 0 ;
} else if (event . key === "End" ) {
destination = quiz. questions . length - 1 ;
}
if (destination !== null ) {
event . preventDefault ();
showQuestion (destination);
stepButtons[destination]. focus ();
}
});
stepButtons. push (step);
tabs. appendChild (step);
setStepStatus (questionIndex, "" );
const panel = document . createElement ("div" );
panel. className = "fcpython-quiz-panel" ;
panel. id = panelId;
panel. setAttribute ("role" , "tabpanel" );
panel. setAttribute ("aria-labelledby" , tabId);
panel. tabIndex = - 1 ;
const fieldset = document . createElement ("fieldset" );
fieldset. className = "fcpython-quiz-question" ;
const legend = document . createElement ("legend" );
legend. textContent = question. prompt ;
fieldset. appendChild (legend);
question. options . forEach ((option, optionIndex) => {
const label = document . createElement ("label" );
label. className = "fcpython-quiz-option" ;
const input = document . createElement ("input" );
input. type = "radio" ;
input. name = ` ${ quiz. id } - ${ question. id } ` ;
input. value = String (optionIndex);
input. addEventListener ("change" , () => {
setStepStatus (questionIndex, "answered" );
feedbackNodes[questionIndex]. textContent = "" ;
feedbackNodes[questionIndex]. className = "fcpython-quiz-feedback" ;
score. textContent = "" ;
showQuestion (questionIndex);
});
const text = document . createElement ("span" );
text. textContent = option;
label. appendChild (input);
label. appendChild (text);
fieldset. appendChild (label);
});
const feedback = document . createElement ("p" );
feedback. className = "fcpython-quiz-feedback" ;
feedback. setAttribute ("aria-live" , "polite" );
feedbackNodes. push (feedback);
fieldset. appendChild (feedback);
panel. appendChild (fieldset);
questionPanels. push (panel);
questions. appendChild (panel);
});
container. appendChild (questions);
const actions = document . createElement ("div" );
actions. className = "fcpython-quiz-actions" ;
const previous = document . createElement ("button" );
previous. type = "button" ;
previous. className = "fcpython-quiz-secondary" ;
previous. textContent = "Previous" ;
previous. addEventListener ("click" , () => {
showQuestion (currentQuestion - 1 , true );
});
const next = document . createElement ("button" );
next. type = "button" ;
next. textContent = "Next question" ;
next. addEventListener ("click" , () => {
showQuestion (currentQuestion + 1 , true );
});
const check = document . createElement ("button" );
check. type = "button" ;
check. textContent = "Check answers" ;
const reset = document . createElement ("button" );
reset. type = "button" ;
reset. className = "fcpython-quiz-secondary fcpython-quiz-reset" ;
reset. textContent = "Reset" ;
const score = document . createElement ("p" );
score. className = "fcpython-quiz-score" ;
score. setAttribute ("aria-live" , "polite" );
check. addEventListener ("click" , () => {
let correctCount = 0 ;
let firstQuestionToReview = null ;
quiz. questions . forEach ((question, questionIndex) => {
const selected = container. querySelector (
`input[name=" ${ quiz. id } - ${ question. id } "]:checked`
);
const feedback = feedbackNodes[questionIndex];
if (! selected) {
feedback. textContent = "Choose an answer before checking." ;
feedback. className = "fcpython-quiz-feedback" ;
setStepStatus (questionIndex, "unanswered" );
if (firstQuestionToReview === null ) {
firstQuestionToReview = questionIndex;
}
return ;
}
const selectedIndex = Number (selected. value );
if (selectedIndex === question. answer_index ) {
correctCount += 1 ;
feedback. textContent = `✅ Correct. ${ question. explanation } ` ;
feedback. className = "fcpython-quiz-feedback is-correct" ;
setStepStatus (questionIndex, "correct" );
} else {
const answer = question. options [question. answer_index ];
feedback. textContent = `❌ Not yet. Correct answer: ${ answer} . ${ question. explanation } ` ;
feedback. className = "fcpython-quiz-feedback is-incorrect" ;
setStepStatus (questionIndex, "incorrect" );
if (firstQuestionToReview === null ) {
firstQuestionToReview = questionIndex;
}
}
});
score. textContent = `Score: ${ correctCount} / ${ quiz. questions . length } ` ;
if (firstQuestionToReview !== null ) {
showQuestion (firstQuestionToReview, true );
}
});
reset. addEventListener ("click" , () => {
container. querySelectorAll ("input[type='radio']" ). forEach ((input) => {
input. checked = false ;
});
feedbackNodes. forEach ((feedback) => {
feedback. textContent = "" ;
feedback. className = "fcpython-quiz-feedback" ;
});
stepButtons. forEach ((_, questionIndex) => {
setStepStatus (questionIndex, "" );
});
score. textContent = "" ;
showQuestion (0 , true );
});
actions. appendChild (previous);
actions. appendChild (next);
actions. appendChild (check);
actions. appendChild (reset);
container. appendChild (actions);
container. appendChild (score);
showQuestion (0 );
return container;
}
9. Make the meteor alert deterministic
Build a small alert workflow with these public functions:
def build_alert(station, risk, clock):
"""Return an alert dictionary stamped with clock()."""
return {
"station" : station,
"risk" : risk,
"created_at" : clock().isoformat(),
}
def publish_if_enabled(alert, sender, enabled):
"""Send red alerts when enabled and report whether delivery occurred."""
if not enabled or alert["risk" ] != "red" :
return False
sender.send(alert)
return True
Add a thin environment adapter:
import os
def notifications_enabled():
raw = os.environ.get("METEOR_NOTIFY" , "0" ).strip().casefold()
return raw in {"1" , "true" , "yes" }
Your tests must:
inject a fixed aware datetime and assert the complete alert;
use a MemorySender to prove red delivery and green non-delivery through final state;
use monkeypatch.setenv and delenv for enabled, disabled, and absent environment cases;
use Mock(spec=Sender) for exactly one test where send-once is the public obligation;
configure side_effect=ConnectionError(...) for an explicitly handled sender failure;
reproduce one wrong-namespace patch and record the real evidence;
replace one assertion on private helper calls with a returned-state or output assertion; and
run every test both alone and as part of the complete file.
Hint 1: make a clock no more complicated than necessary
A zero-argument function returning one timezone-aware datetime is sufficient. The workflow needs clock(), not a fake calendar library.
Hint 2: decide whether state or interaction is the evidence
Use MemorySender.sent for most tests. Reserve assert_called_once_with for the explicit exactly-once delivery obligation.
Hint 3: follow the import statement to the patch target
If workflow.py imported send_alert directly, patch meteor_watch.workflow.send_alert. If it imported the module and accesses an attribute, patch that attribute on the module object it uses.
Show a deterministic core suite
from datetime import datetime, timezone
from unittest.mock import Mock
class Sender:
def send(self , alert):
raise NotImplementedError
class MemorySender:
def __init__ (self ):
self .sent = []
def send(self , alert):
self .sent.append(alert)
def fixed_clock():
return datetime(2035 , 6 , 1 , 12 , 30 , tzinfo= timezone.utc)
def test_build_alert_uses_injected_clock():
assert build_alert("ridge-7" , "red" , fixed_clock) == {
"station" : "ridge-7" ,
"risk" : "red" ,
"created_at" : "2035-06-01T12:30:00+00:00" ,
}
def test_red_alert_reaches_memory_sender():
sender = MemorySender()
alert = build_alert("ridge-7" , "red" , fixed_clock)
assert publish_if_enabled(alert, sender, enabled= True ) is True
assert sender.sent == [alert]
def test_green_alert_is_not_delivered():
sender = MemorySender()
alert = build_alert("lake-2" , "green" , fixed_clock)
assert publish_if_enabled(alert, sender, enabled= True ) is False
assert sender.sent == []
def test_red_alert_is_sent_exactly_once():
sender = Mock(spec= Sender)
alert = build_alert("ridge-7" , "red" , fixed_clock)
delivered = publish_if_enabled(alert, sender, enabled= True )
assert delivered is True
sender.send.assert_called_once_with(alert)
def test_notification_setting_accepts_yes(monkeypatch):
monkeypatch.setenv("METEOR_NOTIFY" , " yes " )
assert notifications_enabled() is True
def test_notification_setting_defaults_to_disabled(monkeypatch):
monkeypatch.delenv("METEOR_NOTIFY" , raising= False )
assert notifications_enabled() is False
Add the specified failure and patch-location tests around this core rather than turning every collaborator into a mock.
Key points
Hidden clocks, randomness, environment state, and services make tests hard to reproduce.
An explicit callable or small collaborator often creates the clearest seam.
Choose a stub, fake, spy, or mock by the evidence required, not by fashion.
monkeypatch changes process state temporarily and restores the real prior value.
Patch the namespace where the system under test looks up the name.
Mock(spec=...) catches some imaginary attributes; it does not prove a good test design.
Assert state or returned output unless an interaction is itself a public obligation.
Avoid real services, sleeps, broad builtin patches, and deep mock chains in focused tests.
Use real deterministic boundaries such as tmp_path when they are simpler and more faithful than a double.
Back to top