FreeCampus Python

Repeat the Quality Gate in CI

Build and diagnose a GitHub Actions workflow that installs a clean project and repeats the same check-only quality commands used locally.
python-foundations code-quality-maintainability continuous-integration github-actions
Open in Colab
  • Level: Python Foundations
  • Estimated time: 3–4 hours
  • You will learn: Explain a workflow from event to exit status, keep local and CI commands aligned, and trace a failed clean run to source, setup, or configuration.
  • Practice in: A local project plus supplied GitHub Actions files and run transcripts

Open this lesson in Google Colab

1. A clean runner tests committed assumptions

The local museum gate passes. That result depends on one checkout, environment, and set of installed tools. Continuous integration starts a new automated run from committed files and a declared workflow.

Before writing YAML, answer these questions:

  1. Which repository event should start the quality check?
  2. What must the runner install before it can run project commands?
  3. How does a failed command stop the job?
  4. Which local and CI commands must remain identical?
  5. What hidden local assumption can a clean environment expose?

Create .github/workflows/quality.yml:

name: Quality

on:
  pull_request:
  push:

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.10"
      - name: Install project and quality tools
        run: python -m pip install -e ".[dev]"
      - name: Lint
        run: ruff check src tests
      - name: Check formatting
        run: ruff format --check src tests
      - name: Check types
        run: mypy src
      - name: Run tests
        run: pytest -q

The file describes one workflow named Quality:

  • on declares events;
  • jobs contains independently scheduled work;
  • quality is a machine ID for one job;
  • runs-on chooses a hosted runner image;
  • steps execute in order within that job;
  • uses invokes a reusable action; and
  • run invokes a shell command.

A real project must declare its development tools in a way the install command can resolve. If the project uses Poetry, a lock file, or another environment manager, use its documented install command rather than copying .[dev] blindly.

A Git event creates a clean runner that reconstructs the project before executing the gate.

flowchart LR
  A[Push or pull request] --> B[Workflow]
  B --> C[Clean runner]
  C --> D[Checkout]
  D --> E[Set up Python]
  E --> F[Install project]
  F --> G[Run quality commands]
  G --> H[Shared pass or failure]

2. Order steps by their dependencies

The linter cannot inspect source before checkout. Tests cannot import the project before installation. Put each prerequisite before its consumer:

  1. check out the commit/ref associated with the run;
  2. select a supported Python interpreter;
  3. install the declared project and development dependencies;
  4. run fast static checks;
  5. run the behavior suite; and
  6. build documentation/package only if those artifacts belong to this gate.

A workflow run uses the workflow version stored in the associated commit or Git reference. Uncommitted local fixes do not reach it. That is a feature: CI asks whether committed state is self-contained.

A nonzero exit status is the gate signal

Ruff, MyPy, and pytest exit zero on success and nonzero on a detected problem or execution failure. GitHub Actions marks a normal run step failed when its command exits nonzero.

This command destroys that signal:

pytest -q || true

If pytest exits 1, the shell runs true, which exits 0. The step looks green while tests are red. Likewise, an accidental continue-on-error: true changes a required gate into informational output.

Use non-blocking steps only when failure is intentionally advisory and clearly labeled. This unit’s quality commands are gates, so let them fail.

Checkpoint: workflow anatomy

3. Make local and CI commands one contract

Drift creates confusing states:

Local hook CI Resulting risk
ruff check src ruff check src tests Test lint failures appear only after push
mypy src no MyPy step CI accepts type gaps local contributors reject
pytest 9 from lock unbounded latest pytest Same commit can receive different behavior
formatter rewrites formatter check-only Fine when the distinction is documented

Write the authoritative check-only sequence in the README, task runner, or a small script that both environments call. Direct commands are easiest for this small project:

ruff check src tests
ruff format --check src tests
mypy src
pytest -q

The local pre-commit repair loop may run ruff format in place. The final local verification and CI must use --check. That is intentional behavior, not drift.

Pin enough state to reproduce the result

A clean environment still changes over time if it installs unconstrained latest tools. Use the project’s chosen dependency record and review updates. This can be a lock file, exact constraints, or a controlled range plus regular testing.

Actions also have versions. A tag such as actions/checkout@v4 tracks a major release. Security-sensitive projects may pin an immutable commit SHA and use an update process. Foundations learners need to recognize that an action is executable dependency code, not memorize one universal pinning policy.

Do not duplicate policy in the workflow

Keep rule selection, target Python, source paths, and MyPy strict settings in pyproject.toml. The workflow should call tools; it should not contain a second copy of every rule flag. One committed configuration reduces disagreement among editor, local command, hook, and CI.

4. Classify the failing stage before editing source

A red workflow is not always a code defect. Start at the earliest failed step and classify it.

Source-quality failure

Run ruff check src tests
F821 Undefined name `rank_attempt`
Found 1 error.
Error: Process completed with exit code 1.

Reproduce locally from the project root:

ruff check src tests

Read the rule and repair the source or a justified policy boundary.

Installation failure

Run python -m pip install -e ".[dev]"
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

Do not change Python logic. The runner is in the wrong directory or the project metadata was not committed. Compare checkout paths and working-directory.

Environment/version failure

SyntaxError: Pattern matching is only supported in Python 3.10 and greater

If a matrix job selected Python 3.9 while the project requires 3.10, align the matrix with the declared support range. If the project promises 3.9, the source is incompatible. The contract determines the repair.

Configuration failure

A malformed YAML file may prevent the workflow from starting, so no shell step exists to inspect. Use an editor/YAML parser, compare indentation, and read the workflow annotation. A correctly parsed workflow can still contain a wrong key or expression; consult the provider’s syntax reference.

Test failure only on CI

Compare:

  • Python and dependency versions;
  • operating system and path case;
  • locale, timezone, environment variables, and clock;
  • files present in Git versus only in the local working tree; and
  • test order or leaked global state.

Do not label CI “random.” State the observed difference and reduce it through a controlled local reproduction or a more explicit fixture.

Checkpoint: diagnose the clean run

5. Use a small version matrix for a real support claim

A library that claims Python 3.10–3.13 support should execute tests on those versions. A matrix avoids copying the job:

jobs:
  quality:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install project
        run: python -m pip install -e ".[dev]"
      - name: Run quality gate
        run: |
          ruff check src tests
          ruff format --check src tests
          mypy src
          pytest -q

This produces one job variant per value. fail-fast: false lets all variants report, which helps distinguish “source fails everywhere” from “only the oldest supported interpreter fails.”

Do not add operating systems, dependency combinations, and dozens of versions without a support question. Every matrix dimension multiplies cost and output. The Midnight Museum exercise can start with Python 3.10; the course repository’s real CI uses a supported-version matrix.

Separate steps or one gate step?

Separate named steps show exactly which tool failed and can make logs easier to scan. One script reduces command duplication if local and CI call the same script. Both are valid:

  • for a small lesson, separate steps reveal the chain;
  • for a mature project, one versioned quality task can keep environments aligned while still printing each command.

Choose the design that makes the contract and failure visible.

6. State what a passing workflow proves

A green workflow demonstrates that the workflow version associated with the run completed its selected jobs and steps successfully in their declared runner environments. It does not prove:

  • every possible input works;
  • every behavior has a test;
  • a security vulnerability is absent;
  • a contributor ran local hooks;
  • production matches the runner; or
  • the code is easy to maintain.

CI reduces dependence on hidden local state and gives collaborators a shared result. Its strength comes from a clear, relevant gate—not from the color green alone.

Checkpoint: CI scope and design

7. Lab: make the remote gate match the local gate

Start with this incomplete workflow:

name: Quality

on:
  pull_request:

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.10"
      - name: Install
        run: python -m pip install -e ".[dev]"
      - name: Quality
        run: |
          ruff check src || true
          pytest -q

Complete these tasks:

  1. remove the hidden-success behavior;
  2. make Ruff inspect src and tests;
  3. add Ruff format check, strict MyPy, and pytest;
  4. keep project policy in pyproject.toml;
  5. name steps so the first failed tool is visible;
  6. compare each CI command with the final local command sequence;
  7. explain how the workflow would reveal an uncommitted local fixture; and
  8. diagnose three supplied cases: missing metadata, one Python-version failure, and a Ruff rule failure.

If you have a GitHub repository, you can push a branch and inspect the real run. If not, retain the authored YAML, parse it locally, and annotate the supplied transcripts. The assessed artifact and reasoning are the same.

Hint A: preserve exit status

Remove || true. Put each required command in a normal run step so a nonzero status fails the job.

Hint B: use check-only formatting

CI should call ruff format --check src tests; it should not rewrite runner files that cannot be committed back to the branch.

Hint C: compare the four commands

The intended sequence is Ruff lint, Ruff format check, MyPy on src, and pytest. Ensure the same paths appear in the local acceptance gate.

Show one complete workflow 
name: Quality

on:
  pull_request:
  push:

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.10"
      - name: Install project and quality tools
        run: python -m pip install -e ".[dev]"
      - name: Lint
        run: ruff check src tests
      - name: Check formatting
        run: ruff format --check src tests
      - name: Check types
        run: mypy src
      - name: Run tests
        run: pytest -q

The install command must match the project’s actual dependency declaration. A passing local gate immediately before push and this clean runner result provide two complementary environment checks.

Key points

  • CI reconstructs committed project state on a declared runner, exposing hidden local files, dependencies, paths, and environment assumptions.
  • Order checkout, interpreter setup, installation, and checks by dependency.
  • Required commands must preserve nonzero exit status; do not hide failures with || true or accidental non-blocking settings.
  • Keep local and CI check-only commands aligned and project policy in committed configuration.
  • Diagnose the earliest failed stage as source, configuration, installation, or environment evidence before editing application code.
  • A version matrix should test a real support claim, and a green result remains bounded by the workflow that actually ran.

References

Continue to Lesson 7: Refactor in Small, Verified Steps

Back to top