FreeCampus Python

Find, Read, and Write Files Reliably

Build predictable paths, inspect and traverse directories, manage text-file resources, and write without surprising truncation.
python-foundations files-paths-external-data pathlib text-files
Open in Colab
  • Level: Python Foundations
  • Estimated time: 4–5 hours
  • You will learn: Build and inspect paths, distinguish files from directories, traverse a workspace in a stable order, manage file handles, choose between whole-file and streaming reads, and select a safe writing mode.
  • Practice in: Google Colab, JupyterLab, or a local editor using temporary workspaces

Open this lesson in Google Colab

A wildlife team returns from the field with notes from several observation stations. Your task is to find the notes, read them without leaking resources, and create a daily summary. That sounds simple, but the first version can fail by starting in a different folder, including yesterday’s generated report as new input, reading a directory as though it were a file, or erasing an existing log with the wrong mode.

Keep these questions beside your notebook:

1. Paths describe locations; files and directories occupy them

A filesystem organizes named entries inside directories. A path is a route to an entry, not the file’s contents. pathlib.Path represents that route with rules appropriate to the operating system.

from pathlib import Path

report_path = Path("field-notes") / "north" / "sightings.txt"

print(report_path)
print(report_path.name)
print(report_path.stem)
print(report_path.suffix)
print(report_path.parent)
print(report_path.parts)

The / operator here joins path components; it does not divide numbers. For the last component sightings.txt:

  • .name is "sightings.txt";
  • .stem is "sightings";
  • .suffix is ".txt";
  • .parent is the path ending in north; and
  • .parts exposes every component as a tuple.

A suffix is only part of a name. Renaming a file from .txt to .json does not convert its contents to JSON. Formats require parsing and serialization, which you will practice in Lesson 3.

Path construction does not create or open anything:

imagined = Path("archive") / "2026" / "observations.tar.gz"
print(bool(imagined))
print(imagined.exists())
print(imagined.name, imagined.stem, imagined.suffix, imagined.suffixes)

bool(imagined) is True even when the path is missing; a Path is an object, not a filesystem-state test. Notice also that .stem removes only the final suffix (.gz), while .suffixes reports ['.tar', '.gz']. Names with several dots need a documented naming rule rather than repeated string slicing.

Do not join components with str(root) + "/north/sightings.txt". Separators, drives, and absolute components vary by platform. Let Path perform path composition, and convert to text only when an external API actually requires a string.

A relative path begins at the process’s current working directory; an absolute path begins at a filesystem anchor such as a root or drive.

flowchart TD
  cwd["Current working directory"] --> project["field-notes/"]
  project --> north["north/"]
  north --> note["sightings.txt"]
  absolute["Filesystem root or drive"] --> cwd

If the current working directory changes, the same relative spelling can refer to a different location. The Path object does not search for the intended project automatically.

Inspect rather than assume:

from pathlib import Path

working_directory = Path.cwd()
candidate = Path("field-notes") / "north" / "sightings.txt"

print("working directory:", working_directory)
print("relative?:", not candidate.is_absolute())
print("candidate from cwd:", working_directory / candidate)

Path.resolve() can produce an absolute path and resolve symbolic components, but its exact filesystem behavior depends on the strict argument and the platform. It does not prove that you selected the correct project root. The best repair for an ambiguous relative path is usually to pass the intended root into the function.

def notes_path(workspace, station_name):
    """Return the notes path for one station under an explicit workspace."""
    return Path(workspace) / "field-notes" / station_name / "sightings.txt"

The function no longer depends silently on wherever the process happened to start.

2. Create a safe field-notes workspace

The next fixture creates every file it needs. Run the cell again whenever you want a clean starting point.

from pathlib import Path
from tempfile import TemporaryDirectory

temporary_directory = TemporaryDirectory()
workspace = Path(temporary_directory.name)
notes_root = workspace / "field-notes"
(notes_root / "north").mkdir(parents=True)
(notes_root / "south").mkdir(parents=True)
(notes_root / "empty-station").mkdir()

(notes_root / "north" / "sightings.txt").write_text(
    "otter,3\nheron,2\n",
    encoding="utf-8",
)
(notes_root / "south" / "sightings.txt").write_text(
    "fox,1\notter,2\n",
    encoding="utf-8",
)
(notes_root / "README.md").write_text(
    "Field notes are recorded once per station.\n",
    encoding="utf-8",
)

print("workspace:", workspace)

parents=True permits Python to create missing parents. Without it, creating field-notes/north would fail if field-notes did not exist. By default, mkdir() also fails if the final directory already exists. exist_ok=True is appropriate only when an existing directory satisfies the contract; do not use it to silence a collision that ought to be investigated.

Ask what each path currently represents:

north_note = notes_root / "north" / "sightings.txt"
missing_note = notes_root / "west" / "sightings.txt"

for path in [notes_root, north_note, missing_note]:
    print(
        path.relative_to(workspace),
        "exists=", path.exists(),
        "file=", path.is_file(),
        "directory=", path.is_dir(),
    )

exists() alone is not enough. A directory exists, but read_text() requires a file. A path can also disappear between a check and the following operation, so handle the operation’s exception at a boundary where absence is genuinely expected.

Checkpoint: make location explicit

3. Discover inputs in a deterministic order

iterdir() yields a directory’s immediate children. glob() selects paths that match a pattern. rglob() searches recursively.

print("immediate entries:")
for path in sorted(notes_root.iterdir(), key=lambda item: item.name):
    print("-", path.name)

print("station notes:")
station_files = sorted(notes_root.glob("*/sightings.txt"))
for path in station_files:
    print("-", path.relative_to(notes_root))

Do not rely on the order returned by the filesystem. Sort when order affects summaries, identifiers, tests, or reproducibility. Sorting Path objects is usually enough for paths with a shared root; an explicit key documents another policy such as case-insensitive names.

Pattern matching should describe intended inputs, not merely all text files. This broad search can accidentally include generated output on the next run:

output_path = notes_root / "daily-summary.txt"
output_path.write_text("generated output\n", encoding="utf-8")

broad_matches = sorted(notes_root.rglob("*.txt"))
print([path.relative_to(notes_root) for path in broad_matches])

The pattern */sightings.txt encodes two useful constraints: one station directory and the exact input filename. You can also filter explicitly:

input_files = [
    path
    for path in sorted(notes_root.rglob("*.txt"))
    if path.name == "sightings.txt" and path != output_path
]

Choose a rule that remains correct after output exists. “Delete the output before discovery” is fragile because a crash can leave it behind.

4. A file handle is a managed, stateful resource

Path.read_text() is convenient for a small file. Underneath, Python opens a text stream, reads it, and closes it. When you need line-by-line processing or more control, open the file explicitly with a with statement:

first_note = station_files[0]

with first_note.open(mode="r", encoding="utf-8") as handle:
    print("inside block, closed?:", handle.closed)
    first_line = handle.readline()
    remaining_text = handle.read()

print("outside block, closed?:", handle.closed)
print("first line:", repr(first_line))
print("remaining:", repr(remaining_text))

A handle has a cursor. readline() advances past one line; the following read() begins from that position, not from the start. Opening the file again creates a new handle with a new cursor.

Make cursor exhaustion observable:

with first_note.open(encoding="utf-8") as handle:
    first_pass = handle.read()
    second_pass = handle.read()

print("first pass:", repr(first_pass))
print("second pass:", repr(second_pass))
assert second_pass == ""

You could call handle.seek(0) before the second read, but reopening is often clearer when you truly need two independent passes. The built-in form open(first_note, encoding="utf-8") also accepts Path objects; Path.open() is used here because it keeps the location and operation visually together.

The context manager closes the resource even when code inside the block raises an exception. It does not suppress that exception:

try:
    with first_note.open(encoding="utf-8") as handle:
        raw_count = handle.readline().split(",")[1]
        int("not-a-number")
except ValueError as error:
    print(type(error).__name__, error)

print("closed after failure?:", handle.closed)

That separation is valuable: resource cleanup is guaranteed, while the unexpected or deliberately caught failure remains observable.

WarningA closed handle cannot read more data

Store the values you need inside the with block. Returning a handle from a helper whose context has already ended returns a closed resource, not deferred file contents.

5. Read small files whole; stream large or record-oriented files

For a small configuration or note, whole-file methods are clear:

text = first_note.read_text(encoding="utf-8")
lines = text.splitlines()
print(lines)

read_text() returns one string. It is then possible to split, search, or parse that value. The complete contents must fit comfortably in memory.

For a large log or naturally line-oriented input, iterate over the handle:

def count_animals(path):
    """Return total observed animals from a small comma-separated note."""
    total = 0
    with Path(path).open(encoding="utf-8") as handle:
        for line_number, raw_line in enumerate(handle, start=1):
            line = raw_line.rstrip("\n")
            animal, raw_count = line.split(",")
            try:
                total += int(raw_count)
            except ValueError as error:
                raise ValueError(
                    f"{path}:{line_number}: invalid count {raw_count!r}"
                ) from error
    return total


for path in station_files:
    print(path.parent.name, count_animals(path))

Only one line at a time needs to be held by the loop. enumerate preserves line context for diagnostics. The narrow rstrip("\n") removes the line terminator without deleting meaningful leading or trailing spaces. Lesson 2 develops newline policies further, and Lesson 3 replaces the deliberately simple comma split with the CSV parser needed for quoted fields.

Choose from the data shape and size, not from a rule that one method is always superior.

Checkpoint: manage reads and discovery

6. Writing modes express different promises

The common text modes are:

Mode Promise Important risk or behavior
"r" read an existing file missing path raises FileNotFoundError
"w" create or replace a file an existing file is truncated when opened
"a" append at the end, creating if needed reruns may duplicate content
"x" create a new file only existing path raises FileExistsError

write() does not add a newline:

mode_demo = workspace / "mode-demo.txt"
with mode_demo.open("w", encoding="utf-8") as handle:
    handle.write("north")
    handle.write("south")

print(repr(mode_demo.read_text(encoding="utf-8")))

write() returns the number of text characters accepted by the stream. The return value can support a focused check, although reopening and comparing the artifact is stronger evidence of the saved result:

with mode_demo.open("a", encoding="utf-8") as handle:
    character_count = handle.write("\nnight shift\n")

assert character_count == len("\nnight shift\n")
assert mode_demo.read_text(encoding="utf-8").endswith("night shift\n")

Run that cell twice and the line appears twice. Append mode preserves existing content, but it does not make retries idempotent.

The result is 'northsouth'. Include the separator required by the file contract:

with mode_demo.open("w", encoding="utf-8", newline="\n") as handle:
    handle.write("north\n")
    handle.write("south\n")

Opening with "w" immediately truncates an existing file, even if a later calculation fails before the first write. Compute and validate the complete text before opening the destination when practical:

summary_lines = []
for path in station_files:
    summary_lines.append(f"{path.parent.name}: {count_animals(path)}")

summary_text = "\n".join(summary_lines) + "\n"
assert summary_text.count("\n") == len(summary_lines)
output_path.write_text(summary_text, encoding="utf-8", newline="\n")
print(summary_text)

Appending is suitable for an event log only when duplicate appends on retry are acceptable or prevented by another rule. Exclusive creation is useful when replacing an existing destination would indicate a mistake:

reservation = workspace / "reserved-name.txt"
reservation.write_text("first owner\n", encoding="utf-8")

try:
    with reservation.open("x", encoding="utf-8") as handle:
        handle.write("second owner\n")
except FileExistsError:
    print("destination already exists; original preserved")

Lesson 2 will build a temporary-write, verify, and replace sequence for outputs that should become official only when complete.

7. Handle expected filesystem failures at the right boundary

Common filesystem exceptions carry different evidence:

  • FileNotFoundError: a required path or parent is missing;
  • FileExistsError: exclusive creation or directory creation collided;
  • IsADirectoryError: an operation expected a file but received a directory;
  • NotADirectoryError: some path component expected as a directory is not one;
  • PermissionError: the operating system denied the operation;
  • broader OSError: other operating-system or filesystem conditions.

Catch only a failure the caller can meaningfully handle:

def read_optional_note(path):
    """Return optional note text, or an empty string when it is absent."""
    try:
        return Path(path).read_text(encoding="utf-8")
    except FileNotFoundError:
        return ""


print(repr(read_optional_note(workspace / "optional.txt")))

This policy is valid only because the docstring declares absence normal. A required configuration should instead let the failure propagate or add useful context. Do not catch Exception and report “file missing”; decoding failures, permissions, and programming bugs require different responses.

8. Build the daily field-note summary

Complete a clean, self-contained lab:

  1. Create a temporary workspace with incoming/north, incoming/south, and output directories.
  2. Write at least two sightings.txt inputs and one unrelated .txt file.
  3. Implement discover_notes(root) to return only station notes in sorted order.
  4. Implement summarize_note(path) to stream records and return one station total with source-and-line context for invalid counts.
  5. Build the complete report in memory, end it with one newline, then write it to output/daily-summary.txt.
  6. Run the pipeline twice and assert that discovery never includes the output and that the report text is identical.
  7. Change one count to "many". Confirm the exception identifies its file and line while the previous report remains available because calculation occurs before writing.

Start from these contracts:

from pathlib import Path


def discover_notes(root):
    """Return station sightings files beneath root in deterministic order."""
    root = Path(root)
    return sorted(root.glob("*/sightings.txt"))


def summarize_note(path):
    """Return (station_name, total_count) for one sightings file."""
    path = Path(path)
    total = 0
    with path.open(encoding="utf-8") as handle:
        for line_number, raw_line in enumerate(handle, start=1):
            animal, raw_count = raw_line.rstrip("\n").split(",")
            if not animal:
                raise ValueError(f"{path}:{line_number}: empty animal name")
            try:
                total += int(raw_count)
            except ValueError as error:
                raise ValueError(
                    f"{path}:{line_number}: invalid count {raw_count!r}"
                ) from error
    return path.parent.name, total
Compare one complete integration after finishing your own 
from pathlib import Path
from tempfile import TemporaryDirectory

with TemporaryDirectory() as temporary_name:
    root = Path(temporary_name)
    incoming = root / "incoming"
    output = root / "output"
    (incoming / "north").mkdir(parents=True)
    (incoming / "south").mkdir()
    output.mkdir()

    (incoming / "north" / "sightings.txt").write_text(
        "otter,3\nheron,2\n", encoding="utf-8"
    )
    (incoming / "south" / "sightings.txt").write_text(
        "fox,1\notter,2\n", encoding="utf-8"
    )
    (incoming / "ignore.txt").write_text("not an input\n", encoding="utf-8")

    paths = discover_notes(incoming)
    rows = [summarize_note(path) for path in paths]
    report = "\n".join(f"{station}: {total}" for station, total in rows) + "\n"
    report_path = output / "daily-summary.txt"
    report_path.write_text(report, encoding="utf-8", newline="\n")

    assert [path.parent.name for path in paths] == ["north", "south"]
    assert report == "north: 5\nsouth: 3\n"
    assert report_path.read_text(encoding="utf-8") == report
    assert discover_notes(incoming) == paths

The narrow discovery rule makes generated output irrelevant. The report is fully constructed before write_text opens the destination. The next lessons will replace the simple record grammar with robust external formats and add a verified replacement step.

Checkpoint: choose the writing contract

9. Key points for reliable file work

  • A Path describes a location. Resolve relative paths from a deliberate root, not an unspoken assumption about the working directory.
  • Check whether a path is a file or directory when the distinction matters, but still handle operation-time failures appropriately.
  • Select intended inputs and sort them when processing order affects output.
  • A with statement closes a file handle on normal and exceptional paths; it does not hide the exception.
  • Read small files whole for clarity and stream large or record-oriented files when memory and context matter.
  • Modes r, w, a, and x express different contracts. In particular, w truncates and write() adds no newline automatically.
  • Compute and validate complete output before opening a destination for replacement whenever practical.

10. References and next step

Next, keep the same path and resource discipline while investigating what turns stored bytes into characters—and why the wrong encoding or newline assumption can damage otherwise correct text.

Back to top