FreeCampus Python

Resolve Configuration Without Exposing Secrets

Merge defaults, TOML, environment variables, and explicit CLI options into one validated settings value while tracing precedence and preventing ordinary secret leaks.
python-foundations command-line-applications configuration secrets
Open in Colab
  • Level: Python Foundations
  • Estimated time: 4–5 hours
  • You will learn: Model validated settings, trace a documented precedence order, load TOML and environment strings safely, distinguish omission from explicit values, and redact sensitive input.
  • Practice in: Google Colab, JupyterLab, or a local Python project with temporary TOML files

Open this lesson in Google Colab

The trail-report command now has explicit arguments and clean streams. Operators also want durable preferences: a default report limit, a local TOML file, an environment-specific status, and a one-time command-line override.

If every function reads whichever source it happens to know about, behavior becomes difficult to predict. This lesson resolves raw sources once, validates them, and passes one ordinary settings value into the application.

As you work, answer these questions:

1. Replace scattered lookups with one settings value

This function is easy to call but difficult to reason about:

import os


def select_reports_bad(records):
    status = os.environ.get("TRAIL_STATUS", "closed")
    limit = int(os.environ.get("TRAIL_LIMIT", "20"))
    return [record for record in records if record["status"] == status][:limit]

The signature hides two inputs. A notebook, process, or test can receive a different answer because global environment state changed. Conversion can fail deep inside otherwise reusable work.

Make the dependency visible with a frozen dataclass:

from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Settings:
    status: str = "closed"
    limit: int = 20
    strict: bool = False
    cache_dir: Path = Path(".trail-cache")

The core accepts the value explicitly:

def select_reports(records, settings):
    selected = [
        record for record in records if record["status"] == settings.status
    ]
    return selected[: settings.limit]

One settings object gives the core typed, validated fields. frozen=True means the object cannot be reassigned field by field after resolution, reducing surprising mid-run changes. It does not turn referenced mutable objects into immutable objects, so use immutable field values here.

The dataclass is not a universal configuration framework. It is a clear result of a boundary process.

Raw configuration belongs outside the reusable core; one resolver converts and validates it before any records are processed.

flowchart LR
  A["Built-in defaults"] --> E["resolve_settings"]
  B["TOML values"] --> E
  C["Environment strings"] --> E
  D["CLI values"] --> E
  E --> F["Validated Settings"]
  F --> G["Reusable core"]

2. Write the precedence rule before implementing it

Use this unit’s rule from weakest to strongest:

  1. built-in default;
  2. TOML file;
  3. environment variable; and
  4. explicit CLI option.

The strongest supplied source wins for each field. A table makes conflict resolution reviewable:

Setting Default TOML Environment CLI Winner
limit 20 50 "30" omitted 30 from environment
status closed limited absent open open from CLI
strict false true "false" omitted false from environment

Notice that False can be the winning explicit value. A resolver must not use truthiness to decide whether a source was supplied:

def choose_bad(cli_value, fallback):
    return cli_value or fallback


print(choose_bad(False, True))
print(choose_bad(0, 20))
True
20

Both answers discard meaningful explicit values. Use None as the omission sentinel when False, 0, or "" can be real input:

def choose(cli_value, fallback):
    if cli_value is not None:
        return cli_value
    return fallback


print(choose(False, True))
print(choose(0, 20))
False
0

This affects parser design. If a setting can come from weaker sources, an omitted CLI option should often default to None, not to the final application default. Otherwise argparse would always appear to provide the strongest value.

Trace one field through ordered updates

One clear strategy starts with defaults and updates only supplied values:

def resolve_limit(default, file_value=None, env_value=None, cli_value=None):
    value = default
    source = "default"

    if file_value is not None:
        value = file_value
        source = "file"
    if env_value is not None:
        value = env_value
        source = "environment"
    if cli_value is not None:
        value = cli_value
        source = "command line"

    return value, source


print(resolve_limit(20, file_value=50, env_value=30))
(30, 'environment')

Tracking a non-sensitive source label helps explain behavior. Do not retain or display raw secrets merely to report provenance.

Read the stack from bottom to top: a stronger supplied source replaces the current value, while an omitted source leaves it unchanged.

flowchart BT
  A["Default"] --> B["TOML if supplied"]
  B --> C["Environment if supplied"]
  C --> D["CLI if supplied"]
  D --> E["Winning value"]

Checkpoint: trace the winning setting

3. Load one real TOML shape

TOML is readable and Python 3.11+ includes the tomllib reader. A small file named trail-report.toml might contain:

[trail]
status = "limited"
limit = 50
strict = true
cache_dir = "cache"

tomllib reads bytes and returns nested dictionaries:

from pathlib import Path
import tomllib


def load_toml(path):
    """Load one TOML document and return its trail table."""
    config_path = Path(path)
    with config_path.open("rb") as source:
        document = tomllib.load(source)

    trail = document.get("trail", {})
    if not isinstance(trail, dict):
        raise ValueError("[trail] must be a table")
    return trail

Use a temporary directory for a reproducible example:

from tempfile import TemporaryDirectory

with TemporaryDirectory() as directory:
    config_path = Path(directory) / "trail-report.toml"
    config_path.write_text(
        '[trail]\nstatus = "limited"\nlimit = 50\nstrict = true\n',
        encoding="utf-8",
    )
    values = load_toml(config_path)

print(values)
{'status': 'limited', 'limit': 50, 'strict': True}

tomllib parses TOML types, but it does not know the application’s contract. The resolver must reject unknown keys and wrong application-level values:

ALLOWED_FILE_KEYS = {"status", "limit", "strict", "cache_dir"}


def reject_unknown_keys(values):
    unknown = sorted(set(values) - ALLOWED_FILE_KEYS)
    if unknown:
        names = ", ".join(unknown)
        raise ValueError(f"unknown [trail] setting(s): {names}")

Silently ignoring limti = 50 would make the user believe a setting applied when it did not. A concise startup failure is safer.

Resolve relative paths from the configuration file

If the file contains cache_dir = "cache", the least surprising meaning is usually a directory beside the configuration file—not a directory beneath whatever current working directory launched the process:

def resolve_config_path(value, config_path):
    path = Path(value)
    if path.is_absolute():
        return path
    return Path(config_path).parent / path

Normalize only when the application needs it. Path.resolve() may touch filesystem assumptions; a simple joined path can preserve intent for later resource handling.

Treat parser failures and policy failures separately

Malformed TOML raises tomllib.TOMLDecodeError. A missing optional file may mean “use defaults,” while an explicitly requested missing file should usually be an error. That policy belongs to the command boundary, not load_toml(). Keeping the loader focused makes both cases expressible.

4. Convert environment strings deliberately

Environment names and values are strings. Read an injected mapping so the resolver does not depend on the process global:

def parse_env_bool(name, text):
    normalized = text.strip().casefold()
    if normalized in {"1", "true", "yes", "on"}:
        return True
    if normalized in {"0", "false", "no", "off"}:
        return False
    raise ValueError(f"{name} must be true/false, yes/no, on/off, or 1/0")

Do not use bool(text):

print(bool("false"))
print(bool("0"))
True
True

Both strings are non-empty. Explicit accepted forms make behavior predictable.

Use one prefix to prevent collisions:

def load_environment(environment):
    """Convert supported TRAIL_REPORT_* strings to partial settings."""
    values = {}

    if "TRAIL_REPORT_STATUS" in environment:
        values["status"] = environment["TRAIL_REPORT_STATUS"]
    if "TRAIL_REPORT_LIMIT" in environment:
        raw_limit = environment["TRAIL_REPORT_LIMIT"]
        try:
            values["limit"] = int(raw_limit)
        except ValueError as error:
            raise ValueError("TRAIL_REPORT_LIMIT must be an integer") from error
    if "TRAIL_REPORT_STRICT" in environment:
        values["strict"] = parse_env_bool(
            "TRAIL_REPORT_STRICT", environment["TRAIL_REPORT_STRICT"]
        )

    return values

Presence and content are different. An absent TRAIL_REPORT_STATUS means no environment override. An empty present value is supplied input and should usually fail validation rather than silently act absent.

print(load_environment({}))
print(load_environment({"TRAIL_REPORT_STRICT": "off"}))
{}
{'strict': False}

Checkpoint: convert and validate raw settings

5. Validate once after merging sources

Keep source loaders partial, then build the complete object:

VALID_STATUSES = {"open", "limited", "closed"}


def validate_settings(values):
    status = values["status"]
    limit = values["limit"]
    strict = values["strict"]
    cache_dir = Path(values["cache_dir"])

    if status not in VALID_STATUSES:
        raise ValueError(f"status must be one of {sorted(VALID_STATUSES)}")
    if isinstance(limit, bool) or not isinstance(limit, int) or limit < 1:
        raise ValueError("limit must be an integer of at least 1")
    if not isinstance(strict, bool):
        raise ValueError("strict must be true or false")

    return Settings(status, limit, strict, cache_dir)

The explicit isinstance(limit, bool) check matters because bool is a subclass of int in Python. True should not accidentally become a limit of one.

Merge in precedence order:

DEFAULT_VALUES = {
    "status": "closed",
    "limit": 20,
    "strict": False,
    "cache_dir": Path(".trail-cache"),
}


def merge_settings(file_values, env_values, cli_values):
    merged = dict(DEFAULT_VALUES)
    sources = {name: "default" for name in merged}

    for source_name, values in (
        ("file", file_values),
        ("environment", env_values),
        ("command line", cli_values),
    ):
        for name, value in values.items():
            if value is not None:
                merged[name] = value
                sources[name] = source_name

    return validate_settings(merged), sources

Check precedence and provenance:

settings, sources = merge_settings(
    {"limit": 50, "status": "limited"},
    {"limit": 30},
    {"status": "open", "strict": False},
)

assert settings.limit == 30
assert sources["limit"] == "environment"
assert settings.status == "open"
assert sources["status"] == "command line"
assert settings.strict is False

cli_values should contain only supported fields extracted from the argparse namespace. Do not use vars(namespace) blindly: it may include command, handler, input path, or other adapter details that are not settings.

Explain the result without dumping raw sources

Configuration bugs often sound like “the program ignored my value.” A safe --show-config mode can report the final non-sensitive value and its winning source:

def describe_settings(settings, sources):
    """Return stable, non-secret lines describing resolved settings."""
    rows = (
        ("status", settings.status),
        ("limit", settings.limit),
        ("strict", settings.strict),
        ("cache_dir", settings.cache_dir),
    )
    return [
        f"{name}={value} (source: {sources[name]})"
        for name, value in rows
    ]
for line in describe_settings(settings, sources):
    print(line)

This report is intentionally assembled from the validated result and safe source labels. Printing the original environment or the complete TOML document would disclose unrelated values and preserve invalid duplicates that did not win. If a future Settings object contains a token, omit that field entirely or display only a fixed redaction. Do not rely on remembering to mask one key in a generic dictionary dump.

Also decide which channel owns the report. If the user explicitly requests --show-config as the command result, stdout may be correct. If provenance is optional diagnostic detail during a normal JSON run, logging on stderr is correct. “Configuration information” does not determine a channel by itself; the public command contract does.

Distinguish an optional file from an explicitly requested file

Many applications search a conventional path but also accept --config PATH. Those cases deserve different missing-file behavior:

  • an absent conventional optional file means “continue with weaker defaults”;
  • a missing path explicitly supplied by the user is an anticipated error; and
  • a present but malformed file is an error in either case because silently ignoring it hides a configuration the user intended to apply.

Keep that decision around the loader. Do not make load_toml() catch every FileNotFoundError and return {} because the loader cannot tell why the path was chosen. The boundary knows whether a default search or an explicit option selected it.

6. Treat environment variables as input, not a vault

An environment variable keeps a secret out of source code, but it does not make the value inherently safe. A secret can leak through:

  • shell history when supplied directly as a command argument;
  • process inspection on systems where arguments or environments are visible;
  • a committed .env, TOML, notebook, or example output;
  • print(settings) or a dataclass representation;
  • debug logs and exception context;
  • a traceback containing a URL or object representation; or
  • copying a real token into a support message or AI prompt.

Avoid this interface:

trail-report upload --token real-secret-value

The value can remain in shell history and process arguments. Prefer a dedicated secret store in production systems. For a small interactive local command, getpass can read without echoing:

from getpass import getpass


def request_token():
    token = getpass("Upload token: ")
    if not token:
        raise ValueError("upload token cannot be empty")
    return token

getpass protects screen echo. It is not encrypted storage, rotation, access control, or a deployment secret manager. Name that limit so learners do not mistake one safe input channel for an entire security system.

Redact representations and diagnostics

Store only what must be used and never include the value in ordinary evidence:

def describe_token(token):
    if token is None:
        return "not configured"
    return f"configured ({len(token)} characters; value redacted)"


print(describe_token("example-not-a-real-token"))
configured (24 characters; value redacted)

Even length can reveal information in some threat models. It is included here only to show that safe metadata must be an explicit choice. A more cautious program would report only “configured.”

A .env file is a convention used by third-party tools, not a protection by itself and not part of Python’s standard library. If a project adopts one, keep real values out of version control, provide a placeholder example, restrict permissions appropriately, and still treat loaded values as untrusted input.

Checkpoint: find the accidental disclosure

7. Resolve trail-report settings from four sources

Build a focused resolver with this contract:

  • defaults: status closed, limit 20, strict false, cache directory .trail-cache;
  • optional [trail] TOML values;
  • TRAIL_REPORT_STATUS, TRAIL_REPORT_LIMIT, and TRAIL_REPORT_STRICT;
  • CLI values represented by a mapping whose omitted fields are None;
  • precedence: default < file < environment < CLI;
  • allowed statuses: open, limited, closed;
  • limit: integer of at least 1, explicitly rejecting booleans;
  • unknown TOML keys fail;
  • empty or invalid environment values fail; and
  • return (Settings, sources) without recording any secret values.

Use temporary files and plain mappings. Check at least these cases:

ordinary_file = {"status": "limited", "limit": 50}
ordinary_environment = {"TRAIL_REPORT_LIMIT": "30"}
ordinary_cli = {"status": "open", "limit": None, "strict": False}

The result should have status open from CLI, limit 30 from environment, strict false from CLI, and default cache directory. Add cases for no overrides, file only, an explicit zero, an empty environment string, an unknown file key, and a relative cache path.

Hint 1: normalize one source at a time Write separate functions for the TOML table and environment mapping. Make each return a partial dictionary whose values are already converted. Do not mix precedence logic into parsing.
Hint 2: update from weakest to strongest Copy defaults, then loop through (file, environment, command line) in that order. Update only values that are not None, and update a parallel source dictionary at the same moment.
Hint 3: construct Settings only after validation Validate the merged status, limit, strict flag, and path once. If validation passes, create the frozen dataclass. The core should never receive a half-valid dictionary.
Show a complete resolver after attempting the lab 
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class TrailSettings:
    status: str
    limit: int
    strict: bool
    cache_dir: Path


DEFAULTS = {
    "status": "closed",
    "limit": 20,
    "strict": False,
    "cache_dir": Path(".trail-cache"),
}
ALLOWED_KEYS = set(DEFAULTS)
ALLOWED_STATUSES = {"open", "limited", "closed"}


def validate_partial(values, source_name):
    unknown = sorted(set(values) - ALLOWED_KEYS)
    if unknown:
        raise ValueError(f"unknown {source_name} setting(s): {', '.join(unknown)}")
    return values


def environment_values(environment):
    values = {}
    names = {
        "TRAIL_REPORT_STATUS": "status",
        "TRAIL_REPORT_LIMIT": "limit",
        "TRAIL_REPORT_STRICT": "strict",
    }
    for environment_name, setting_name in names.items():
        if environment_name not in environment:
            continue
        raw = environment[environment_name]
        if setting_name == "limit":
            try:
                values[setting_name] = int(raw)
            except ValueError as error:
                raise ValueError(f"{environment_name} must be an integer") from error
        elif setting_name == "strict":
            values[setting_name] = parse_env_bool(environment_name, raw)
        else:
            values[setting_name] = raw
    return values


def resolve_trail_settings(file_values, environment, cli_values):
    partials = (
        ("file", validate_partial(dict(file_values), "file")),
        ("environment", environment_values(environment)),
        ("command line", validate_partial(dict(cli_values), "CLI")),
    )
    merged = dict(DEFAULTS)
    sources = {name: "default" for name in merged}

    for source_name, values in partials:
        for name, value in values.items():
            if value is not None:
                merged[name] = value
                sources[name] = source_name

    status = merged["status"]
    limit = merged["limit"]
    strict = merged["strict"]
    if status not in ALLOWED_STATUSES:
        raise ValueError("status must be open, limited, or closed")
    if isinstance(limit, bool) or not isinstance(limit, int) or limit < 1:
        raise ValueError("limit must be an integer of at least 1")
    if not isinstance(strict, bool):
        raise ValueError("strict must be true or false")

    settings = TrailSettings(
        status=status,
        limit=limit,
        strict=strict,
        cache_dir=Path(merged["cache_dir"]),
    )
    return settings, sources
This solution accepts already-loaded file values so file policy remains at the outer boundary. Connect load_toml() only after its missing/explicit-file behavior is specified.

Key points

  • Resolve raw sources once and pass one validated settings value to reusable code.
  • Document precedence before implementation; this unit uses default < TOML < environment < explicit CLI.
  • Use None for omission when false, zero, or empty values have distinct meanings. Never choose configuration by truthiness alone.
  • Read one real format completely, reject unknown keys, convert environment strings explicitly, and resolve relative paths from the configuration file.
  • Environment variables are inputs, not automatic secret storage. Protect command history, representations, logs, tracebacks, notebooks, and committed files from sensitive values.
  • Track safe source labels when they help explain behavior, but never retain secrets just to show provenance.

Continue learning

Back to top