FreeCampus Python

Unit Challenge: Build a Baseline before a Bigger Model

Evaluate a simple classifier against a baseline while keeping test data out of fitting.

↗ Open in Colab

• Goal: Evaluate a simple classifier against a baseline while keeping test data out of fitting.
• Estimated time: 60–120 minutes of active work.
• Before starting: Complete the unit lessons.
• Keep: Your source, observed outputs, checks, and a debugging note.

Open this lesson in Google Colab

This community challenge is ungraded. Use the checks to review your own work; there is no submission service or automatic grade. Completion is self-reported in this browser on this device.

1. Build a baseline before a bigger model

Use sklearn.datasets.load_iris and train_test_split with test_size=0.25, random_state=42, stratify=y. Fit DummyClassifier(strategy=“most_frequent”) and DecisionTreeClassifier(max_depth=3, random_state=42) using training data only. Report test size, accuracy, and confusion matrix for each; explain matrix rows and columns.

Predict an ordinary result before coding. Start with the smallest part you can run and inspect. Use Colab for notebooks and the local workspace introduced in the units for packaging or command-line tasks.

2. Set up the experiment

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y
)
# Fit both estimators on training data only.

The starter is deliberately incomplete. Implement the contract; do not change the expected results to match an incorrect implementation.

3. Check the observable results

1. There are 112 training examples and 38 test examples, with all three classes in each.
2. Each confusion matrix sums to 38; trace divided by 38 agrees with accuracy.
3. The baseline predicts only its learned majority class; explain what the comparison tests.
4. Record scikit-learn version, seed, parameters, and metrics; do not require exact accuracy across versions.

Record actual outputs beside predictions. Reduce a failure to a small example, change one cause at a time, and repeat the failed check after repairing it.

4. Change one condition

Try a deeper tree using a validation partition within the training set. Choose depth there, then evaluate once on the held-out test set. Explain why repeated test-driven choices would compromise the test.

5. Review a hint after attempting the task

Hint: connect the relevant tools

Fit only on the training partition and use accuracy_score and confusion_matrix. A baseline shows whether extra complexity improves the selected measure.

6. Explain and self-review

Keep partition counts, predictions, matrices, and an explanation of one error. Discuss why this small curated dataset cannot establish performance for new field conditions or justify using a transformer.

Review path: compare your result with the contract

Check the ordinary result first, then the boundary cases and changed condition. Explain which rule justifies each result. An implementation that differs from the hint can still satisfy the contract; explain why yours does.

{
  const scripts = Array.from(
    document.querySelectorAll("script.fcpython-ojs-quiz-config")
  );
  const script = scripts.find(
    (node) => node.dataset.fcpythonRendered !== "true"
  );

  if (!script) {
    return html`<div class="fcpython-quiz fcpython-quiz-warning">
      Quiz configuration was not found.
    </div>`;
  }

  script.dataset.fcpythonRendered = "true";
  const quiz = JSON.parse(script.textContent);
  const container = html`<div class="fcpython-quiz"></div>`;
  const title = document.createElement("h3");
  title.textContent = quiz.title;
  container.appendChild(title);

  const instructions = document.createElement("p");
  instructions.textContent = quiz.instructions;
  container.appendChild(instructions);

  const progress = document.createElement("div");
  progress.className = "fcpython-quiz-progress";

  const counter = document.createElement("p");
  counter.className = "fcpython-quiz-counter";
  counter.setAttribute("aria-live", "polite");
  progress.appendChild(counter);

  const tabs = document.createElement("div");
  tabs.className = "fcpython-quiz-steps";
  tabs.setAttribute("role", "tablist");
  tabs.setAttribute("aria-label", "Quiz questions");
  progress.appendChild(tabs);
  container.appendChild(progress);

  const questions = document.createElement("div");
  questions.className = "fcpython-quiz-questions";

  const feedbackNodes = [];
  const questionPanels = [];
  const stepButtons = [];
  let currentQuestion = 0;

  function setStepStatus(questionIndex, status) {
    const step = stepButtons[questionIndex];
    const question = quiz.questions[questionIndex];
    const statuses = {
      answered: "answered",
      correct: "correct",
      incorrect: "incorrect",
      unanswered: "not answered",
    };

    step.classList.remove(
      "is-answered",
      "is-correct",
      "is-incorrect",
      "is-unanswered"
    );
    if (status) {
      step.classList.add(`is-${status}`);
    }

    const statusLabel = status ? `, ${statuses[status]}` : "";
    step.setAttribute(
      "aria-label",
      `Question ${questionIndex + 1}: ${question.prompt}${statusLabel}`
    );
  }

  function showQuestion(questionIndex, focusPanel = false) {
    currentQuestion = Math.max(
      0,
      Math.min(questionIndex, quiz.questions.length - 1)
    );

    questionPanels.forEach((panel, index) => {
      panel.hidden = index !== currentQuestion;
    });
    stepButtons.forEach((step, index) => {
      const isCurrent = index === currentQuestion;
      step.classList.toggle("is-current", isCurrent);
      step.setAttribute("aria-selected", String(isCurrent));
      step.tabIndex = isCurrent ? 0 : -1;
    });

    counter.textContent = `Question ${currentQuestion + 1} of ${quiz.questions.length}`;

    const selected = questionPanels[currentQuestion].querySelector(
      "input[type='radio']:checked"
    );
    const isLastQuestion = currentQuestion === quiz.questions.length - 1;
    previous.hidden = currentQuestion === 0;
    next.hidden = isLastQuestion || !selected;
    check.hidden = !isLastQuestion;

    if (focusPanel) {
      questionPanels[currentQuestion].focus();
    }
  }

  quiz.questions.forEach((question, questionIndex) => {
    const tabId = `${quiz.id}-question-tab-${questionIndex + 1}`;
    const panelId = `${quiz.id}-question-panel-${questionIndex + 1}`;

    const step = document.createElement("button");
    step.type = "button";
    step.className = "fcpython-quiz-step";
    step.id = tabId;
    step.textContent = String(questionIndex + 1);
    step.setAttribute("role", "tab");
    step.setAttribute("aria-controls", panelId);
    step.setAttribute("aria-selected", "false");
    step.tabIndex = -1;
    step.addEventListener("click", () => showQuestion(questionIndex));
    step.addEventListener("keydown", (event) => {
      let destination = null;
      if (event.key === "ArrowRight") {
        destination = (questionIndex + 1) % quiz.questions.length;
      } else if (event.key === "ArrowLeft") {
        destination =
          (questionIndex - 1 + quiz.questions.length) % quiz.questions.length;
      } else if (event.key === "Home") {
        destination = 0;
      } else if (event.key === "End") {
        destination = quiz.questions.length - 1;
      }

      if (destination !== null) {
        event.preventDefault();
        showQuestion(destination);
        stepButtons[destination].focus();
      }
    });
    stepButtons.push(step);
    tabs.appendChild(step);
    setStepStatus(questionIndex, "");

    const panel = document.createElement("div");
    panel.className = "fcpython-quiz-panel";
    panel.id = panelId;
    panel.setAttribute("role", "tabpanel");
    panel.setAttribute("aria-labelledby", tabId);
    panel.tabIndex = -1;

    const fieldset = document.createElement("fieldset");
    fieldset.className = "fcpython-quiz-question";

    const legend = document.createElement("legend");
    legend.textContent = question.prompt;
    fieldset.appendChild(legend);

    question.options.forEach((option, optionIndex) => {
      const label = document.createElement("label");
      label.className = "fcpython-quiz-option";

      const input = document.createElement("input");
      input.type = "radio";
      input.name = `${quiz.id}-${question.id}`;
      input.value = String(optionIndex);
      input.addEventListener("change", () => {
        setStepStatus(questionIndex, "answered");
        feedbackNodes[questionIndex].textContent = "";
        feedbackNodes[questionIndex].className = "fcpython-quiz-feedback";
        score.textContent = "";
        showQuestion(questionIndex);
      });

      const text = document.createElement("span");
      text.textContent = option;

      label.appendChild(input);
      label.appendChild(text);
      fieldset.appendChild(label);
    });

    const feedback = document.createElement("p");
    feedback.className = "fcpython-quiz-feedback";
    feedback.setAttribute("aria-live", "polite");
    feedbackNodes.push(feedback);
    fieldset.appendChild(feedback);
    panel.appendChild(fieldset);
    questionPanels.push(panel);
    questions.appendChild(panel);
  });

  container.appendChild(questions);

  const actions = document.createElement("div");
  actions.className = "fcpython-quiz-actions";

  const previous = document.createElement("button");
  previous.type = "button";
  previous.className = "fcpython-quiz-secondary";
  previous.textContent = "Previous";
  previous.addEventListener("click", () => {
    showQuestion(currentQuestion - 1, true);
  });

  const next = document.createElement("button");
  next.type = "button";
  next.textContent = "Next question";
  next.addEventListener("click", () => {
    showQuestion(currentQuestion + 1, true);
  });

  const check = document.createElement("button");
  check.type = "button";
  check.textContent = "Check answers";

  const reset = document.createElement("button");
  reset.type = "button";
  reset.className = "fcpython-quiz-secondary fcpython-quiz-reset";
  reset.textContent = "Reset";

  const score = document.createElement("p");
  score.className = "fcpython-quiz-score";
  score.setAttribute("aria-live", "polite");

  check.addEventListener("click", () => {
    let correctCount = 0;
    let firstQuestionToReview = null;

    quiz.questions.forEach((question, questionIndex) => {
      const selected = container.querySelector(
        `input[name="${quiz.id}-${question.id}"]:checked`
      );
      const feedback = feedbackNodes[questionIndex];

      if (!selected) {
        feedback.textContent = "Choose an answer before checking.";
        feedback.className = "fcpython-quiz-feedback";
        setStepStatus(questionIndex, "unanswered");
        if (firstQuestionToReview === null) {
          firstQuestionToReview = questionIndex;
        }
        return;
      }

      const selectedIndex = Number(selected.value);
      if (selectedIndex === question.answer_index) {
        correctCount += 1;
        feedback.textContent = `✅ Correct. ${question.explanation}`;
        feedback.className = "fcpython-quiz-feedback is-correct";
        setStepStatus(questionIndex, "correct");
      } else {
        const answer = question.options[question.answer_index];
        feedback.textContent = `❌ Not yet. Correct answer: ${answer}. ${question.explanation}`;
        feedback.className = "fcpython-quiz-feedback is-incorrect";
        setStepStatus(questionIndex, "incorrect");
        if (firstQuestionToReview === null) {
          firstQuestionToReview = questionIndex;
        }
      }
    });

    score.textContent = `Score: ${correctCount}/${quiz.questions.length}`;
    if (firstQuestionToReview !== null) {
      showQuestion(firstQuestionToReview, true);
    }
  });

  reset.addEventListener("click", () => {
    container.querySelectorAll("input[type='radio']").forEach((input) => {
      input.checked = false;
    });
    feedbackNodes.forEach((feedback) => {
      feedback.textContent = "";
      feedback.className = "fcpython-quiz-feedback";
    });
    stepButtons.forEach((_, questionIndex) => {
      setStepStatus(questionIndex, "");
    });
    score.textContent = "";
    showQuestion(0, true);
  });

  actions.appendChild(previous);
  actions.appendChild(next);
  actions.appendChild(check);
  actions.appendChild(reset);
  container.appendChild(actions);
  container.appendChild(score);
  showQuestion(0);
  return container;
}

7. Save your work and record completion

Save the notebook or project somewhere you control. Include one decision, one failed check, its repair, and a result you can reproduce. Record completion after reviewing the checks; you can undo it later.

Not yet recorded.

Return to the unit overview.