Capstone Project — Reference Solution

About this document

This document is fully self-contained. It covers everything you need to load the reference solution, run it, verify every part of it actually works, and use it properly while you build your own version in 03-do-it-yourself.md. pp4de-solution.zip is the only other file you need.

Part 1: Load it and verify it, step by step

1. Extract


bash
cd ~/courses
cp /mnt/c/Users/yourname/Downloads/pp4de-solution.zip .
unzip pp4de-solution.zip
code pp4de-solution

Extract this alongside — not inside — your own pp4de folder. They're independent, and won't conflict.

2. Set up your `.env`


bash
cd ~/courses/pp4de-solution
cp .env.example .env

Open .env and set POSTGRES_PASSWORD to anything — this is local only.

3. Start the infrastructure


bash
docker compose start
docker compose ps

Checkpoint: both pp4de_solution_postgres and pp4de_solution_mock_api should show healthy. This uses different default ports from your own pp4de folder — Postgres on 5433, the mock API on 8001 — specifically so both can run at once if you ever want to compare them side by side.

4. Set up the Python environment


bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e .
pip install mypy ruff pytest pytest-asyncio hypothesis

5. Run the three checks — the real verification


bash
ruff check .

Checkpoint: All checks passed!


bash
mypy --strict src/pipeline/ tests/

Checkpoint: Success: no issues found in 16 source files


bash
pytest tests/ -v

Checkpoint: 12 passed, 1 skipped — the one skip is the Postgres integration test (see step 7), not a failure.

If all three checkpoints match, the reference solution is genuinely working, not just installed.

6. Run the real pipeline


bash
pipeline ingest --date 2026-07-01

Checkpoint: structured JSON log lines, ending in a summary like Ingested 5xx records, ~1x dead-lettered. Run ID: <a uuid>. The exact numbers vary slightly run to run — the mock API deliberately corrupts a small percentage of records every time.

Now run the exact same command again — the real proof of idempotency:


bash
pipeline ingest --date 2026-07-01


bash
docker exec -it pp4de_solution_postgres psql -U pipeline_user -d warehouse -c \
  "SELECT external_id, COUNT(*) FROM warehouse.records GROUP BY external_id HAVING COUNT(*) > 1;"

Checkpoint: no rows returned. That's the pipeline proving it's safe to re-run, not just claiming to be.

7. Run the live-Postgres integration test

Skipped by default in step 5, since it needs the database directly:


bash
POSTGRES_PASSWORD=<whatever you set in .env> pytest tests/test_writer_integration.py -v

Checkpoint: 1 passed.

8. Shut down when you're done


bash
docker compose stop

If every checkpoint above matched, the reference solution is fully functional — not a claim, something you just watched happen.

Part 2: How to actually use it while you build your own

Get it running first, before you write anything yourself

Do everything in Part 1 once, before you start 03-do-it-yourself.md. Running the finished system, and watching every checkpoint genuinely pass, gives you a real, working target to build toward — not just a written spec to interpret.

Compare at the end of each phase, not just at the end of the whole project

The old approach — read a reference solution once, at the very end — is exactly what this new structure is trying to fix. Used that way, a reference solution either gets ignored until the last minute, or gets copied instead of understood. Use it differently: 03-do-it-yourself.md is broken into phases, and each one ends with a specific comparison step — which file to look at, and what to actually check for. Don't skip ahead and look at a later phase's file before you've built the corresponding part yourself.

Compare decisions, not syntax

Your retry.py should not look identical to the reference solution's, and that's fine — there's more than one reasonable way to write a classified retry helper. What's worth comparing: did you also separate retryable from non-retryable exceptions explicitly? Did you also bind a run_id once instead of threading it through every function call? Compare the shape of the decision, not the exact code.

If something differs and you're not sure why, that's the valuable moment

Don't just silently adopt the difference. Figure out whether you found a genuinely different, equally valid approach, or whether the reference solution caught something yours missed. Both outcomes teach you something. Neither does if you don't stop and ask the question.

One honest limitation, worth knowing before you look

AI_GUARDRAIL_LOG.md inside the reference solution is intentionally blank. Every real entry in your log, from your own build, comes from an actual AI-assisted session you ran yourself — the reference solution's code was assembled directly, not through real AI-assisted sessions the way yours will be. Don't take the blank log as a model; your own log, full of real entries, is the actual artifact that matters.

Once every checkpoint in Part 1 has matched, start 03-do-it-yourself.md and begin your own build.