Production Python for Data Engineers

Setting Up Your Environment

Before we write a single line of Python in this course, we need to get your laptop ready. Think of this like setting up your kitchen before you cook. It takes a little time now, but it saves you from a lot of frustration later.

Two things need to be installed:

  • Docker — this runs the mock API and the Postgres database you'll use all through the course
  • VS Code — this is where you'll write and run your code

Don't worry if none of this is installed yet. We'll assume you're starting from zero, and walk through every step.

Already have some of this installed? That's fine too — just jump to the verification checklist for your operating system, and confirm it actually works before moving on. "I think it's installed" and "I confirmed it works" are two very different things, and this course only cares about the second one.

Most learners on this course use Windows, so that path is written in full detail below. A shorter Mac path follows after it. If you're already on Linux, you probably know how to install Docker and an editor — feel free to skip ahead to the verification checklist.


Part A: Windows setup

Here's a question you might be asking: why can't we just install Docker directly on Windows, the normal way?

The honest answer — Docker was originally built for Linux. On Windows, it needs a small piece of Linux running underneath it to work properly. That piece is called WSL2 (Windows Subsystem for Linux). It sounds like extra complexity, but it's actually the standard, officially supported way to run Docker on Windows.

There's a bonus too. WSL2 gives you a real Linux terminal, right there on your Windows machine. This matters more than you might think — later in the course, we'll be running Linux-style commands, and having a real Linux terminal means you won't be fighting strange Windows-only bugs while you're trying to learn something else entirely.

A1. Enable WSL2

  1. Open PowerShell as Administrator. (Right-click the Start button → "Terminal (Admin)", or search "PowerShell", right-click it, and choose "Run as administrator".)

  2. Run this command:

    powershell
    wsl --install

    In theory, this one command does three things at once: turns on the Windows features WSL needs, installs WSL2 itself, and installs Ubuntu (a friendly, beginner-common flavor of Linux) as your default Linux system.

  3. Restart your computer when it asks you to. This step is not optional. Windows won't actually turn on the features WSL needs until after a reboot.

  4. After restarting, check whether Ubuntu installed and opened on its own.

    Here's something worth knowing before you worry unnecessarily: this step doesn't always work perfectly in one go. On some machines, the command above turns on the right Windows features, but doesn't actually finish installing Ubuntu. Nothing pops up. No error. Just... nothing.

    So don't assume something is broken just because no window appeared. Instead, let's check directly. Open PowerShell again (regular, not admin this time) and run:

    powershell
    wsl -l -v
    • See Ubuntu in the list? Good — it installed correctly. Now search "Ubuntu" in the Start menu and open it yourself, rather than waiting for it to open on its own.
    • See a message like "Windows Subsystem for Linux has no installed distributions"? That means Ubuntu didn't get installed. No problem — just install it directly:
      powershell
      wsl --install -d Ubuntu
      Let this finish completely — it's downloading and installing an entire small Linux system, so it can take a few minutes. Then search "Ubuntu" in the Start menu and open it.
  5. You might see a separate window pop up called "Welcome to WSL." This is just a Microsoft information panel about WSL's features. It is not your terminal. You can close it and ignore it completely.

  6. The first time Ubuntu actually opens — a real terminal window, not the "Welcome to WSL" panel — it will ask you to create a Linux username and a password. Keep it simple; you won't need to type it often. One quirk worth knowing in advance: when you type the password, nothing appears on screen — no dots, no stars, nothing. That's completely normal. It's not broken. Just type it and press Enter.

  7. Let's confirm you landed in the right place. Run:

    bash
    pwd

    You should see /home/yourname. This is your Linux home folder, and from now on, it's where all your course work will live.

A2. Double-check WSL2 is really working

Open PowerShell again (regular, not admin) and run:

powershell
wsl --status

Look for Default Version: 2. If you instead see version 1, fix it with:

powershell
wsl --set-default-version 2

A3. Install Docker Desktop

  1. Go to docker.com and download Docker Desktop for Windows. (Search "Docker Desktop download" and get it straight from Docker's own site — not a random mirror.)
  2. Run the installer.
    • It might ask you to choose between "Install for me only (per-user)" and "Install for all users." Either works fine for this course. If you're unsure, go with per-user — it's simpler, and you won't need admin rights for future updates.
    • Older versions of the installer show a checkbox for "Use WSL 2 instead of Hyper-V." If you see it, check it. Newer versions don't show this option at all — they just use WSL2 automatically behind the scenes. If you don't see the checkbox, that's completely normal. There's nothing missing, and nothing for you to configure.
  3. Restart if the installer asks you to.
  4. Launch Docker Desktop. On first launch, it might ask you to install a small WSL2 kernel update. If it does, click through, install it, and relaunch Docker Desktop. If this prompt never shows up, that's fine too — it only appears when an update is actually needed, and on newer Windows setups, it often isn't.
  5. Inside Docker Desktop, go to Settings → Resources → WSL Integration, and make sure integration is switched on for your Ubuntu distribution.

A4. Prove Docker actually works

Open your Ubuntu terminal (search "Ubuntu" in the Start menu — this is your Linux environment, and from here on, it's where you'll run every course command, not PowerShell). Then run:

bash
docker --version docker compose version docker run hello-world

That last command downloads a tiny test image and should print a message starting with "Hello from Docker!" If you see that message, Docker is genuinely working — not just installed, but working.

A5. If something went wrong

Don't panic — almost every Windows setup issue falls into one of a small number of known patterns. Before asking anyone for help, check the Troubleshooting section near the end of this document. There's a good chance your exact problem is already answered there.


Part B: Mac setup

  1. First, check which chip your Mac has. Click the Apple menu → About This Mac. Note whether it says Apple chip (M1, M2, M3, M4) or Intel. You'll need to know this in the next step.
  2. Download Docker Desktop for Mac from docker.com, making sure to pick the build that matches your chip (Apple Silicon or Intel).
  3. Open the downloaded .dmg file, drag Docker into your Applications folder, then launch it from there. Grant whatever permissions it asks for — these are normal and expected.
  4. Open Terminal (Applications → Utilities → Terminal) and confirm everything works:
    bash
    docker --version docker compose version docker run hello-world
    You're looking for the exact same result described in the Windows verification step above — a "Hello from Docker!" message.

Part C: Install VS Code

Why VS Code, out of all the editors out there?

A few honest reasons: it's free, it works the same way on Windows, Mac, and Linux, and it has the best Python and Docker support of any free editor available. For Windows users especially, VS Code has one more trick — it can connect directly into your Linux (WSL2) environment, so you edit files as if you were sitting inside Linux itself. This quietly avoids a whole category of annoying path and line-ending bugs you'd otherwise run into later in the course.

  1. Download VS Code from code.visualstudio.com and install it (a normal installer on Windows, drag-to-Applications on Mac).
  2. Open VS Code and install these extensions. Click the Extensions icon in the left sidebar, search for each one by name, and click Install:
    • Python (by Microsoft)
    • Pylance (by Microsoft — this usually installs itself alongside Python)
    • Docker (by Microsoft)
    • Windows only: WSL (by Microsoft) — this is what lets VS Code reach into your Ubuntu environment directly
  3. Windows only: open your Ubuntu terminal, navigate to wherever you'll keep the course files, and run:
    bash
    code .
    The first time you do this, VS Code quietly installs a small helper component inside WSL — this takes about a minute. Once the window opens, glance at the bottom-left corner. It should say "WSL: Ubuntu." That's your confirmation that you're editing inside Linux, not Windows. From this point on, always open the course folder this exact way — code . from your Ubuntu terminal — rather than launching VS Code from the Windows Start menu.
  4. Mac users: just open the course folder normally, either with code . from Terminal, or via File → Open Folder.

Don't worry about installing every extension right now. We'll introduce a few more later, exactly when a module actually needs them — for example, a test runner extension when we reach Module 5.


Part D: Bring your course environment to life

You'll receive the starter files as a zip file, not as a link to clone. You'll extract it, and then turn it into your own GitHub repository. From that point forward, this repository is where your capstone project actually lives — growing module by module, all the way through the course.

D1. Open the right terminal

  • Windows: open the Ubuntu app from the Start menu. Not PowerShell. Not Command Prompt.
  • Mac: open Terminal.

D2. Make sure you're inside Linux, not Windows (Windows only)

bash
pwd

If you see /home/yourname, you're in the right place. If instead you see something like /mnt/c/Users/yourname, step back out first:

bash
cd ~ pwd

Why does this matter so much? Anything living under /mnt/c/... from inside WSL is really sitting on your Windows hard drive, not your Linux one. And Docker running against files on the Windows side is, by far, the single biggest cause of slow or flaky behavior on WSL2. Keep everything under /home/yourname/... from here on, and you'll avoid this entirely.

D3. Make a folder for your course work

bash
mkdir -p ~/courses cd ~/courses

(Mac users: same commands — there's no filesystem trap to worry about here. Just use whatever folder you normally keep projects in.)

D4. Extract the zip file

One small thing to check first — fresh Ubuntu installs don't come with the unzip tool by default. Let's check:

bash
unzip -v

If you get a "command not found" message, install it like this:

bash
sudo apt update sudo apt install -y unzip

(This will ask for the Linux password you created back in Part A1.)

Now, about the zip file itself. It probably landed in your regular Windows Downloads folder, even though you're about to work with it from inside Linux. So let's copy it across first, and extract it from the Linux side — not the Windows side:

bash
cp /mnt/c/Users/yourname/Downloads/pp4de.zip ~/courses/ cd ~/courses unzip pp4de.zip cd pp4de

(Mac users: skip the /mnt/c/... copy step — just move the zip into your projects folder using Finder, or mv, then run unzip as shown above.)

Let's confirm you're standing in the right place:

bash
pwd

You're looking for something like /home/yourname/courses/pp4de — and definitely not a path starting with /mnt/c/....

D5. Turn this into your own GitHub repository

Here's something worth sitting with for a second: this project is yours for the rest of the course. Every module's work gets committed here, piece by piece. You are not cloning someone else's course repo — you're creating your own, starting from these files.

  1. If you don't already have one, create a free account at github.com.

  2. On GitHub, click New repository. Name it production-python, and set its visibility to Private — this means only you can see it (you can always invite others later if you want to). Leave it completely empty. Don't let GitHub add a README, .gitignore, or license for you — you already have files locally, and that would create a conflict. Once it's created, keep the page open; you'll need the URL it shows you, something like https://github.com/yourusername/production-python.git.

  3. Back in your terminal, check whether git is already installed:

    bash
    git --version

    If it says "command not found" (fairly common on a fresh Ubuntu install), install it:

    bash
    sudo apt update sudo apt install -y git

    (Same Linux password as before, if it asks.)

  4. Git needs to know who you are before it lets you save any work. This is a one-time setup for this machine, not something you'll repeat for every repo:

    bash
    git config --global user.email "your-github-email@example.com" git config --global user.name "Your Name"

    Use the same email your GitHub account is registered with. The name is really just a label attached to your commits — git doesn't check it against GitHub at all. But since this repository is something you may end up showing an interviewer one day, use your real name, not a placeholder.

  5. Now, from inside ~/courses/pp4de, let's turn these extracted files into a real git repository, and send them up to the one you just created on GitHub:

    bash
    git init git add . git commit -m "Initial capstone starter files" git branch -M main git remote add origin https://github.com/yourusername/production-python.git git push -u origin main

    Make sure to swap in the exact URL GitHub gave you in step 2.

  6. At this point, GitHub will ask for a username and a password. This is exactly where most people get stuck, so let's slow down here.

    Here's the surprising part: GitHub no longer accepts your real account password for this, even though the prompt still says "Password." Instead, you need something called a Personal Access Token (PAT) — think of it as a special, limited password made just for this purpose.

    Here's how to get one:

    • In a browser, go to https://github.com/settings/tokens
    • Click Generate new tokenGenerate new token (classic)
    • Give it a short note so you remember what it's for (something like "production-python course laptop"), choose an expiration (90 days comfortably covers the whole course), and check the repo box (this is required since your repository is private)
    • Click Generate token, and copy it immediately — GitHub only ever shows it to you once
    • Back in your terminal: type your GitHub username when it asks for "Username," and paste the token when it asks for "Password"
    • To avoid typing this token in every single time you push, run:
      bash
      git config --global credential.helper store
      and push one more time — git will remember the token from then on. (Worth knowing: this saves the token in plain text on your machine. Perfectly fine on your own personal laptop, but avoid this setting on a shared or public computer.)
  7. Refresh the repository page on GitHub, and check — your files should all be there.

From here on, every time you finish a module, you'll commit and push your progress to this same repository. By the end of the course, it becomes the one project you point to in interviews.

D6. Open the project in VS Code

bash
code .

(Windows users: double-check the bottom-left corner of VS Code says "WSL: Ubuntu." See Part C if you're unsure why this matters.)

D7. Start the infrastructure

In the VS Code terminal (this should default to your Linux or Mac shell automatically):

bash
cp .env.example .env docker compose up -d docker compose ps

You're looking for both services to show up as healthy.

D8. Confirm it's actually working

bash
curl http://localhost:8000/health

You should see: {"status":"ok"}

If all of this worked, congratulations — your environment, and your own GitHub repository, are both ready. Module 0 is waiting for you.

Output / Note

One quick note about .env and git: the .env file is deliberately listed inside .gitignore, which means it will never get pushed to your GitHub repo — even though .env.example will. Never remove .env from .gitignore, and never force it in. This isn't just a rule for this course — it's a real habit that protects real secrets in real jobs. (You'll learn exactly why in Module 8.)


Part E: Shutting down and starting up again

You'll only go through the full setup above once. But you'll stop and restart your environment nearly every day of this course — so it's worth learning the right rhythm now, rather than guessing later.

Shutting down for the day

From your terminal, inside the project folder, run:

bash
docker compose stop

This stops both containers cleanly, without deleting anything. Your Postgres data stays exactly as it was — which matters a lot once you reach the idempotency and resumability labs later in the course.

One important warning: don't run docker compose down -v out of habit. That little -v flag deletes your entire database volume, wiping out all your warehouse data along with it. stop is what you want for an everyday pause. down -v is something else entirely.

If you'd like to free up some memory overnight, you can also:

  • Quit Docker Desktop (its icon in the system tray or menu bar → Quit)
  • Windows only: run wsl --shutdown in PowerShell to fully stop WSL2

Closing VS Code itself needs no special steps — just close the window.

Starting back up the next day

  1. Launch Docker Desktop again (if you quit it), and wait until it shows "Running."
  2. Open your terminal and head back to the project folder:
    bash
    cd ~/courses/pp4de
  3. Bring the containers back up:
    bash
    docker compose start docker compose ps
    Notice we used start here, not up -d. up is really meant for the first time you create something, or after you've changed configuration. start simply resumes what's already there, and it's faster. (up -d would still work here too, and won't break anything — start is just the more efficient habit.)
  4. Reopen the project in VS Code:
    bash
    code .
  5. One quick sanity check:
    bash
    curl http://localhost:8000/health

Final verification checklist

Run through this list no matter which operating system you're on — from this point forward, everything should look and behave identically:

  • docker --version prints a version number
  • docker compose version prints a version number
  • docker run hello-world succeeds
  • VS Code opens the course folder (Windows: bottom-left says "WSL: Ubuntu")
  • docker compose up -d brings up two healthy services
  • curl http://localhost:8000/health returns {"status":"ok"}

Troubleshooting

Think of this section as a friend who's already been through the same setup problems you're about to hit. Most Windows/Docker setup issues fall into one of the patterns below — check here before you go searching the internet.

"Virtualization is not enabled" / WSL install fails right at the start Somewhere in your PC's BIOS or UEFI settings, virtualization (called Intel VT-x or AMD-V, depending on your processor) is switched off. Restart your computer, enter BIOS setup (usually by pressing Del, F2, or F10 during startup — this varies by manufacturer), find a setting called something like "Virtualization Technology," "Intel VT-x," or "SVM Mode," turn it on, save, and reboot.

wsl --install finished, you restarted, but nothing happened — no Ubuntu window appeared This is genuinely common, and doesn't mean anything is broken. Run wsl -l -v in PowerShell to check whether Ubuntu actually installed. If the list comes back empty, run wsl --install -d Ubuntu directly, then open "Ubuntu" from the Start menu yourself instead of waiting for it to launch on its own. Full details are in Part A1 above.

A "Welcome to WSL" window opened instead of a terminal That's just a Microsoft information panel — not your terminal. Close it, and open your actual terminal by searching "Ubuntu" in the Start menu.

git commit fails with "Author identity unknown" / "Please tell me who you are" Git just doesn't know who you are yet — a quick, one-time fix per machine:

bash
git config --global user.email "your-github-email@example.com" git config --global user.name "Your Name"

Then run the commit again. Full context is in Part D5.

git push fails with "Invalid username or token. Password authentication is not supported" This is expected, not a sign of a mistake. GitHub stopped accepting plain account passwords for git operations some years ago, even though the prompt still says "Password." You need a Personal Access Token instead — the full walkthrough for generating and using one is in Part D5, step 6.

"WSL 2 requires an update to its kernel component" Docker Desktop shows a link right when this happens — click it, download and run the small kernel update installer, then relaunch Docker Desktop. This is a one-time fix, and you won't see it again after.

Docker Desktop won't start, or just spins forever on launch Fully quit Docker Desktop (right-click its system tray icon → Quit), then restart your computer once, then launch Docker Desktop again. Still stuck? Check Settings → General, and try toggling "Use the WSL 2 based engine" off and back on.

"Port is already allocated" when you run docker compose up Something else on your machine is already using port 8000 or 5432. You can either close whatever that is, or simply change MOCK_API_PORT / POSTGRES_PORT in your .env file to a free port (for example, 8001 or 5433), and run it again.

Permission errors on volume mounts, or files that mysteriously belong to "root" This almost always means your repo is sitting on the Windows filesystem (something like /mnt/c/Users/... from inside WSL) instead of the Linux one (~/...). Move the repo into your Linux home directory, and try again.

Your laptop is a managed/corporate machine, and won't let you enable virtualization or install Docker Desktop Some work laptops block BIOS changes or software installs through IT policy. If this is you, it's better to flag it early than to lose a whole week discovering it mid-course. Your options include using a personal machine for this course, or setting up a cloud-based development environment — but this is a decision worth making deliberately, not one you want to stumble into during Module 3.

docker compose ps shows a service as unhealthy, and it's staying that way Before anything else, check its logs:

bash
docker compose logs mock-api docker compose logs postgres

The actual reason is almost always sitting right there in the last 20 lines.

Your VS Code terminal shows a Windows-style path (C:\Users\...) instead of a Linux one (/home/...) This means VS Code was opened the normal Windows way, instead of through code . from your Ubuntu terminal. Close the window, reopen it correctly (see Part C, step 3), and confirm "WSL: Ubuntu" appears in the bottom-left corner.

If you run into something not listed here, take one extra second before searching for it: copy down the exact error message first. Most Docker/WSL2 problems are common enough that the literal error text is the fastest way to find your answer.