Python Foundation for Data Engineers

Chapter 5 — Control Flow: Decisions & Loops

Lesson 5.1 — If, Elif, Else: Teaching Code to Decide

Every script you've written so far does the exact same thing, every single time you run it, top to bottom, no matter what. Real pipelines don't get to work that way. A file might be empty. A price might be negative. A row might be missing a required field. Your code needs to notice, and react differently depending on what it finds.

That's what an if statement gives you: the ability to make your code decide.

python
row_count = 0 if row_count == 0: print("Warning: file is empty")
Output / Note

Warning: file is empty

Read that like a sentence: "if row count equals zero, then print this warning." Notice the double equals sign, == — that's different from the single = you've been using to store values. A single = means "store this." A double == means "check if these are equal." Mixing the two up is one of the most common early mistakes in Python, so it's worth slowing down on this now, before it becomes a habit.

Real decisions usually have more than one branch. That's where elif — short for "else if" — and else come in:

python
row_count = 4500 if row_count == 0: print("Empty file — stop and investigate") elif row_count < 100: print("Unusually small file — double check") else: print("File looks normal")
Output / Note

File looks normal

Python checks each condition in order, top to bottom, and runs the first one that matches — then skips the rest entirely. If row_count had been 0, it would have printed the first message and never even looked at the others. else is your catch-all — "if none of the above conditions were true, do this instead."

You can also combine conditions using and and or:

python
price = -5 quantity = 3 if price < 0 or quantity < 0: print("Invalid row — negative value found")
Output / Note

Invalid row — negative value found

and means both conditions must be true. or means at least one of them needs to be. This single line — checking for negative values — is a genuinely common first line of defense in real data validation, and you'll write something very close to it again later in this course.

Try It Yourself: Create a variable discount_percent = 15. Write an if/elif/else that prints "No discount" if it's 0, "Standard discount" if it's between 1 and 20, and "Review required" if it's above 20.


Lesson 5.2 — For Loops: Doing Something for Every Item

You've already seen a for loop sneak into the last two chapters, before we'd properly covered it — that was on purpose. Now let's actually learn it, because it's one of the tools you'll use in nearly every single script you write from here on.

Here's the problem it solves: you have a list of things, and you want to do the same action to every single one of them, one at a time.

python
source_systems = ["CRM", "ERP", "Billing"] for system in source_systems: print(f"Connecting to {system}...")
Output / Note

Connecting to CRM... Connecting to ERP... Connecting to Billing...

Read it like a sentence: "for each system in source_systems, do this." Python walks through the list one item at a time, and on each pass, system holds whatever the current item is. You choose the name system — it's just a label for "whichever item we're currently looking at."

Loops become genuinely powerful once you combine them with the if statements from the last lesson:

python
row_counts = {"CRM": 4500, "ERP": 0, "Billing": 3200} for system, count in row_counts.items(): if count == 0: print(f"{system}: no data — needs investigation") else: print(f"{system}: {count} rows loaded")
Output / Note

CRM: 4500 rows loaded ERP: no data — needs investigation Billing: 3200 rows loaded

That .items() bit is how you loop through a dictionary and get both the key and the value at once, on every pass — you'll use this constantly once you're working with real batches of records. Notice, too, what just happened: three source systems checked, and one flagged automatically, without you having to look at each one by hand. That's the entire point of loops — doing a repetitive check reliably, across as many items as you have, without missing one out of tiredness or human error.

You can also build up a result as you loop, using a pattern you'll recognize from the comprehensions lesson:

python
prices = [24.99, 15.50, 8.25] total = 0 for price in prices: total = total + price print(f"Total: {total}")
Output / Note

Total: 48.74

Here, total starts at zero, and on every pass through the loop, it grows by adding the current price. This "start with an empty value, then build it up one item at a time" pattern shows up constantly in data work — totals, counts, collected error messages, all built this same way.

Try It Yourself: Create a list order_totals = [120.50, 0, 340.00, -15.00, 89.99]. Loop through it, and for each value, print "Valid order: $X" if it's greater than 0, or "Skipping invalid order" otherwise.


Lesson 5.3 — While Loops, Break & Continue

for loops are perfect when you know exactly what you're looping over — a list, a dictionary, a fixed batch of records. But sometimes you don't know in advance how many times you'll need to repeat something — you just know you want to keep going until something becomes true. That's what a while loop is for.

python
attempts = 0 connected = False while attempts < 3 and not connected: attempts += 1 print(f"Attempt {attempts}: trying to connect...") if attempts == 2: connected = True print("Connected!" if connected else "Failed after 3 attempts")
Output / Note

Attempt 1: trying to connect... Attempt 2: trying to connect... Connected!

Read the condition like a sentence: "keep looping while attempts is less than 3, and we're not connected yet." Every time through, attempts += 1 increases the count by one — that's shorthand for attempts = attempts + 1, and you'll see it everywhere. This exact shape — retry a few times, then give up — is genuinely how real connection logic works, and you'll build a proper version of it later in this course, when we start calling real APIs.

One real word of caution here, because it catches everyone at least once: if the condition inside a while loop never becomes false, your code will run forever, and you'll have to manually stop it. Always make sure something inside the loop is actually working toward ending it — like attempts climbing toward its limit above.

Two more small but genuinely useful tools work inside any loop, for or while: break and continue.

python
row_ids = [101, 102, 103, 999, 104, 105] for row_id in row_ids: if row_id == 999: print("Found the poison row — stopping early") break print(f"Processing {row_id}")
Output / Note

Processing 101 Processing 102 Processing 103 Found the poison row — stopping early

break exits the loop immediately, completely — nothing after it runs, even if there were more items left. Here, it let us stop the moment we hit a known bad value, instead of wasting time processing the rest.

continue is milder — it skips just the current item, and moves on to the next one:

python
row_ids = [101, -1, 102, -1, 103] for row_id in row_ids: if row_id < 0: continue print(f"Processing {row_id}")
Output / Note

Processing 101 Processing 102 Processing 103

Notice the invalid IDs never even got a print statement — continue quietly skipped straight past them and kept the loop going. That's a small, honest habit worth building: skip bad data on purpose, visibly, rather than letting it slip through unnoticed.

Try It Yourself: Given values = [45, -3, 22, 0, -8, 17], write a loop that prints each positive value, uses continue to skip anything less than or equal to 0, and uses break to stop entirely the moment it encounters the value 0.


Lesson 5.4 — Hands-On: Classify Pipeline Health

Let's put this whole chapter to work on something that looks a lot like a real, everyday task: checking the health of last night's pipeline runs.

Here's a batch of pipeline run results, exactly as messy and mixed as a real morning check tends to be:

python
pipeline_runs = [ {"name": "orders_sync", "status": "success", "rows": 4500}, {"name": "customer_sync", "status": "failed", "rows": 0}, {"name": "inventory_sync", "status": "success", "rows": 12}, {"name": "billing_sync", "status": "success", "rows": 3200}, ]

Notice inventory_sync succeeded, but only loaded 12 rows — that's suspicious for a sync that usually carries thousands. A "success" status alone doesn't always mean everything's actually fine, and that's a genuinely important lesson for real pipeline monitoring.

Your task, using if/elif/else and a for loop together:

  1. Loop through every pipeline run.
  2. Label each one "Failed" if the status is "failed".
  3. Label it "Warning" if it succeeded but loaded fewer than 100 rows.
  4. Otherwise, label it "Healthy".
  5. Print a clear summary line for each one.

Here's the shape to build from:

python
pipeline_runs = [ {"name": "orders_sync", "status": "success", "rows": 4500}, {"name": "customer_sync", "status": "failed", "rows": 0}, {"name": "inventory_sync", "status": "success", "rows": 12}, {"name": "billing_sync", "status": "success", "rows": 3200}, ] for run in pipeline_runs: if run["status"] == "failed": health = "Failed" elif run["rows"] < 100: health = "Warning" else: health = "Healthy" print(f"{run['name']}: {health} ({run['rows']} rows)")
Output / Note

orders_sync: Healthy (4500 rows) customer_sync: Failed (0 rows) inventory_sync: Warning (12 rows) billing_sync: Healthy (3200 rows)

Four pipelines checked, each one correctly classified — including the tricky one, inventory_sync, which "succeeded" but still deserved a warning. That distinction, catching a technically-successful run that's still probably broken, is exactly the kind of judgment call real monitoring scripts are built to make.

One more useful step — counting how many pipelines need attention, using the running-total pattern from Lesson 5.2:

python
failed_or_warning = 0 for run in pipeline_runs: if run["status"] == "failed" or run["rows"] < 100: failed_or_warning += 1 print(f"{failed_or_warning} pipeline(s) need attention this morning")
Output / Note

2 pipeline(s) need attention this morning

You've just built the exact core logic behind a real morning pipeline-health check — the kind of thing that, later in this course, will run automatically every day instead of you checking it by hand.

Try It Yourself: Add a fifth pipeline run to the list, with "status": "success" and "rows": 0. Run the classification loop again, and think about why this one is arguably the most dangerous case of all — a pipeline that reports success while loading nothing. Adjust the elif condition so this case is also correctly caught as a "Warning".