FIELD NOTES ISSUE 01
JOINs
without
surprises.
A visual companion for seeing row matches, missing keys, and fanout before your totals betray you.
CASE FILE 01 KEY CARDINALITY
A duplicate lookup key
changed the result.
A duplicated customer key made two of four orders match twice, producing six input rows for SUM().
Predict the row pairs before you touch the aggregate.
02 RELATIONS BETWEEN STAGES
A query builds a new row set.
SQL does not save and update one value at a time like a Python loop. Each logical stage accepts a relation and produces another relation.
HAVINGform groups, then filter them
DISTINCTproject the final row shape
total = total + order.amountOne mutable value changes over time.
SELECT customer_id, SUM(total)
FROM joined_rows
GROUP BY customer_id;Define the input relation and output grain before applying the aggregate.
03 BEFORE THE JOIN
Read the key constraints
before the JOIN.
A primary key identifies one row. A foreign key points at that identity. The foreign key may repeat because one order can own many fills.
orders
order_idPKunique · not nullcreated_atone order factfills
fill_idPKone fill identityorder_idFKmay repeatPrimary key
orders.order_id is the identity of an order. It cannot be NULL or duplicated.
Foreign key
fills.order_id says which order a fill belongs to. Several fills may point to the same order.
Join condition
o.order_id = f.order_id compares values. It does not require both columns to be primary keys.
04 THE PAIRING PROMISE
A JOIN produces one row
for each matching pair.
For every row on one side, SQL finds all rows on the other side that satisfy the ON condition. Each match becomes one output row.
SELECT o.id, c.name, o.total FROM orders o JOIN customers c ON c.id = o.customer_id;
orders
| id | customer_id | total |
|---|
customers
| id | name |
|---|
Order 101 carries customer_id = 1.
The lookup contains exactly one id = 1.
One match means one joined row. The database repeats this search for every order.
Four matches become four rows
| order_id | name | total |
|---|
05 CHOOSE THE PRESERVED SIDE
JOIN type decides which non-matches survive.
Skip the Venn diagram. Ask which row pairs qualify, then decide which unmatched input rows must still appear.
Matched pairs only
Unmatched rows disappear from both inputs.
Use when the relationship must exist.Preserve the left
Every left row survives. Missing right values become synthetic NULLs.
Use for “all customers, even zero orders.”Preserve both
Keep unmatched rows from each input after the matched pairs.
Use for reconciliation.Every possible pair
m rows × n rows = m × n outputs.
06 THE MISSING MATCH
LEFT JOIN keeps the left row alive.
When no fill matches, SQL still emits the order. Every requested column from the missing right side appears as NULL in the result.
SELECT o.order_id, f.fill_id FROM orders o LEFT JOIN fills f ON f.order_id = o.order_id;two matching fillszero matching fillsThe NULL is a placeholder in the result
| order_id | fill_id | what happened |
|---|---|---|
| 101 | F1 | first match |
| 101 | F2 | second match |
| 102 | NULL | left row preserved |
07 THE SAME CONDITION, A DIFFERENT QUESTION
ON chooses matches.
WHERE chooses survivors.
A right-side filter inside ON still preserves the left row. Move it to WHERE, and the synthetic NULL row can fail the filter.
LEFT JOIN orders o
ON o.customer_id = c.id
AND o.created_at >=
DATE '2026-07-01'The date is part of the matching rule. Customers without a recent order remain.
LEFT JOIN orders o
ON o.customer_id = c.id
WHERE o.created_at >=
DATE '2026-07-01'The output is filtered after NULL padding. Missing orders cannot satisfy the date predicate.
08 COUNT THE RIGHT THING
COUNT(*) sees output rows.
COUNT(f.id) sees matches.
After a LEFT JOIN, the unmatched customer still owns one result row. COUNT(*) counts it; COUNT(f.id) ignores its NULL right-side value.
09 PROVING ABSENCE
Use row identity
to prove absence.
For stale orders with no fills at all, test a non-null match key or write the intent directly with NOT EXISTS.
LEFT JOIN anti-join
LEFT JOIN fills f
ON f.order_id = o.order_id
WHERE f.order_id IS NULLThe joined key can only be NULL when no matching fill row survived.
Correlated NOT EXISTS
WHERE NOT EXISTS (
SELECT 1 FROM fills f
WHERE f.order_id = o.order_id
)Reads as the business rule: retain the order when no related fill exists.
Nullable attribute
LEFT JOIN fills f
ON f.order_id = o.order_id
WHERE f.filled_at IS NULLAlso accepts a real fill row whose timestamp happens to be NULL.
CASE FILE 02 · PROVE ABSENCE10 THE FANOUT INCIDENT
Duplicate lookup keys
multiply matching pairs.
Four facts
| id | customer_id | total |
|---|
Lookup key grain
| id | name | state |
|---|
One matching customer row per key keeps the result at the order grain.
One result row per order
| order_id | name | total | matched customer row |
|---|
11 SYMPTOM HIDING
DISTINCT edits the final projection.
It can collapse identical display rows. It cannot undo a total already multiplied by the join.
-- DISTINCT can collapse identical projected customer rows.
SELECT DISTINCT c.id, c.name
FROM orders o
JOIN customers c ON c.id = o.customer_id;
-- It cannot repair an aggregate fed multiplied rows.
SELECT SUM(o.total) AS total
FROM orders o
JOIN customers c ON c.id = o.customer_id;
12 REPAIR THE GRAIN
Repair the relationship at its earliest valid stage.
Choose among a uniqueness constraint, an explicit survivor rule, or pre-aggregation according to the data contract.
Reject the duplicate
PRIMARY KEY (customer_id)Correct when the lookup promises one row per customer.
Choose a survivor
QUALIFY ROW_NUMBER()
OVER (...) = 1Correct only when the survivor rule is explicit and auditable.
Match the target grain
GROUP BY order_idReduce many fill rows to one order summary before the order-level join.
many / order→fill totals
one / order→orders joined
one / order
13 PENCIL DOWN · QUERY UP
Ten forecasts before the answer key.
Define one result row and estimate the row count before writing SQL.
Grain contract
State what one row means in accounts and invoices.
Predict: which key may repeat?
Duplicated lookup
Invoices are 10→125, 20→80, 10→45, 30→300, 50→60. The lookup contains account 10 twice and no account 50.
Predict: joined rows and SUM(total).
Choose the JOIN
Match: only accounts with invoices; every account; dates present in either cash table; every account × date cell.
Predict: INNER, LEFT, FULL, or CROSS.
The filter moved
Return members with no login since July 1.
Predict: ON or WHERE for the date?
Aggregate first
Find overfilled orders without multiplying order quantities.
Predict: the grain of fill_totals.
Count the missing
An account has no invoice match after a LEFT JOIN.
Predict: COUNT(*), COUNT(invoice_id), and their difference.
13B ONE MORE CORE · THREE STRETCH
Exists, never refunded
Paid at least once and refunded never.
Predict: one positive and one negative test.
Role + compound match
Join assets twice, then match deposits and trades by user plus date.
Predict: why both predicates are required, though neither guarantees one-to-one matching.
Expected rows
Build a date grid, then find the missing candles.
Predict: CROSS first, anti-join second.
Reconcile and classify
FULL OUTER daily cash flow plus half-open fee tiers.
Predict: the boundary at exactly 100.
14 DETACHABLE DESK REFERENCE
The eight-question JOIN preflight.
Run this before every production join. If one answer is vague, the query is not ready for an aggregate.
- 01What does one row represent in each input?
- 02Which key should be unique?
- 03How many right rows can match each left row?
- 04Which non-matches must survive?
- 05Is the right-side filter part of the match or a final filter?
- 06What is the output grain after the JOIN?
- 07Did aggregation happen before a many-sided JOIN?
- 08Can a duplicate-key probe prove the result?
AFTER CLASS COLLECTIBLE PLATES
Five field-note keepsakes.
The educational guide stays broadly shareable. These mild, non-spoiler plates are optional after-class material.
01 · CorrectOpen full-resolution plate
02 · TurnOpen full-resolution plate
03 · First bossOpen full-resolution plate
04 · Last callOpen full-resolution plate
05 · Faculty roomOpen full-resolution plate
APPENDIX A SHARP-PENCIL REVIEW
Answers, with the reason each query holds.
Several SQL shapes can be correct. The invariant is what matters: output grain, match key, and treatment of missing rows.
01 State the grain, then count matches
SELECT a.account_id,
COUNT(i.invoice_id) AS invoice_matches
FROM accounts a
LEFT JOIN invoices i ON i.account_id = a.account_id
GROUP BY a.account_id
ORDER BY a.account_id;accounts has one row per account and invoices has one per invoice. The GROUP BY sets the result grain to one row per account. Counting the non-null invoice key preserves Dee with zero matches.
02 Predict the fanout, then prove its cause
SELECT COUNT(*) AS joined_rows,
SUM(i.total) AS joined_total
FROM invoices i
JOIN account_import a
ON a.account_id = i.account_id;SELECT account_id, COUNT(*) AS rows_per_key
FROM account_import
GROUP BY account_id
HAVING COUNT(*) > 1;The repeated account_id = 10 pairs both Ari invoices with two lookup rows. Four matching invoices become six joined rows and the joined total rises from 550 to 720. The second query identifies the duplicated lookup key.
03 Choose what survives
-- Matching account/invoice pairs only: 4 rows
SELECT COUNT(*)
FROM accounts a JOIN invoices i
ON i.account_id = a.account_id;
-- Keep every account: 5 rows
SELECT COUNT(*)
FROM accounts a LEFT JOIN invoices i
ON i.account_id = a.account_id;-- Build a deliberate 2 x 2 grid: 4 rows
SELECT *
FROM (VALUES (DATE '2026-01-10'), (DATE '2026-01-11')) d(flow_date)
CROSS JOIN (VALUES (10), (20)) a(account_id);INNER keeps matches. LEFT preserves the account side. CROSS creates every possible pair. FULL OUTER is reserved for reconciliation when unmatched rows from either input must survive.
04 The filter moved
SELECT m.member_id, m.name
FROM members m
LEFT JOIN logins l
ON l.member_id = m.member_id
AND l.logged_at >= DATE '2026-07-01'
WHERE l.login_id IS NULL;The date belongs in ON because it defines a recent-login match. Testing the non-null login key proves that no qualifying login row exists.
05 Aggregate before joining
WITH fill_totals AS (
SELECT order_id, SUM(quantity) AS filled_quantity
FROM fills
GROUP BY order_id
)
SELECT o.order_id, o.quantity, f.filled_quantity
FROM orders o
JOIN fill_totals f ON f.order_id = o.order_id
WHERE f.filled_quantity > o.quantity;fill_totals is one row per order. The later join therefore cannot repeat an order-level quantity once per raw fill.
06 Count the missing
SELECT a.account_id,
COUNT(*) AS joined_rows,
COUNT(i.invoice_id) AS matched_invoices,
COUNT(*) - COUNT(i.invoice_id) AS missing_invoices
FROM accounts a
LEFT JOIN invoices i ON i.account_id = a.account_id
GROUP BY a.account_id;The unmatched account owns one output row, zero non-null invoice keys, and therefore one missing relationship.
07 Paid at least once, refunded never
SELECT u.user_id, u.name
FROM users u
WHERE EXISTS (
SELECT 1 FROM payments p
WHERE p.user_id = u.user_id AND p.status = 'completed'
)
AND NOT EXISTS (
SELECT 1 FROM refunds r WHERE r.user_id = u.user_id
);Membership tests keep each user once regardless of how many payment rows match. EXISTS checks for a completed payment, while NOT EXISTS checks for any refund.
08 Role aliases and a compound match
SELECT t.trade_id, base.name, quote.name
FROM trades t
JOIN assets base ON base.symbol = t.base_asset
JOIN assets quote ON quote.symbol = t.quote_asset
ORDER BY t.trade_id;SELECT DISTINCT d.user_id
FROM deposits d
JOIN trades t
ON t.user_id = d.user_id
AND t.executed_at::DATE = d.completed_at::DATE
WHERE d.status = 'completed'
ORDER BY d.user_id;Aliases describe two roles played by the same table. The second relationship needs both user and calendar day; remove either and unrelated facts can pair. Both predicates are required, but neither alone promises one-to-one cardinality.
09 Find missing expected rows
WITH expected AS (
SELECT p.pair_id, c.trade_date
FROM pairs p
CROSS JOIN (
SELECT DATE '2026-01-01' + CAST(i AS INTEGER) AS trade_date
FROM range(0, 3) t(i)
) c
WHERE c.trade_date >= p.listed_at
)
SELECT e.pair_id, e.trade_date
FROM expected e
LEFT JOIN candles c
ON c.pair_id = e.pair_id
AND c.trade_date = e.trade_date
WHERE c.pair_id IS NULL
ORDER BY e.pair_id, e.trade_date;The generated calendar and pairs create the rows that should exist. Any expected row left unmatched by the anti-join is a missing candle.
10 Reconcile and classify
WITH dep AS (
SELECT completed_at AS flow_date,
SUM(usd_value) AS deposits_usd
FROM cash_deposits
WHERE status = 'completed'
GROUP BY completed_at
), wd AS (
SELECT completed_at AS flow_date,
SUM(usd_value) AS withdrawals_usd
FROM cash_withdrawals
WHERE status = 'completed'
GROUP BY completed_at
)
SELECT COALESCE(dep.flow_date, wd.flow_date) AS flow_date,
COALESCE(dep.deposits_usd, 0)
- COALESCE(wd.withdrawals_usd, 0) AS net_flow_usd
FROM dep
FULL OUTER JOIN wd ON wd.flow_date = dep.flow_date
ORDER BY flow_date;SELECT v.user_id, v.volume, f.tier_name
FROM user_volume v
JOIN fee_tiers f
ON v.volume >= f.min_volume
AND (v.volume < f.max_volume OR f.max_volume IS NULL)
ORDER BY v.user_id;FULL OUTER JOIN preserves a date found on either side. The half-open range assigns the exact boundary of 100 to the next tier once, not twice.
