Introduction
Do taller people weigh more? Do pricier houses sit closer to the city centre? Questions about how two numerical variables relate are answered by a scatter plot: one point per item, placed by its two values. The eye picks up direction, strength, curves, clusters and outliers immediately, and a little arithmetic then puts a number on what it sees.
How it works
Each point gets a column from its x value and a row from its y value, using the linear scale formula on each axis separately. Screen rows count from the top, so the row index is flipped: the largest y goes on the top line.
When many points share a cell, overplotting hides how many there are: a hundred customers at the same spot look like one. On a character grid you can write the count instead of a single mark; graphical libraries use transparency, smaller points, small random offsets ("jitter"), or a two-dimensional histogram of counts.
Pearson's correlation coefficient r summarises how closely the points follow a straight line:
r = sxy / (sxx * syy) ** 0.5
where sxy is the sum of products of the deviations from the means, and sxx and syy are the sums of squared deviations. r runs from −1 (a perfect falling line) through 0 (no straight-line relation) to 1 (a perfect rising line). Common rough labels are strong above 0.7 in size, moderate from 0.3, and weak below. r only measures straight lines: a perfect U shape can have r = 0. Always look at the plot.
The least-squares line has slope sxy / sxx and passes through the point of means. Each point's residual is its observed y minus the y the line predicts; large residuals mark points the line explains poorly. R squared, which for one variable equals r², is the share of the variation in y that the line explains.
Finally, correlation is not causation. Ice cream sales and drownings both rise in summer; neither causes the other. A scatter plot shows that two things move together, not why.
In Matplotlib, plt.scatter(heights, weights, alpha=0.5) draws semi-transparent points, np.corrcoef(x, y)[0, 1] computes r, and np.polyfit(x, y, 1) returns the slope and intercept of the least-squares line.
Example
# A scatter plot on a character grid, with the correlation and the least-squares line.
heights = [150, 155, 160, 162, 165, 170, 172, 175, 180, 185] # cm
weights = [50, 54, 58, 57, 62, 66, 65, 70, 74, 80] # kg
W, H = 30, 8
grid = [["."] * W for _ in range(H)]
for x, y in zip(heights, weights):
col = int((x - min(heights)) / (max(heights) - min(heights)) * (W - 1) + 0.5)
row = int((y - min(weights)) / (max(weights) - min(weights)) * (H - 1) + 0.5)
grid[H - 1 - row][col] = "*" # row 0 is the top line on screen
for line in grid:
print("|" + "".join(line))
print("+" + "-" * W)
n = len(heights)
mx, my = sum(heights) / n, sum(weights) / n
sxy = sum((x - mx) * (y - my) for x, y in zip(heights, weights))
sxx = sum((x - mx) ** 2 for x in heights)
syy = sum((y - my) ** 2 for y in weights)
slope = sxy / sxx
print(f"r = {sxy / (sxx * syy) ** 0.5:.3f}")
print(f"weight = {my - slope * mx:.1f} + {slope:.3f} x height")Line by line
- The grid is 8 rows of 30 dots; each person becomes a
*at the column of their height and the row of their weight. H - 1 - rowflips the row so that heavier people appear higher up.- The sums of deviations give r and the slope of the least-squares line.
- The intercept comes from the line passing through the means.
Expected result
|.............................*
|.........................*....
|.....................*........
|.................**...........
|............*.................
|........*.*...................
|....*.........................
|*.............................
+------------------------------
r = 0.990
weight = -76.0 + 0.834 x height
In the real world
Scientists plot dose against response, economists plot income against life expectancy, and engineers plot load against failure. Data scientists inspect scatter plots of every pair of features before modelling, and residual plots after fitting, to find patterns the model missed.
Common mistakes
- Reading causation into correlation, without considering a third factor or chance.
- Ignoring overplotting, which makes crowded regions look sparse.
- Trusting r without the plot, missing curves, clusters or a single influential point.
- Fitting a straight line to an obviously curved pattern.
Best practices
- Plot first, then compute r and fit a line only if the pattern looks straight.
- Use transparency or counts for large datasets.
- Look at the residuals: they should show no pattern.
Summary
A scatter plot places each item by two numerical values, with rows flipped for the screen and overlaps counted or made transparent. Pearson's r measures straight-line association from −1 to 1; the least-squares line and its residuals describe and test the relationship. Correlation shows association, never causation by itself.
Quick check
Key takeaways
- Each point is placed by its x and y on their own scales; screen rows count from the top.
- Count or make transparent points that overlap, or the plot understates dense areas.
- r measures straight-line association only, and correlation does not show causation.
Practice problems for this lesson
My notes
Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.