Correlation and regression

Do students who study more get higher marks? Look at pairs (x, y): hours studied and mark.

Correlation

Pearson's r measures how closely the points follow a straight line, from −1 (perfect downward line) through 0 (no straight-line relation) to +1 (perfect upward line):

r = Σ(x − x̄)(y − ȳ) ÷ √(Σ(x − x̄)² × Σ(y − ȳ)²)

The least-squares line

The best straight line y = a + b·x minimizes the squared vertical distances:

b = Σ(x − x̄)(y − ȳ) ÷ Σ(x − x̄)², a = ȳ − b·x̄

xs = [1, 2, 3, 4, 5, 6]
ys = [52, 55, 61, 64, 70, 73]
n = len(xs)
mx, my = sum(xs) / n, sum(ys) / n
sxy = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
sxx = sum((x - mx) ** 2 for x in xs)
syy = sum((y - my) ** 2 for y in ys)
r = sxy / (sxx * syy) ** 0.5
b = sxy / sxx
a = my - b * mx
print("r =", round(r, 3))
print("mark ≈", round(a, 2), "+", round(b, 2), "× hours")

Correlation is not causation

Ice-cream sales and drownings rise together, because both rise in summer. A third factor, a confounder, can create a correlation with no cause between the two. Only a controlled experiment, where you change x yourself, shows causation.

Common mistakes

• r near 0 does not mean "no relation": the relation may be curved.
• One outlier can create or destroy a strong r. Always plot the data.

Mark lesson complete