Why visualize data

Intermediate

Introduction

A table of numbers answers questions slowly: you have to read every value and hold them in your head. A chart turns the same numbers into shapes, and the eye sees a trend, a cluster or an odd value in a fraction of a second. That is why data analysts, scientists and journalists draw charts before they trust any summary.

This course builds charts from first principles in plain Python. You will compute what every chart needs (scales, ticks, bins, quartiles, colours) and draw charts as text or as SVG markup. Where it helps, the lessons also show the equivalent Matplotlib code, the Python library most people use for charts, so you can recognise it in real projects.

How it works

In 1973 the statistician Francis Anscombe published four small datasets, now called Anscombe's quartet. Each has the same mean of x (9), nearly the same mean of y (7.50), the same variance and the same correlation (0.816). Judged by its summary statistics, every set is the same. Drawn as a chart, one is a noisy straight line, one a smooth curve, one a perfect line with a single outlier, and one a vertical stack of points with one far away. The summary hid everything that mattered.

Every chart is made of marks and channels. Marks are the shapes: points in a scatter plot, lines in a line chart, bars, areas. Channels are the properties of those marks that carry the data: their position, length, angle, area, colour and shape. The choice of which data goes to which channel is the encoding.

Channels are not equally good. Experiments by Cleveland and McGill in the 1980s showed that people compare positions on a common scale most accurately, then lengths, then angles and areas, and colour shades least accurately. That is why a bar chart (length from a common baseline) usually beats a pie chart (angles and areas), and why colour is better for telling categories apart than for showing amounts.

Good charts start from a question, not from the data: "Which region sold most?", "Is the trend going up?", "Do these two things move together?". The question decides the chart, and the next lesson turns that into rules.

In Matplotlib, a quick look at one of these datasets is a scatter plot:

import matplotlib.pyplot as plt
plt.scatter(x, y1)
plt.show()

Example

# Two datasets from Anscombe's quartet: the same summary numbers, very different shapes.
x = [10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5]
y1 = [8.04, 6.95, 7.58, 8.81, 8.33, 9.96, 7.24, 4.26, 10.84, 4.82, 5.68]
y2 = [9.14, 8.14, 8.74, 8.77, 9.26, 8.10, 6.13, 3.10, 9.13, 7.26, 4.74]


def mean(values):
    return sum(values) / len(values)


def correlation(a, b):
    ma, mb = mean(a), mean(b)
    cov = sum((p - ma) * (q - mb) for p, q in zip(a, b))
    va = sum((p - ma) ** 2 for p in a)
    vb = sum((q - mb) ** 2 for q in b)
    return round(cov / (va * vb) ** 0.5, 12)


for name, y in (("set 1", y1), ("set 2", y2)):
    print(f"{name}: mean x {mean(x):.2f}, mean y {mean(y):.2f}, r {correlation(x, y):.3f}")

for name, y in (("set 1", y1), ("set 2", y2)):
    print(name)
    for xi, yi in sorted(zip(x, y)):
        print(f"  x={xi:2d} {'#' * round(yi)}")
Open in compiler

Line by line

  1. x, y1 and y2 are the first two sets of Anscombe's quartet, which share the same x values.
  2. mean and correlation compute the usual summary numbers.
  3. The first loop prints the summaries: they are the same to two decimals.
  4. The second loop draws each set as a row of # per point, sorted by x, a crude bar chart.
  5. Set 1 rises unevenly; set 2 rises and then falls, a curve that the correlation cannot reveal.

Expected result

set 1: mean x 9.00, mean y 7.50, r 0.816
set 2: mean x 9.00, mean y 7.50, r 0.816
set 1
  x= 4 ####
  x= 5 ######
  x= 6 #######
  x= 7 #####
  x= 8 #######
  x= 9 #########
  x=10 ########
  x=11 ########
  x=12 ###########
  x=13 ########
  x=14 ##########
set 2
  x= 4 ###
  x= 5 #####
  x= 6 ######
  x= 7 #######
  x= 8 ########
  x= 9 #########
  x=10 #########
  x=11 #########
  x=12 #########
  x=13 #########
  x=14 ########

In the real world

Data teams look at distributions before reporting averages, because one large customer or one broken sensor can move a mean without anyone noticing in a table. Newsrooms draw charts to check a story before publishing it. Dashboards exist because a manager can read a line chart in seconds, while the same numbers in a spreadsheet take minutes.

Common mistakes

  • Trusting a summary without looking at the data, which can hide outliers, clusters and curves.
  • Encoding amounts with colour shade when length or position would be read far more accurately.
  • Starting from the data you have instead of the question you need to answer.
  • Adding decoration that carries no data and makes the real marks harder to see.

Best practices

  • Plot the raw data before you summarise it.
  • Put the most important comparison on position or length along a common scale.
  • Write the question a chart answers before you draw it.

Summary

Summary statistics compress a dataset into a few numbers and can hide its shape, as Anscombe's quartet shows. Charts encode data in marks and channels, of which position and length are read most accurately. A chart should start from the question it answers.

Quick check

1. What does Anscombe's quartet show?
2. Which channel do people read most accurately?
3. In a bar chart, what is the mark and which channel carries the value?
4. What should come first when making a chart?

Key takeaways

  • Summary numbers hide shape; a chart shows it.
  • Position and length are read most accurately; angle, area and colour shade less so.
  • Decide the question first, then choose the marks and channels that answer it.

Practice problems for this lesson

My notes

Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.