Introduction
Before you fit a model or write a report, you need to know what is in the data. Exploratory data analysis, or EDA, is that first look: how many rows and columns there are, what each column holds, how much is missing, and what the numbers look like. Experienced analysts spend a large part of every project here, because surprises found early are cheap and surprises found after a report is published are expensive.
How it works
EDA usually follows three passes.
1. Profile every column. For each column, record its type (numeric, category, date or text), how many values are missing, and its range or its most common values. A column of ages that contains "unknown", or a price column whose maximum is 99999, shows up immediately.
2. Look at the shape of numeric columns. Summary numbers hide shape, so draw a histogram: cut the range into equal bins, such as 20–29, 30–39 and so on, and count the values in each. In plain Python, value // width * width gives the start of a value's bin. A text histogram prints one row of # characters per bin. Look for:
- the centre and spread;
- skew, a long tail on one side (incomes, house prices and waiting times are usually skewed to the right);
- gaps and spikes, such as a pile of values at exactly 0 or 999, which often mean a missing value written as a number;
- outliers far from the rest.
3. Look at categories and pairs. Count the values of each category column, and compare a number across categories (the grouping lesson). Then look at pairs of numeric columns, which the next lesson covers.
Bin width matters: too wide hides detail, too narrow turns the histogram into noise. Try two or three widths. Empty bins matter too: printing them shows gaps that a list of only the filled bins would hide.
Throughout, keep a written list of questions and surprises: "why are 12% of ages missing?", "why does one store have no sales in March?". EDA does not answer them; it tells the rest of the project what to check.
Example
# Exploratory analysis: a text histogram shows the shape of a column.
ages = [23, 35, 31, 47, 52, 29, 38, 41, 33, 27, 36, 58, 44, 30, 25]
counts = {}
for age in ages:
start = age // 10 * 10
counts[start] = counts.get(start, 0) + 1
for start in sorted(counts):
print(f"{start}-{start + 9}: {'#' * counts[start]} {counts[start]}")
print("youngest:", min(ages), " oldest:", max(ages))Line by line
age // 10 * 10turns 47 into 40, the start of its ten-year bin.- The dictionary counts how many ages fall in each bin.
- The loop prints the bins in order, with one
#per person. - The last line prints the range, the first thing to check for impossible values.
Expected result
20-29: #### 4
30-39: ###### 6
40-49: ### 3
50-59: ## 2
youngest: 23 oldest: 58
Most people are in their twenties and thirties, with a tail towards the fifties: a slight skew to the right.
In the real world
Data teams start projects with a profiling notebook; tools such as pandas' describe() and value_counts() produce the same summaries in one line each. Data quality checks in production pipelines are often automated EDA: an alert fires when a column's missing share or range changes overnight.
Common mistakes
- Skipping EDA and modelling straight away, then discovering that half a column was a placeholder value.
- Trusting the mean of a skewed column without looking at its histogram.
- Leaving out empty bins, which hides gaps in the data.
- Fixing problems silently during EDA instead of writing them down and deciding later.
Best practices
- Profile every column, even the ones you think you understand.
- Draw a histogram of every important numeric column, with more than one bin width.
- Keep a written log of surprises, questions and cleaning decisions.
Summary
EDA is the first look at a dataset: profile every column, draw histograms of numeric columns by counting values in bins, count categories, and look for skew, gaps, spikes and outliers. It turns raw data into a list of facts, problems and questions for the rest of the analysis.
Quick check
Key takeaways
- Profile every column before trusting it: type, missing values, range and common values.
- A histogram shows shape, gaps and outliers that a mean hides.
- EDA produces questions and cleaning tasks, not final answers.
Practice problems for this lesson
My notes
Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.