Introduction
Spreadsheets are where most people meet tables of data, and they stop scaling quickly: a million rows, a cleaning step repeated every week, or a calculation nobody can check. pandas is the Python library that data analysts, scientists and engineers use for exactly that work. It loads a table from a file, cleans it, reshapes it, combines it with other tables and summarises it, all in a few readable lines that can be run again tomorrow on new data.
This course teaches pandas' ideas and its everyday methods. One thing works differently here, and it is important to know why. pandas is a large library built on NumPy, and StepVioz runs every program in your browser with plain Python, without installing extra packages. So the lessons show real pandas code as text, and every runnable example and exercise is a small plain-Python model of what that pandas method does. Writing the model yourself is how you learn exactly what groupby, merge or dropna do with your data, and the pandas code then reads like a summary of something you already understand.
How it works
pandas has two main structures. A Series is one column: a list of values together with an index, the labels of its rows. A DataFrame is a table: several named columns of the same length that share one index.
import pandas as pd
df = pd.DataFrame({
"city": ["Dhaka", "Paris", "Tokyo", "Lima"],
"visitors": [120, 340, 510, 90],
})
df.shape # (4, 2): rows, then columns
df.columns.tolist() # ['city', 'visitors']
df["visitors"].tolist() # [120, 340, 510, 90]
df[df["visitors"] > 100]["city"].tolist() # ['Dhaka', 'Paris', 'Tokyo']
A DataFrame is stored as a column store: one array per column, not one record per row. That is why pandas code works on whole columns. df["visitors"] * 2 doubles every value at once; df["visitors"].mean() averages the column; df["visitors"] > 100 gives a column of True and False, a mask, which then picks the rows to keep. Such vectorised operations replace the loops you would write in plain Python, and pandas runs them in fast compiled code.
In our models, a table is a dictionary of lists, one list per column, which is the same idea pandas uses inside. A row is one position across every list. A mask is a list of booleans, and filtering keeps the positions where it is true. When a lesson shows df.groupby("city")["sales"].sum(), its example builds the same result with a dictionary and a loop, so you can watch every step in the visualizer.
Example
# A table the way pandas stores it: one list per column, all the same length.
table = {
"city": ["Dhaka", "Paris", "Tokyo", "Lima"],
"visitors": [120, 340, 510, 90],
"country": ["Bangladesh", "France", "Japan", "Peru"],
}
rows = len(table["city"])
print("shape:", (rows, len(table)))
print("columns:", list(table))
# A whole column at once, like df["visitors"].mean()
visitors = table["visitors"]
print("mean visitors:", sum(visitors) / len(visitors))
# One row is one position across every column, like df.iloc[2]
row = {name: values[2] for name, values in table.items()}
print("row 2:", row)
# Filter with a mask, like df[df["visitors"] > 100]
mask = [v > 100 for v in visitors]
print("mask:", mask)
print("busy cities:", [c for c, keep in zip(table["city"], mask) if keep])Line by line
tableholds three columns, each a list of four values: a column store, like a DataFrame.- The shape is the number of rows (the length of any column) and the number of columns.
- The mean works on the whole
visitorscolumn, asdf["visitors"].mean()does. - Row 2 is the third position in every column, as
df.iloc[2]returns it. - The mask is one boolean per row; zipping it with the cities keeps the rows where it is True.
Expected result
shape: (4, 3)
columns: ['city', 'visitors', 'country']
mean visitors: 265.0
row 2: {'city': 'Tokyo', 'visitors': 510, 'country': 'Japan'}
mask: [True, True, True, False]
busy cities: ['Dhaka', 'Paris', 'Tokyo']
In the real world
Analysts use pandas to clean exports from sales systems, scientists to prepare experiment results, and engineers to check logs and build the training data for machine learning models. A notebook with twenty lines of pandas often replaces a weekly afternoon of spreadsheet work. pandas' design also influenced newer libraries, such as Polars, and the same ideas (columns, masks, grouping, joins) appear in SQL and in every data tool you will meet.
Common mistakes
- Looping over rows to do what one column operation can do, which is slow and harder to read.
- Confusing the index with a column: the index labels rows, and is not one of the data columns.
- Forgetting that most methods return a new table rather than changing the old one.
- Treating the models in this course as pandas itself: they show the behaviour, and real projects use the library.
Best practices
- Think in columns: ask "what happens to the whole column?" before writing a loop.
- Keep raw data unchanged and build cleaned versions step by step.
- Check the shape and the first rows after every step while you learn.
Summary
pandas is Python's main library for tables. A Series is one labelled column, and a DataFrame is a set of columns sharing an index, stored column by column so that whole-column operations are natural and fast. In this course pandas code is shown as text, and every runnable example and exercise models what it does in plain Python, so it runs in your browser and shows each step.
Quick check
Key takeaways
- pandas stores a table as columns that share an index, so whole-column operations are natural.
- Most pandas work is selecting, cleaning, transforming, grouping and combining columns.
- Here every exercise is a small plain-Python model of what a pandas method does, so it runs in your browser.
Practice problems for this lesson
My notes
Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.