pandas

pandas from Series and DataFrames to selection, missing data, transforms, groupby, reshaping, merges, text, dates, file input and output, and a full analysis.

Level
Intermediate
Time
About 28 hours
Modules
5
Lessons
15
Exercises
45

pandas is the everyday tool of data analysis in Python: it loads tables, cleans them, combines them and summarises them in a few lines. This course teaches its core objects and operations, the rules behind them, and the traps that produce silently wrong numbers.

pandas cannot run in this site's Python runner, so every lesson shows real pandas code with results written as plain lists and numbers, and every runnable example and exercise is a plain-Python model of what pandas does. Implementing the rules yourself makes the real library predictable when you use it on your own computer.

What you will learn

  • Create and inspect Series and DataFrames and select data with loc, iloc and masks
  • Handle missing data and transform, sort and rank columns
  • Summarise with groupby and reshape with pivot_table and melt
  • Merge and stack tables safely, with validation
  • Work with text and date columns and read and write files correctly
  • Write clear, fast analyses as method chains from question to report

Skills you will gain

  • Series and DataFrames
  • Selection and filtering
  • Missing data
  • groupby and pivot tables
  • Merging tables
  • Text and date columns

Before you start

  • The Python course: lists, dictionaries, loops, functions and string methods
  • The NumPy course helps but is not required
  • Nothing to install for the course; install pandas with pip later to use it on your own computer

Where it is used

  • Data analysis and reporting
  • Data cleaning pipelines
  • Preparing data for machine learning
  • Business and research analytics

Modules

Module 1: pandas foundations

Why pandas exists, Series with their index and alignment, and DataFrames with their shape, types and summaries.

  1. 1

    Why pandas, and how this course runs it Intermediate

    What pandas is for, its two structures (Series and DataFrame), how a DataFrame stores a table as labelled columns, and why the exercises here model pandas in plain Python.

  2. 2

    Series: one labelled column Intermediate

    Creating a Series, its values and index, selecting by label and by position, vectorised arithmetic, and alignment: why adding two Series lines values up by label and gives NaN where a label is missing.

  3. 3

    DataFrames: tables of labelled columns Intermediate

    Creating a DataFrame, its shape, columns, index and dtypes, looking at the first rows, selecting one or several columns, and summarising the numeric columns with describe.

Module practice and quiz →

Module 2: Selecting, cleaning and transforming

Selecting with loc, iloc and masks, finding and handling missing values, and creating and recoding columns.

  1. 1

    Selecting rows and columns: loc, iloc and masks Intermediate

    Choosing parts of a DataFrame by label with loc, by position with iloc, and by condition with boolean masks combined with &, | and ~, plus assigning values through a selection.

  2. 2

    Missing data: finding, dropping and filling Intermediate

    How pandas marks missing values with NaN and None, counting them with isna, how statistics skip them, and the two ways to deal with them: dropping rows with dropna or filling with fillna, ffill and bfill.

  3. 3

    Transforming columns Intermediate

    Creating and changing columns: arithmetic between columns, assign, map with a dictionary, apply with a function, astype, rename and drop, and binning numbers into categories with cut.

Module practice and quiz →

Module 3: Summarising and reshaping

Sorting and ranking, value counts, grouping with aggregation and transform, and reshaping with pivot_table, melt and crosstab.

  1. 1

    Sorting, ranking and counting Intermediate

    Ordering rows with sort_values and sort_index, sorting by several columns in different directions, where missing values go, ranking with ties, the top rows with nlargest, and counting values with value_counts.

  2. 2

    groupby: split, apply, combine Intermediate

    Grouping rows by one or more keys and summarising each group with sum, mean, count, size and agg, the difference between count and size, transform for per-row results, and filter for keeping whole groups.

  3. 3

    Reshaping: pivot_table, melt and crosstab Advanced

    Turning long data into a wide grid with pivot and pivot_table (and why duplicates need an aggregation), turning wide data back into long rows with melt, and counting combinations with crosstab, with margins for totals.

Module practice and quiz →

Module 4: Combining tables, text and dates

Merging and stacking tables with checks, cleaning and extracting text with the .str accessor, and parsing and resampling dates.

  1. 1

    Merging and joining DataFrames Intermediate

    Combining tables in pandas: pd.merge with inner, left, right and outer joins, keys with different names, suffixes, the indicator column, validate for catching duplicate keys, concat for stacking, and join on the index.

  2. 2

    Working with text columns Intermediate

    Cleaning and analysing text in pandas with the .str accessor: case, spaces and replacements, contains and startswith with case and missing values, split and expand, and pulling structured fields out of codes with str.extract and regular expressions.

  3. 3

    Dates and times in pandas Intermediate

    Working with dates in pandas: parsing with to_datetime and errors="coerce", the .dt accessor for years, months and day names, date differences as Timedeltas, and resampling a series to monthly totals with empty months filled.

Module practice and quiz →

Module 5: pandas in practice

Reading and writing files correctly, method chains and fast vectorised code, and a complete analysis from question to report.

  1. 1

    Reading and writing files Intermediate

    Getting data in and out of pandas: read_csv with separators, missing-value markers, column selection, types, dates and chunks; writing with to_csv and its quoting rules; and the other formats pandas reads and writes.

  2. 2

    Method chaining, apply and performance Advanced

    Writing clear, fast pandas: method chains with assign, query and pipe, when apply with axis=1 is acceptable and how to replace it with vectorised code, np.where and np.select, and avoiding the SettingWithCopyWarning.

  3. 3

    A complete analysis with pandas Advanced

    One analysis from start to finish: framing the question, loading and inspecting, cleaning, merging, grouping, ranking with nlargest, computing shares and per-customer summaries, checking the numbers and presenting the result.

Module practice and quiz →

Projects

Final assessment →

What to learn next