Prediction and practice
Training and test sets, baselines and simple models, evaluating classifiers, and running a reproducible, honest analysis.
About 5 hours of study
Learning objectives
- Fit and judge a simple model against a baseline
- Evaluate a classifier with precision and recall
- Run a reproducible pipeline without leakage
Lessons
- 1
Prediction basics Advanced
The core ideas of predictive modelling in plain Python: features and targets, training and test sets, a baseline to beat, a least-squares line, nearest neighbours, and error measures such as MAE and RMSE.
- 2
Evaluating classifiers Advanced
Judging yes/no predictions properly: the confusion matrix, accuracy and why it misleads on rare classes, precision, recall, the F1 score, and choosing a decision threshold.
- 3
The data science workflow Advanced
Putting the steps together into a reproducible analysis: a pipeline from raw data to report, train and test discipline, data leakage, documenting cleaning decisions, ethics and privacy, and communicating results honestly.
Module practice
- Data science: MAE and RMSEEasy
- Data science: Does the model beat the baseline?Medium
- Data science: k-nearest neighboursHard
- Data science: How accurate is the classifier?Easy
- Data science: Precision, recall and F1Medium
- Data science: Choose a decision thresholdHard
- Data science: A reproducible splitEasy
- Data science: Standardise without leakageMedium
- Data science: A cleaning pipeline with a logHard