Machine learning in practice

Choosing the algorithm is a small part of a real project. Most of the work is around it.

The workflow

1. Define the problem: what exactly is predicted, for whom, and which metric says it works (recall for disease screening, precision for spam).
2. Collect data, and check it is legal and fair to use.
3. Clean and explore: missing values, mistakes, outliers, and a look at the distributions.
4. Build features: scaling, one-hot encoding, new columns such as "day of the week".
5. Split into training, validation and test sets.
6. Start with a simple baseline (always predicting the most common class; a straight line). A complex model must beat it to earn its keep.
7. Train, compare and tune on the validation set.
8. Evaluate once on the test set.
9. Deploy, then monitor: the world changes (data drift), and a model that was accurate last year can quietly go wrong.

The libraries

Professionals rarely write gradient descent by hand. In Python:

The logistic regression of this course, in scikit-learn:

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = LogisticRegression().fit(X_train, y_train)
print(accuracy_score(y_test, model.predict(X_test)))

(StepVioz's Python runs in your browser without these libraries, so this code is shown, not run. Install Python, then pip install scikit-learn, to use them on your own computer.)

Data leakage

Leakage is when information about the answer sneaks into the features, so the model looks brilliant in testing and fails in real use. Examples: scaling with statistics computed on the whole dataset, test set included; a feature "was treated for the disease" when predicting the disease; random splits of time-series data, which let the model peek at the future. If a model is suspiciously good, look for leakage first.

Responsible ML

A model learns whatever its data contains, including past unfairness. A hiring model trained on years of biased decisions learns the bias, and then applies it at scale, with an air of objectivity.

# For each applicant: (group, actual outcome, the model's prediction)
results = [
    ("A", 1, 1), ("A", 0, 0), ("A", 1, 1), ("A", 0, 0), ("A", 1, 1),
    ("A", 0, 0), ("A", 1, 1), ("A", 0, 0), ("A", 1, 1), ("A", 0, 0),
    ("A", 1, 1), ("A", 0, 1), ("B", 1, 0), ("B", 0, 0), ("B", 1, 0),
    ("B", 1, 1), ("A", 1, 1), ("A", 0, 0), ("B", 1, 0), ("B", 0, 0),
]

def accuracy(rows):
    return sum(actual == predicted for _, actual, predicted in rows) / len(rows)

print(f"overall: {accuracy(results):.0%} of {len(results)}")
for group in ["A", "B"]:
    rows = [r for r in results if r[0] == group]
    print(f"group {group}: {accuracy(rows):.0%} of {len(rows)}")

The model is 80% accurate overall, but only 50% accurate for group B: the overall number hides who pays for the errors. Check performance for each group that matters. Ask, too:

• Privacy: was the data collected with consent? Could the model reveal personal information?
• Transparency: can the people affected get an explanation, and challenge a decision?
• Accountability: who answers when the model is wrong?
• Misuse: could it be used to harm people?

Laws increasingly require these answers, such as the EU AI Act. More importantly, the people your model affects deserve them.

Mark lesson complete