Probabilities, trees and ensembles

Bayes' rule and Naive Bayes, decision trees, and combining many models into one.

About 5 hours of study

Learning objectives

Lessons

  1. 1

    Bayes' rule and Naive Bayes Intermediate

    Update a belief with evidence using Bayes' rule, avoid the base-rate fallacy, and build a Naive Bayes spam filter with word counts, Laplace smoothing and log probabilities.

  2. 2

    Decision trees Intermediate

    Classify by asking one question after another: entropy, information gain and the Gini index, thresholds for numeric features, and how depth limits and pruning stop a tree from memorizing.

  3. 3

    Ensembles: random forests and boosting Intermediate

    Many weak models can beat one strong one: majority voting, bagging with bootstrap samples, random forests, and boosting, where each new model fixes the mistakes of the last.

Module practice

Module quiz

1. In Bayes' rule, the belief before seeing evidence is the...
2. The entropy of a pure group is...
3. A bootstrap sample is drawn...
4. A test finds 90% of a rare disease (1% of people) but flags 9% of healthy people. After a positive test, the chance of disease is about...
5. Why do decision trees need a depth limit or pruning?
6. In AdaBoost, a weak model has weighted error 0.25. What is its say α?

‹ ClassificationClustering, neural networks and practice ›