Probabilistic & Bayesian Learning

One theorem runs through this entire topic. Bayes' rule turns "how likely is this data given the hypothesis?" into "how likely is this hypothesis given the data?", and every algorithm here is a different answer to what you do with that posterior: take its maximum, average over it, or optimise it when half the variables are hidden.

The pages

• Bayes' Theorem & Concept Learning - the theorem, and what it means to learn a concept probabilistically.
• MAP, ML and the Cancer Example - the two estimators, and the worked example that shows why priors matter.
• Brute-Force MAP Learning & Find-S - what a consistent learner is doing in Bayesian terms.
• Maximum Likelihood & Least Squared Error - why fitting a line by least squares is a maximum likelihood estimate.
• Bayes Optimal Classifier & Gibbs - the best possible classifier, and the cheap approximation to it.
• Naive Bayes - the independence assumption that should not work but does.
• Expectation Maximization - learning when some of the data is missing.
• Markov Processes & Markov Chains - the memoryless assumption, and where it holds.
• Hidden Markov Models - states you cannot see, inferred from observations you can.
• HMM Algorithms - forward-backward, Viterbi, Baum-Welch.
• Cheat Sheet - the whole topic on one page.