# Probability, Information Theory and Bayesian Inference

Posted in Science on September 07, 2008

Lecture slides:

• Probabilistic Machine Learning
• Why Probabilistic Models for Learning?
• Probabilities and Ensembles
• Basic Rules of Probability
• Expectation and Variance (Moments)
• Example of Joint Probability - Bigrams
• An Exercise on Mammographies
• Solving the Mammography Exercise Writing Down Probabilities
• Solving the Mammography Exercise Playing with Concrete Numbers
• Solving the Mammography Exercise Apply Bayes’ Rule
• Do You Trust Your Doctor?
• Information, Probability and Entropy
• Entropy
• Entropy of a Binary Random Variable
• Information Between Two Random Variables
• Kullback-Leibler Divergence
• Shannon’s Source Coding Theorem
• What is Probability?
• Beliefs and Probability
• Bayesian Learning
• Bayesian Learning: A Coin Toss Example
• Priors for Coin Tossing
• Posterior for Coin Tossing
• Before and After Observing One Head
• Making Predictions
• Some Terminology