# Foundations of Statistical Learning Theory : Empirical Infe-rence in high-dimention spaces

Posted in Science on October 13, 2008

Lecture Slides:

• Empirical Inference Science
• Summary
• Contents
• I. Inductive Inferences
• Pattern Recognition Problem
• Complexity Concept
• Occam’s Razor in Physical Science
• the Main Theorem of Vc Theory
• Capacity Concepts: the Vc Entropy and the Growth Function
• the Structure of the Growth Function: the Vc Dimension
• the Main Results of the Vc Theory
• Concept of Falsifiability
• Vc Dimension and Popper Dimension
• Vc and Popper Dimension: Illustration
• Vc Bounds and Srm Principle
• the Occam’s Razor Principle and the Srm Principle
• the Crucial Point
• Example 1: the Vc Dimension Is Equal to the Number of Entities (Parameters)
• Example 2: the Vc Dimension Is Larger Than the Number of Entities (Parameters)
• Example 3: the Vc Dimensions Is Less Than the Number of Entities (Parameters)
• the Idea of Support Vector Machines
• Illustration
• Technical Details
• More Technical Details
• Ii. Transductive Inferences
• Inductive and Transductive Inferences
• What Is the Transduction Problem
• Equivalence Classes
• Prediction of Molecular Bioactivity
• Selective Inference
• the Imperative for the Complex World
• Iii. Problems of Non-Inductive Inferences
• What Is Wrong With Large Margin?
• Back to Vc Entropy: the Concept of Contradiction
• Idea of Universum
• Inference Based on the Number of Contradictions on Universum
• Experiments With Digit Recognition
• Further Capacity Control: Svm+
• Svm+: Formulation
• Svm+: Dual Space Solution
• One Step More: Learning Hidden Information
• Master-Class Learning
• Original Digits
• Corrupted Digits
• Master-Class Digit Recognition Learning: Technical Space
• Master-Class Digit Recognition Learning: Holistic Space
• Codes for Holistic Description
• Big Picture
• Machine Learning and Empirical Inference Science
• Conclusion: Two Metaphors for a Simple World
• Conclusion: Two Metaphors for a Complex World
• What Is Empirical Inference Science About?

Author: Léon Bottou, Nec Research Coauthor: Vladimir Vapnik, University of London