Stability for selecting the number of clusters: literature review, questions, and ideas

Posted in Science on August 29, 2008


Stability for selecting the number of clusters: literature review, questions, and ideas

Lecture slides:

  • Clustering Stability — a literature review, <br>many questions, and a few ideas for answers
  • Overview
  • The principle of stability
  • Stability as a tool for model selection in clustering
  • Stabilty – the general principle
  • The toy figure in favor of stability
  • Generating artificial data sets
  • How to use the clustering algorithm
  • Distances between the clusterings
  • Which clusterings to compare?
  • Stability scores
  • Normalization
  • Selecting K, finally
  • Stability in theory
  • Negative results on stability
  • First catch: large vs. small sample size
  • Possible solution: “stability window”
  • Second catch: attaining the global minimum
  • Possible solution: exploring objective function
  • Catch 3: What is “the right K”,<br> actually?
  • The “correct K”, first approach
  • The “correct” K, second approach
  • Idea: hierarchy of cluster core sets
  • Summary

Author: Ulrike Von Luxburg, Max Planck Institute For Biological Cybernetics

Watch Video

Tags: Science, Lectures, Computer Science, Clustering, Machine Learning, VideoLectures.Net