Suffix tree construction. Mention the new linear time array constructions - - using suffix trees for finding motifs with gaps (some new observations: 0.5 - 1 hours). - finding cis-regulatory motifs by comparative genomics (1 hour) - Hidden Markov techniques for haplotyping

Author: Esko Ukkonen, University Of Helsinki

Read more about this video…

Graphical Models, such as Markov random fields, are a powerful methodology for modeling probability distributions over large numbers of variables. These models, in principle, offer a natural approach to learning and inference of many computer vision problems, such as stereo, denoising, segmentation, and image labeling. However, graphical models face severe computational problems when dealing with images, due to the fact that the uncertainty structure is a "grid", and not a one-dimensional tree or chain.

In this talk, I will discuss a practical and efficient framework for joint learning and inference in situations where a normal graphical model would be intractable. This framework is based on two basic ideas:

1. Iteratively using a series of tractable models.
2. New loss functions measuring only univariate accuracy.

That is, the problem is attacked through a sequence of models, each of which is tractable. The motivating example is an image-- the first model is defined over scanlines, while the next model is defined over columns, "crossing over" the first model. The results of each model can be computed efficiently by dynamic programming, and are used by the next layer. During learning, the parameters of the entire "stack" of models are simultaneously fit to give maximally accurate univariate marginal distributions.

This talk will include experimental results on several problems, including automatic labeling of outdoor scenes.

Speaker: Justin Domke
Google Tech Talks September 9, 2008

Bio
Justin Domke is pursing a Ph.D. at the University of Maryland. Before coming to Maryland he received B.S. degrees in Physics and Computer Science from Washington University is St. Louis. His research interest is efficient learning and inference with graphical models and applications to computer vision and image processing problems.

Read more about this video…

Optical music recognition (OMR) enables early music collections to be digitized on a large scale. The workflow for such digitisation projects also includes scanning and preprocessing, but the cost of expert human labour to correct automatic recognition errors dominates the cost of these other two steps. To reduce the number of recognition errors in the OMR process, we present an innovative application of maximum a posteriori (MAP) adaptation for hidden Markov models (HMMs) to build book-adaptive models, taking advantage of the new learning data generated from human editing work, which is part of any music digitization project. We also experimented with using the generated data to build book-dependent models from scratch, which sometimes outperform the book-adaptive models after enough corrected data is available. Our experiments show that these approaches can reduce human editing costs by more than half and that they are especially well suited to highly variable sources like early or degraded documents.

Author: Douglas Eck, Department of Computer Science, University of Montreal

Read more about this video…

Lecture slides:

• Agenda - Necessity of robustly estimating high-order Markov process models
• Necessity of robustly estimating high-order Markov process models - Natural language and Markov models
• Necessity of robustly estimating high-order Markov process models - Problem caused by data sparseness
• Necessity of robustly estimating high-order Markov process models - Introducing smoothing methods
• Agenda - Prior work: estimating Markov process models by hierarchical Bayesian approaches
• Prior work - Two major smoothing criteria
• Prior work - Smoothing methods = Hierarchical Bayesian estimation
• Prior work - Known performances of existing methods
• Prior work - Frequency modification by an indicator function
• Agenda - Our proposition: Separating precision and mean in Dirichlet prior
• Our proposition - Our direction
• Our proposition - Discounting factor should depend on current states.
• Our proposition - Separating precision and mean in Dirichlet prior
• Our proposition - New formulation : context-dependent Dirichet prior
• Our proposition - Effective frequency for more precise lower-order distribution
• Our proposition - New Dirichlet prior will outperform when # of states is small.
• Agenda - Experimental result
• Experimental result - Checking the performances depending on the # of states.
• Experimental result : evaluating test-set perplexity - Natural language modeling : slightly worse than Kneser-Ney smoothing
• Experimental result : evaluating test-set perplexity - Protein sequence modeling : outperformed Kneser-Ney smoothing (1)
• Experimental result : evaluating test-set perplexity - Protein sequence modeling : outperformed Kneser-Ney smoothing (2)
• Agenda - Conclusion
• Conclusion

Author: Rikiya Takahashi, IBM Research

Read more about this video…

Markov jump processes (MJPs) underpin our understanding of many important systems in science and technology. They provide a rigorous probabilistic framework to model the joint dynamics of groups (species) of interacting individuals, with applications ranging from information packets in a telecommunications network to epidemiology and population levels in the environment. These processes are usually non-linear and highly coupled, giving rise to non-trivial steady states (often referred to as emerging properties). Unfortunately, this also means that exact statistical inference is unfeasible and approximations must be made in the analysis of these systems. A traditional approach, which has been very successful throughout the past century, is to ignore the discrete nature of the processes and to approximate the stochastic process with a deterministic process whose behaviour is described by a system of non-linear, coupled ODEs. This approximation relies on the stochastic fluctuations being negligible compared to the average population counts. There are many important situations where this assumption is untenable: for example, stochastic fluctuations are reputed to be responsible for a number of important biological phenomena, from cell differentiation to pathogen virulence. Researchers are now able to obtain accurate estimates of the number of macromolecules of a certain species within a cell, prompting a need for practical statistical tools to handle discrete data.

Author: Guido Sanguinetti, University of Sheffield

Read more about this video…

We present a general and efficient optimisation methodology for for max-margin sructured classification tasks. The efficiency of the method relies on the interplay of several techiques: marginalization of the dual of the structured SVM, or max-margin Markov problem; partial decomposition via a gradient formulation; and finally tight coupling of a max-likelihood inference algorithm into the optimization algorithm, as opposed to using inference as a working set maintenance mechanism only.

Author: Juho Rousu, University of Helsinki

Read more about this video…

Lecture slides:

• Hierarchical Multilabel Classifcation: union of partial paths model
• Frequently used learning strategies for hierarchies
• The classification model
• Feature vectors
• Loss functions for hierarchies
• Max-margin Structured output learning
• Optimization problem
• Marginalized problem
• Decomposing the model
• Conditional Gradient method
• Conditional Gradient Ascent
• Using inference to find update directions
• Experiments
• Optimization efficiency
• Prediction accuracy: Levelwise F1
• Scalability?
• Conclusions

Author: Juho Rousu, University of Helsinki

Read more about this video…

Lecture slides:

• Hidden Markov Models with Applications
• Outline
• Clustering by Mixture Models
• Clustering
• Approaches to Clustering
• Mixture Model-based Clustering
• Mixture of Normals
• Advantages
• EM Algorithm
• EM for the Mixture of Normals
• Computation Issues
• Examples
• Classification Likelihood
• Classification EM
• Outline

Author: Jia Li, Pennsylvania State University

Read more about this video…

Analysis of genetic variation in human populations is critical to the understanding of the genetic basis for complex diseases. Although genomes of several species have been sequenced, it is still too expensive to sequence genomes of several individuals to analyze genetic variation. Furthermore, most of the genome is invariant among individuals.

Author: Niels Landwehr, University of Freiburg

Read more about this video…

Lecture slides:

• Outline
• Nonlinear dynamical systems
• Nonlinear state-space models (NSSMs)
• Nonlinear state-space models (NSSMs)01
• Variational inference for the NSSM
• Discrete-time models: pros and cons
• Continuous-time NSSM
• Stochastic Differential Equations
• Continuous-time NSSM
• Approximations
• Variational continuous-time NSSM
• State inference
• Faster state inference
• Experiment: Continuous-time NSSM
• Experiment: Continuous-time NSSM01
• Experiment: Continuous-time NSSM02
• Experiment: Continuous-time NSSM03
• Experiment: State inference
• Experiment: State inference01
• Conclusion

Author: Antti Honkela, Helsinki University of Technology

Read more about this video…

I will present the use of layered probabilistic representations for modeling the activities of people in a system named S-SEER. I will describe how we use the representation to do sensing, learning and inference at multiple levels of temporal granularity and abstraction. The approach centers on the use of a cascade of Hidden Markov Models (HMMs) named Layered Hidden Markov Models (LHMMs) to diagnose states of a user's activity based on real-time streams of evidence from video, audio and computer (keyboard and mouse) interactions.

Author: Nuria Oliver, Tel Aviv University

Read more about this video…

Lecture content:

• AR Processes: Discrete-time Gaussian Markov Processes
• From discrete to continuous time
• Vector processes
• The Wiener Process
• Discretized Wiener Process
• Gaussian Processes
• SDEs
• Simulation of an SDE
• Stochastic Integration
• General form of a Diffusion process
• Simple Examples
• Infinitesimal moments
• Stationary Processes
• Fourier Analysis
• Power spectrum of SDE
• Vector OU process
• Mean square differentiability
• Relating Discrete-time and Sampled Continuous-time GMPs
• Inference
• Fokker-Planck Equations
• Simple example: Wiener process with drift
• Fokker-Planck Boundary Conditions
• Parameter Estimation

Author: Chris Williams, University of Edinburgh

Read more about this video…

Lecture content: 

• Data Assimilation
• Collaborators
• Three Estimation Problems
• Turning a model into a state estimation problem
• Statement of the Problem
• GOAL: estimate moments
• A Nonlinear Example
• Observations
• Extended Kalman Filter
• Alternative Approaches
• Observations
• KSP Filter Results
• Why not KSP?
• A Statistical-Mechanical Digression
• Fact: log n! 1/4 n log n - n
• BAYESIAN STATEMENT
• Path Integral Method
• Otherwise use sampling
• Hybrid Monte Carlo
• The HMC algorithm
• What’s going on?
• Unigrid Monte Carlo
• Generalized HMC
• PIMC Results
• RESULTS: decorrelation time
• Conclusions (Sampling)
• OVERALL CONCLUSIONS
• Further Information

Author: Juan Restrepo, Mathematics Department, University of Arizona

Read more about this video…

We demonstrate the superiority of Population Monte Carlo techniques over standard Metropolis Markov Chain Monte Carlo (MCMC) methods for inferring optimal parameters for a particular mechanistic model of a biological process given noisy experimental data. As our understanding of biological processes increases, the proposed models to describe them become more complex. With such potentially large numbers of equations and parameters, it is no longer feasible to hand-pick parameter values and be sure that the most appropriate values have been chosen. Monte Carlo methods are becoming more widely used for estimating parameter values, however we show that the standard Metropolis MCMC approach fails to converge on optimal values for even relatively simple models and that a more sophisticated method, in the form of non-Markovian Population Monte Carlo, may be successfully employed to produce consistent and accurate results. We illustrate the basic problem using the minimal model for the circadian genetic network in Arabidopsis thaliana, which consists of 3 linked differential equations containing a total of 6 parameters, with an additional noise parameter incorporated to estimate the variance of noise in the data.

Joint work with Mark Girolami.

Read more about this video…

0. A fundamental theorem of simulation
1. Markov chain basics
2. Slice sampling
3. Gibbs sampling
4. Metropolis-Hastings algorithms
5. Variable dimension models and reversible jump MCMC
6. Perfect sampling
7. Adaptive MCMC and population Monte Carlo

Read more about this video…