Advanced Topics in Graphical Models

CSCi 8980 Topics (Tentative)

Finite Mixture Models, EM, Exponential Families
- Wikipedia.
  EM Algorithm
- R. Neal and G. Hinton.
  A view of the EM algorithm that justifies incremental, sparse, and other variants.
  M. Jordan (editor), Learning in Graphical Models, 1999.
- R. A. Redner and H. F. Walker.
  Mixture Densities, Maximum Likelihood, and the EM Algorithm.
  SIAM Review, Vol. 26, No. 2, 1984.
- A. Banerjee, S. Merugu, I. Dhillon and J. Ghosh.
  Clustering with Bregman Divergences.
  Journal of Machine Learning Research, 2005.
Aspect Models, PLSI, Latent Dirichlet Allocation
- T. Hofmann, and J. Puzicha.
  Unsupervised Learning from Dyadic Data.
  ICSI TR-98-042, 1998.
- T. Hofmann.
  Probabilistic Latent Semantic Indexing.
  SIGIR, 1999.
- D. Blei, A. Ng, and M. Jordan.
  Latent Dirichlet Allocation.
  Journal of Machine Learning Research, 2003.
- T. Griffiths, and M. Steyvers.
  Finding Scientific Topics.
  Proceedings National Academy of Science, 2004.
MCMC, Gibbs Sampling
- C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan.
  An introduction to MCMC for machine learning.
  Machine Learning, 2003.
- Wikipedia.
  Gibbs Sampling.
- G. Casella and E. George.
  Explaining the Gibbs sampler.
  The American Statistician, 1992.
Dirichlet Processes
- T. S. Ferguson.
  A Bayesian analysis of some nonparametric problems.
  Annals of Statistics, 1973.
- D. Blackwell and J. MacQueen.
  Ferguson distributions via Polya urn schemes.
  Annals of Statistics, 1973.
- C. Antoniak.
  Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems.
  Annals of Statistics, 1974.
- J. Sethuraman.
  A constructive definition of Dirichlet priors.
  Statistica Sinica, 1994.
- R. Neal.
  Markov chain sampling methods for Dirichlet processes mixture models.
  Technical Report, University of Toronto, 1998.
- D. M. Blei and M. I. Jordan.
  Variational methods for the Dirichlet process.
  Journal of Bayesian Analysis, 2006.
Exact Inference
- S. M. Aji and R. J. McEliece
  The Generalized Distributive Law.
- F. R. Kschischang, B. J. Frey, and H.-A. Loeliger
  Factor Graphs and the Sum-Product Algorithm.
Variational Inference
- N. Lawrence
  Variational Inference Guide.
- M. Wainwright and M. Jordan
  Graphical Models, Exponential Families, and Variational Inference.
  Tech Report 649, UC Berkeley, Dept of Statistics.
- D. Blei and J. Lafferty
  A Correlated Topic Model of Science.
  Annals of Applied Probability, 2007.
Expectation Propagation
- T. Minka
  Expectation Propagation for Approximate Bayesian Inference.
  Proceedings Uncertainty in Artificial Intelligence (UAI), 2001.
- M. Seeger
  Expectation Propagation for Exponential Families.
  Tech Report, UC Berkeley, 2007.
Applications: Topic Modeling, Social Network Analysis
- C. Chemudugunta, P. Smyth, and M. Styevers.
  Modeling General and Specific Aspects of Documents with a Probabilistic Topic Model.
  Proceedings Neural Information Processing Systems (NIPS), 2007.
- M. Steyvers, P. Smyth, P. Rosen-Zvi, and T. Griffiths.
  Probabilistic Author-Topic Models for Information Discovery.
  Proceedings Knowledge Discovery and Data Mining (KDD), 2004.
- A. McCallum, A. Corrada-Emmanuel, and X. Wang.
  Topic and Role Discovery in Social Networks.
  Proceedings International Joint Conference on Artificial Intelligence (IJCAI), 2005.
- X. Wang, N. Mohanty, and A. McCallum.
  Group and Topic Discovery from Relations and Their Attributes.
  Proceedings Neural Information Processing Systems (NIPS), 2006.
Infinite Mixture Models
- C. Rasmussen.
  The Infinite Gaussian Mixture Model.
  Proceedings Neural Information Processing Systems (NIPS), 1999.
- T. Griffiths and Z. Ghahramani
  Infinite Latent Feature Models and the Indian Buffet Process.
  GCNU TR 2005-001, University College London, 2005.
- R. Thibaux and M. Jordan
  Hierarchical Beta processes and the Indian buffet process.
  Proceedings of the Conference on Artificial Intelligence and Statistics (AISTATS), 2007.
Applications: Computational Biology
- E. P. Xing, M. Jordan, and R. Roded.
  Bayesian Haplotype Inference via the Dirichlet Process.
  Journal of Computational Biology, 2007.
- E. P. Xing and K. Sohn.
  A New Nonparametric Bayesian Model for Genetic Inference in Open Ancestral Space.
  Bayesian Analysis, 2007.
- E. P. Xing and K. Sohn.
  A Nonparametric Bayesian Approach for Haplotype Reconstruction from Single and Multi-Population Data.
  Technical Report, CMU, 2007.
- M. Beal and Krishnamurthy.
  Clustering Gene Expression Time Course Data with Countably Infinite Hidden Markov Models.
  UAI, 2006.
- M. Medvedovic and S. Sivaganesan.
  Bayesian Infinite Mixture Model based clustering of gene expression profile.
  Bioinformatics, 2002.
- M. Medvedovic, K. Yeung, and R. Bumgarner.
  Bayesian mixture model based clustering of replicated microarray data.
  Bioinformatics.
Hierarchical Models
- Y. Teh, M. Jordan, M. Beal, and D. Blei.
  Hierarchical Dirichlet Processes.
  Journal of American Statistical Association, 2006.
- R. Thibaux and M. Jordan.
  Hierarchical Beta Processes and the Indian Buffet Process.
  Proceedings Artificial Intelligence and Statistics (AISTATS), 2007.
- W. Li and A. McCallum.
  Pachinko Allocation: DAG-Structure Mixture Models of Topic Correlation.
  International Conference on Machine Learning (ICML), 2006.
- D. Mimno, W. Li, and A. McCallum.
  Mixtures of Hierarchical Topics with Pachinko Allocations.
  International Conference on Machine Learning (ICML), 2007.
Gaussian Processes
- C. Williams.
  Prediction with Gaussian Processes: From linear regression to linear prediction and beyond.
  M. Jordan, editor, Learning in Graphical Models, 1999.
- D. Mackay.
  Introduction to Gaussian Processes.
  C. Bishop, editor, Neural Networks and Machine Learning, 1998.
- M. Seeger.
  Gaussian Processes for Machine Learning.
  Tech Report, UC Berkeley, 2004.