Paper Abstract

Bayesian Learning in Sparse Graphical Factor Models via Variational Mean-Field Annealing

Ryo Yoshida (Institute of Statistical Mathematics, Tokyo) & Mike West (StatSci@Duke)

Published in:Journal of Machine Learning Research, 11, 1771-1798 (May 2010)

Original Manuscript: May 2009

We describe a novel annealed entropy approach to Bayesian computation in sparse multivariate statistical models, focused on key examples of graphical latent factor models. The method uses an augmented posterior distribution with an artificial regularizer on the posterior entropy of sparsity configurations defining the model structure. This defines an annealing algorithm for iterative approximation of posterior modes in the joint space of model structure and parameters. We provide theoretical related to convergence of the algorithm to posterior modes, a stochastic variant for higher-dimensional problems, demonstrate its utility and compare with sparse PCA in simulation studies, and explore analysis of a breast cancer genomics data set for applied illustration.

Keywords: Bayesian annealing, Graphical factor models, Latent factor models, MAP estimation, Sparse factor analysis, Gene expression profiling


Interested readers can visit the web site of the first author for supplementary material and code. This site contains general R software implementing the models and methods of the paper, together with data and complete information needed to recapitulate the example analyses reported. Also there is a additional supporting technical material in an earlier, extended technical report related to the theory and methods of the paper.


Elements of the research reported here were developed while Ryo Yoshida was visiting SAMSI and Duke University during 2008-09. Aspects of the research of Mike West were partially supported by grants from the U.S. National Science Foundation (DMS-0342172) and National Institutes of Health (NCI U54-CA-112952). Any opinions, findings and conclusions or recommendations expressed in this work are those of the authors and do not necessarily reflect the views of the NSF.