CaltechAUTHORS
  A Caltech Library Service

A Unifying review of linear gaussian models

Roweis, Sam and Ghahramani, Zoubin (1999) A Unifying review of linear gaussian models. Neural Computation, 11 (2). pp. 305-345. ISSN 0899-7667. http://resolver.caltech.edu/CaltechAUTHORS:ROWnc99

[img]
Preview
PDF - Published Version
See Usage Policy.

287Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:ROWnc99

Abstract

Factor analysis, principal component analysis, mixtures of gaussian clusters, vector quantization, Kalman filter models, and hidden Markov models can all be unified as variations of unsupervised learning under a single basic generative model. This is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity. Through the use of other nonlinearities, we show how independent component analysis is also a variation of the same basic generative model.We show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term. We introduce a new model for static data, known as sensible principal component analysis, as well as a novel concept of spatially adaptive observation noise. We also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1162/089976699300016674DOIUNSPECIFIED
http://www.mitpressjournals.org/doi/abs/10.1162/089976699300016674PublisherUNSPECIFIED
Additional Information:© 1999 Massachusetts Institute of Technology. Received September 5, 1997; accepted April 23, 1998; posted online March 13, 2006. We thank Carlos Brody, Sanjoy Mahajan, and ErikWinfree for many fruitful discussions in the early stages, the anonymous referees for helpful comments, and Geoffrey Hinton and John Hopfield for providing outstanding intellectual environments and guidance. S.R. was supported in part by the Center for Neuromorphic Systems Engineering as a part of the National Science Foundation Engineering Research Center Program under grant EEC-9402726 and by the Natural Sciences and Engineering Research Council of Canada under an NSERC 1967 Award. Z.G. was supported by the Ontario Information Technology Research Centre.
Funders:
Funding AgencyGrant Number
NSFEEC-9402726
Natural Sciences and Engineering Research Council of CanadaNSERC 1967
Ontario Information Technology Research CentreUNSPECIFIED
Subject Keywords:MAXIMUM-LIKELIHOOD; COMPONENT ANALYSIS; EM ALGORITHM; SEPARATION; NETWORKS; IMAGES
Record Number:CaltechAUTHORS:ROWnc99
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:ROWnc99
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:13697
Collection:CaltechAUTHORS
Deposited By: Jason Perez
Deposited On:25 Jun 2009 16:53
Last Modified:26 Dec 2012 10:54

Repository Staff Only: item control page