CaltechAUTHORS
  A Caltech Library Service

A Universal Analysis of Large-Scale Regularized Least Squares Solutions

Panahi, Ashkan and Hassibi, Babak (2017) A Universal Analysis of Large-Scale Regularized Least Squares Solutions. In: 31st Conference on Neural Information Processing Systems (NIPS). Advances in Neural Information Processing Systems. No.30. , pp. 1-10. http://resolver.caltech.edu/CaltechAUTHORS:20190107-105602343

[img] PDF - Published Version
See Usage Policy.

403Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20190107-105602343

Abstract

A problem that has been of recent interest in statistical inference, machine learning and signal processing is that of understanding the asymptotic behavior of regularized least squares solutions under random measurement matrices (or dictionaries). The Least Absolute Shrinkage and Selection Operator (LASSO or least-squares with ℓ_1 regularization) is perhaps one of the most interesting examples. Precise expressions for the asymptotic performance of LASSO have been obtained for a number of different cases, in particular when the elements of the dictionary matrix are sampled independently from a Gaussian distribution. It has also been empirically observed that the resulting expressions remain valid when the entries of the dictionary matrix are independently sampled from certain non-Gaussian distributions. In this paper, we confirm these observations theoretically when the distribution is sub-Gaussian. We further generalize the previous expressions for a broader family of regularization functions and under milder conditions on the underlying random, possibly non-Gaussian, dictionary matrix. In particular, we establish the universality of the asymptotic statistics (e.g., the average quadratic risk) of LASSO with non-Gaussian dictionaries.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://papers.nips.cc/paper/6930-a-universal-analysis-of-large-scale-regularized-least-squares-solutionsPublisherArticle
Additional Information:© 2017 Neural Information Processing Systems Foundation, Inc.
Record Number:CaltechAUTHORS:20190107-105602343
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20190107-105602343
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:92113
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:07 Jan 2019 19:27
Last Modified:07 Jan 2019 19:27

Repository Staff Only: item control page