CaltechAUTHORS
  A Caltech Library Service

H∞ optimality of the LMS algorithm

Hassibi, Babak and Sayed, Ali H. and Kailath, Thomas (1996) H∞ optimality of the LMS algorithm. IEEE Transactions on Signal Processing, 44 (2). pp. 267-280. ISSN 1053-587X. http://resolver.caltech.edu/CaltechAUTHORS:HASieeetsp96

[img]
Preview
PDF
See Usage Policy.

1488Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:HASieeetsp96

Abstract

We show that the celebrated least-mean squares (LMS) adaptive algorithm is H∞ optimal. The LMS algorithm has been long regarded as an approximate solution to either a stochastic or a deterministic least-squares problem, and it essentially amounts to updating the weight vector estimates along the direction of the instantaneous gradient of a quadratic cost function. We show that the LMS can be regarded as the exact solution to a minimization problem in its own right. Namely, we establish that it is a minimax filter: it minimizes the maximum energy gain from the disturbances to the predicted errors, whereas the closely related so-called normalized LMS algorithm minimizes the maximum energy gain from the disturbances to the filtered errors. Moreover, since these algorithms are central H∞ filters, they minimize a certain exponential cost function and are thus also risk-sensitive optimal. We discuss the various implications of these results and show how they provide theoretical justification for the widely observed excellent robustness properties of the LMS filter.


Item Type:Article
Additional Information:© Copyright 1996 IEEE. Reprinted with permission. Manuscript received August 5, 1993; revised June 1, 1995. This work was supported by the Air Force Office of Scientific Research, Air Force Systems Command, under Contract AFOSR91-0060 and by the Army Research Office under Contract DAAL03-89-K-0109. The work of A.H. Sayed was supported by a grant from NSF under award MIP-9409319. The associate editor coordinating the review of this paper and approving it for publication was Dr. Virginia L. Stonick. The first author would like to thank Prof. L. Ljung for contributing to the discussion in Section VI-A.
Subject Keywords:H∞ optimisation; adaptive filters; adaptive signal processing; filtering theory; least mean squares methods; minimax techniques; prediction theory
Record Number:CaltechAUTHORS:HASieeetsp96
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:HASieeetsp96
Alternative URL:http://dx.doi.org/10.1109/78.485923
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:5273
Collection:CaltechAUTHORS
Deposited By: Archive Administrator
Deposited On:06 Oct 2006
Last Modified:26 Dec 2012 09:04

Repository Staff Only: item control page