A Caltech Library Service

Statistically optimum pre- and postfiltering in quantization

Tuqan, Jamal and Vaidyanathan, P. P. (1997) Statistically optimum pre- and postfiltering in quantization. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 44 (12). pp. 1015-1031. ISSN 1057-7130. doi:10.1109/82.644563.

See Usage Policy.


Use this Persistent URL to link to this item:


We consider the optimization of pre- and postfilters surrounding a quantization system. The goal is to optimize the filters such that the mean square error is minimized under the key constraint that the quantization noise variance is directly proportional to the variance of the quantization system input. Unlike some previous work, the postfilter is not restricted to be the inverse of the prefilter. With no order constraint on the filters, we present closed-form solutions for the optimum pre- and postfilters when the quantization system is a uniform quantizer. Using these optimum solutions, we obtain a coding gain expression for the system under study. The coding gain expression clearly indicates that, at high bit rates, there is no loss in generality in restricting the postfilter to be the inverse of the prefilter. We then repeat the same analysis with first-order pre- and postfilters in the form 1+αz-1 and 1/(1+γz^-1 ). In specific, we study two cases: 1) FIR prefilter, IIR postfilter and 2) IIR prefilter, FIR postfilter. For each case, we obtain a mean square error expression, optimize the coefficients α and γ and provide some examples where we compare the coding gain performance with the case of α=γ. In the last section, we assume that the quantization system is an orthonormal perfect reconstruction filter bank. To apply the optimum preand postfilters derived earlier, the output of the filter bank must be wide-sense stationary WSS which, in general, is not true. We provide two theorems, each under a different set of assumptions, that guarantee the wide sense stationarity of the filter bank output. We then propose a suboptimum procedure to increase the coding gain of the orthonormal filter bank.

Item Type:Article
Related URLs:
URLURL TypeDescription
Vaidyanathan, P. P.0000-0003-3003-7042
Additional Information:© Copyright 1997 IEEE. Reprinted with permission. Manuscript received June 12, 1995; revised July 8, 1996. This work was supported in part by the Office of Naval Research under Grant N00014-93-1-0231, by Tektronix, Inc., and by Rockwell International.
Subject Keywords:Half-whitening scheme, noise shaping, optimum pre- and postfiltering, subband coding
Issue or Number:12
Record Number:CaltechAUTHORS:TUQieeetcsII97
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:4936
Deposited By: Archive Administrator
Deposited On:13 Sep 2006
Last Modified:08 Nov 2021 20:21

Repository Staff Only: item control page