Optimum pre- and postfilters for quantization

Abstract

We consider the optimization of pre- and post filters surrounding a uniform quantizer such that the mean square error due to quantization is minimized. Unlike some previous work, the postfilter is not restricted to be the inverse of the prefilter. With no order constraint on the filters, we present closed form solutions for the optimum pre- and post filters. Using these optimum solutions, we obtain a coding gain expression for the system under study. We then repeat the same analysis with first order pre- and post filters in the form 1+αz^-1 and 1/(1+γz^-1) providing some examples where we compare coding gain performance with the case of α=γ.