A Caltech Library Service

Entropy of Highly Correlated Quantized Data

Marco, Daniel and Neuhoff, David L. (2010) Entropy of Highly Correlated Quantized Data. IEEE Transactions on Information Theory, 56 (5). pp. 2455-2478. ISSN 0018-9448.

PDF - Published Version
See Usage Policy.


Use this Persistent URL to link to this item:


This paper considers the entropy of highly correlated quantized samples. Two results are shown. The first concerns sampling and identically scalar quantizing a stationary continuous-time random process over a finite interval. It is shown that if the process crosses a quantization threshold with positive probability, then the joint entropy of the quantized samples tends to infinity as the sampling rate goes to infinity. The second result provides an upper bound to the rate at which the joint entropy tends to infinity, in the case of an infinite-level uniform threshold scalar quantizer and a stationary Gaussian random process. Specifically, an asymptotic formula for the conditional entropy of one quantized sample conditioned on the previous quantized sample is derived. At high sampling rates, these results indicate a sharp contrast between the large encoding rate (in bits/sec) required by a lossy source code consisting of a fixed scalar quantizer and an ideal, sampling-rate-adapted lossless code, and the bounded encoding rate required by an ideal lossy source code operating at the same distortion.

Item Type:Article
Related URLs:
Additional Information:© 2010 IEEE. Manuscript received August 30, 2006; revised August 21, 2008. Current version published April 21, 2010. This work was supported in part by NSF Grants ANI-0112801 and CCF 0329715 and in part by the Center for the Mathematics of Information at California Institute of Technology. Portions of this work were presented at the IEEE International Symposium on Information Theory, Adelaide, Australia, July 2005.
Funding AgencyGrant Number
NSFCCF 0329715
Center for the Mathematics of Information at CaltechUNSPECIFIED
Subject Keywords:Entropy; entropy-rate; Gaussian random process; oversampling; quantization threshold crossing; quantized random process; rate-distortion; sampling
Other Numbering System:
Other Numbering System NameOther Numbering System ID
INSPEC Accession Number11256627
Issue or Number:5
Record Number:CaltechAUTHORS:20100616-093118997
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:18700
Deposited By: Tony Diaz
Deposited On:17 Jun 2010 05:39
Last Modified:03 Oct 2019 01:46

Repository Staff Only: item control page