The Ultimate Limits of Information Density

Abstract

In a conference devoted to studying the ultimate limits of communication systems, we wish to make an information-theoretic contribution. It is surely appropriate to do this, since Shannon’s theorem tells us exactly what the ultimate communication limit of a noisy channel is. Nevertheless, it has seemed to us for some time that the usual models of information theory are inadequate for a study of the ultimate limits of many practical communication and information storage systems, because of a key missing parameter. This missing parameter we call the scaling parameter. In this paper we hope to remedy this situation a bit by introducing a class of models for channels with noise scaling. Rather than give a formal definition immediately, we begin with a “thought experiment” to illustrate what we mean.