Information-theoretic Interpretations of Compressive Sampling.

Compressive sampling or compressed sensing (CS) is a new paradigm for data acquisition and signal recovery. There are two-way relationships between CS and information theory: the former should and can be analyzed from the perspective of the latter, while the latter's content and extent are enriched and broadened by the former. Specifically, some basic concepts and theorems in information theory, such as source, channel, source coding, channel coding, rate distortion, Fano inequality, and the data processing theorem, provide theoretical foundation for research on CS, in particular, that concerning performance limits (e. g., sampling rates). CS provides a highly efficient strategy for collecting, storing, transmitting, and reconstructing sparse signals through its unique concepts and algorithms, such as the sparsity of real signals (enabling CS sampling at a rate lower than Nyquist rate), the information sensing capacity of random sampling matrices (which preserve information ; and information reconstruction based on convex optimization (different from signal reconstruction by Sine kernels in the Shannon-Nyquist sampling theorem). Thus, CS is a mechanism for direct information sampling and processing, extending the domain of classic information theory. This paper seeks to clarify and explain the relationships between CS and information theory, revealing some of the fundamental issues in CS, in particular, those concerning CS sampling, and providing guidance for CS research directions. [ABSTRACT FROM AUTHOR]