Search

All physical systems register bits of information, whether it be an atom, an electrical current, the location of a billiard ball, or a switch. Information can be classical, quantum, or a hybrid of both, depending on the system. For example, an atom or an electron or a superconducting system can register quantum information because the quantum theory applies to each of these systems, but we can safely argue that the location of a billiard ball registers classical information only. These atoms or electrons or superconducting systems can also register classical bits because it is always possible for a quantum system to register classical bits. The term information , in the context of information theory, has a precise meaning that is somewhat different from our prior “everyday” experience with it. Recall that the notion of the physical bit refers to the physical representation of a bit, and the information bit is a measure of how much we learn from the outcome of a random experiment. Perhaps the word “surprise” better captures the notion of information as it applies in the context of information theory. This chapter begins our formal study of classical information. Recall that Chapter 2 reviewed some of the major operational tasks in classical information theory. Here, our approach is somewhat different because our aim is to provide an intuitive understanding of information measures, in terms of the parties who have access to the classical systems. We define precise mathematical formulas that measure the amount of information encoded in a single physical system or in multiple physical systems. The advantage of developing this theory is that we can study information in its own right without having to consider the details of the physical system that registers it. We first introduce the entropy in Section 10.1 as the expected surprise of a random variable. We extend this basic notion of entropy to develop other measures of information in Sections 10.2–10.6 that prove useful as intuitive informational measures, but also, and perhaps more importantly, these measures are the answers to operational tasks that one might wish to perform using noisy resources. While introducing these quantities, we discuss and prove several mathematical results concerning them that are important tools for the practicing information theorist.

This chapter sets the basis of quantum information theory (QIT) . The central purpose of QIT is to qualify the transmission of either classical or quantum information over quantum channels . The starting point of the QIT description is von Neumann entropy , S (ρ), which represents the quantum counterpart of Shannon's classical entropy, H ( X ). Such a definition rests on that of the density operator (or density matrix ) of a quantum system, ρ, which plays a role similar to that of the random-events source X in Shannon's theory. As we shall see, there also exists an elegant and one-to-one correspondence between the quantum and classical definitions of the entropy variants relative entropy , joint entropy , conditional entropy , and mutual information . But such a similarity is only apparent. Indeed, one becomes rapidly convinced from a systematic analysis of the entropy's additivity rules that fundamental differences separate the two worlds. The classical notion of information correlation between two event sources for quantum states shall be referred to as quantum entanglement . We then define a quantum communication channel, which encodes and decodes classical information into or from quantum states. The analysis shows that the mutual information H ( X ; Y ) between originator and recipient in this communication channel cannot exceed a quantity χ, called the Holevo bound , which itself satisfies χ ≤ H ( X ), where H ( X ) is the entropy of the originator's classical information source.

This chapter has two goals. The first is to develop the quantum version of multiplicity. That will show us how entropy is expressed in quantum theory. The second goal is to develop a general quantitative definition of temperature. Entropy plays a key role in that definition. The density of states When a physical system has reached thermal equilibrium, its macroscopic properties do not change with time. In quantum theory, the energy eigenstates of an isolated system provide predictions and estimates that are constant in time; therefore such states are appropriate for a quantum description of thermal equilibrium. To be sure, the information at hand will not enable us to select a single state as uniquely the correct state to use. We will be driven to consider many states and to form sums over them. This section develops a mathematical technique for working with such sums. When the system is both isolated and of finite size (as we shall specify here), the energy eigenstates form a discrete set, whose members we can arrange and label in order of increasing energy. (If any two distinct states happen to have the same energy, we just assign them consecutive labels.) Typically, the states will be densely spaced in energy, and so a sum over a range of states can often be approximated adequately by an integral with respect to energy, provided that we have constructed an appropriate density of states : a function that specifies the number of energy eigenstates per unit energy interval.