Projection Theorems for the Rényi Divergence on $\alpha $ -Convex Sets.

This paper studies forward and reverse projections for the Rényi divergence of order $\alpha \in (0, \infty )$ on $\alpha $ -convex sets. The forward projection on such a set is motivated by some works of Tsallis et al. in statistical physics, and the reverse projection is motivated by robust statistics. In a recent work, van Erven and Harremoës proved a Pythagorean inequality for Rényi divergences on $\alpha $ -convex sets under the assumption that the forward projection exists. Continuing this study, a sufficient condition for the existence of a forward projection is proved for probability measures on a general alphabet. For $\alpha \in (1, \infty )$ , the proof relies on a new Apollonius theorem for the Hellinger divergence, and for $\alpha \in (0,1)$ , the proof relies on the Banach–Alaoglu theorem from the functional analysis. Further projection results are then obtained in the finite alphabet setting. These include a projection theorem on a specific $\alpha $ -convex set, which is termed an $\alpha $ -linear family, generalizing a result by Csiszár to $\alpha \neq 1$ . The solution to this problem yields a parametric family of probability measures, which turns out to be an extension of the exponential family, and it is termed an $\alpha $ -exponential family. An orthogonality relationship between the $\alpha $ -exponential and $\alpha $ -linear families is established, and it is used to turn the reverse projection on an $\alpha $ -exponential family into a forward projection on an $\alpha $ -linear family. This paper also proves a convergence result of an iterative procedure used to calculate the forward projection on an intersection of a finite number of $\alpha $ -linear families. [ABSTRACT FROM PUBLISHER]