Search

In this paper, we present a general framework that provides a comprehensive and uniform treatment of integral majorization inequalities for convex functions and finite signed measures. Along with new results, we present unified and simple proofs of classical statements. To apply our results, we deal with Hermite-Hadamard-Fejér-type inequalities and their refinements. We present a general method to refine both sides of Hermite-Hadamard-Fejér-type inequalities. The results of many papers on the refinement of the Hermite-Hadamard inequality, whose proofs are based on different ideas, can be treated in a uniform way by this method. Finally, we establish a necessary and sufficient condition for when a fundamental inequality of f-divergences can be refined by another f-divergence.

In this paper, we present a new refinement of Jensen’s inequality with applications in information theory. The refinement of Jensen’s inequality is obtained based on the general functional in the work of Popescu et al. As the applications in information theory, we provide new tighter bounds for Shannon’s entropy and some f-divergences.

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Others were the fruit of generalisation and the mathematical genius of bright minds like Rényi, Csizsár and Sibson. These powerful objects allow us to manipulate probabilities intuitively and seem always to be somehow connected to concrete settings in communication, coding or estimation theory. A common theme is: take a problem in one of these areas, try to control (upper or lower-bound) the expected value of some function of interest (often, probabilities of error) and, with enough work, an information measure appears as a fundamental limit of the problem. The most striking example of this is in Shannon's seminal paper in 1948: his purpose was to characterise the smallest possible expected length of a uniquely decodable encoding that compresses the realisations of a random variable. As he brilliantly proved, the smallest expected length one can hope for is the Entropy of the random variable. In establishing this connection, another quantity needed to be implicitly controlled: the Kraft's sum of the code. Seemingly unrelated before, these three objects joined forces in harmony to provide a beautiful and fundamental result. But why are they related? The answer seems to be: duality. Duality is an abstract notion commonly used in linear algebra and functional analysis. It has been expanded and generalised over the years. Several incarnations have been discovered throughout mathematics. One particular instance of this involves vector spaces: given two vector spaces and a "duality pairing" one can jump from one space to the other (its dual) through Legendre-Fenchel-like transforms. In the most common settings in Information Theory, the two spaces and the pairing are, respectively: 1) the space of (probability)measures defined on X; 2) the space of bounded functions defined on X; 3) the Lebesgue integral of the function (the expected value of the function if the measure is a probability measure). Once these are set, Legendre-Fenchel-like transforms allow us to connect a) a functional acting on the space described in 1), b) a functional acting on the space described in 2) and the anchor point is c) the (expected) value described in 3). These three pieces (a), b) and c)) represent the actors of many of the results provided in Information Theory. Once they are found, one usually bounds the functional described in b) and obtains a bound connecting the expected value and the functional of measures (e.g., an information measure). Going back to Shannon's result, fixed a random variable (and thus, a probability measure) and selected the function to be the length of a code: the functional a) is the Shannon Entropy of the source; the functional b) is the Kraft sum of the code; the pairing c) is the expected length of the code. We explore this connection and this pattern throughout the thesis. We will see how it can be found in notable results like Coding Theorems for one-to-one codes, Campbell's Coding Theorem, Arikan's Guessing Theorem, Fano-like and Transportation-Cost Inequalities and so on. Moreover, unearthing the pattern allows us to generalise it to other information measures and apply the technique in a variety of fields, including Learning Theory, Estimation Theory and Hypothesis Testing.

Funder: Cantab Capital Institute for the Mathematics of Information, Funder: National Physical Laboratory; doi: https://doi.org/10.13039/501100007851, Funder: Alan Turing Institute; doi: https://doi.org/10.13039/100012338, We study variational regularisation methods for inverse problems with imperfect forward operators whose errors can be modelled by order intervals in a partial order of a Banach lattice. We carry out analysis with respect to existence and convex duality for general data fidelity terms and regularisation functionals. Both for a priori and a posteriori parameter choice rules, we obtain convergence rates of the regularised solutions in terms of Bregman distances. Our results apply to fidelity terms such as Wasserstein distances, φ-divergences, norms, as well as sums and infimal convolutions of those.

Different terms such as variability, inequality and dispersion, which occur in various engineering problems and scientific fields, in mathematics are most simply described by the concept of majorization, a powerful mathematical tool which allows one to see the existing connections between vectors that can be used. In majorization theory, majorization inequalities play an important role. In this paper, using known properties of superquadratic functions, extensions and improvements of majorization inequalities are obtained. Also their converse inequalities are presented. For superquadratic functions, which are not convex, results analog ones for convex functions are presented. For superquadratic functions which are convex, improvements are given. At the end, applications to $$\varphi $$ -divergences are discussed. New estimates for the Renyi entropy are derivated.

The relative entropy and chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these relations, their information-theoretic applications, and some generalizations pertaining to the rich class of $f$-divergences. Applications that are studied in this paper refer to lossless compression, the method of types and large deviations, strong~data-processing inequalities, bounds on contraction coefficients and maximal correlation, and the convergence rate to stationarity of a type of discrete-time Markov chains., Comment: Published in the Entropy journal, May 18, 2020. Journal version (open access) is available at https://www.mdpi.com/1099-4300/22/5/563

International Zurich Seminar on Information and Communication (IZS 2020). Proceedings

International audience; Community detection is a research area from network science dealing withthe investigation of complex networks such as social or biological networks, aimingto identify subgroups (communities) of entities (nodes) thatare more closely relatedto each other inside the community than with the remaining entities in the network.Various community detection algorithms have been developed and used in the literaturehowever evaluating community structures that have been automatically detected isa challenging task due to varying results in different scenarios.Current evaluationmeasures that compare extracted community structures with the reference structure orground truth suffer from various drawbacks; some of them having beenpoint out in theliterature. Information theoretic measures form a fundamental classin this domain andhave recently received increasing interest. However even the well employed measures(NVI and NID) also share some limitations, particularly they are biased toward thenumber of communities in the network. The main contribution ofthis paper is tointroduce a new measure that overcomes this limitation while holding the importantproperties of measures. We review the mathematical properties of our measure based on¿2divergence inspired fromf-divergence measures in information theory. Theoreticalproperties as well as experimental results in various scenarios show the superiority of theproposed measure to evaluate community detection over the ones from the literature.

In this paper we give a refinement of Jensen’s integral inequality and its generalization for linear functionals. We also present some applications in Information Theory.

This paper is focused on $f$-divergences, consisting of three main contributions. The first one introduces integral representations of a general $f$-divergence by means of the relative information spectrum. The second part provides a new approach for the derivation of $f$-divergence inequalities, and it exemplifies their utility in the setup of Bayesian binary hypothesis testing. The last part of this paper further studies the local behavior of $f$-divergences., Comment: Final edits before publication. To appear in the Entropy journal, special issue on Entropy and Information Inequalities, May 2018

Let X be a real-valued Lévy process under P in its natural filtration. The minimal entropy martingale measure is defined as an absolutely continuous martingale measure that minimizes the relative entropy with respect to P. We show in this paper that the sufficient conditions for its existence, known in literature, are also necessary and give an explicit formula for the infimum of the relative entropy.

This note provides a bibliography of investigations based on or related to divergence measures for theoretical and applied inference problems.

The goal of binary classification is to estimate a discriminant function $\gamma$ from observations of covariate vectors and corresponding binary labels. We consider an elaboration of this problem in which the covariates are not available directly but are transformed by a dimensionality-reducing quantizer $Q$. We present conditions on loss functions such that empirical risk minimization yields Bayes consistency when both the discriminant function and the quantizer are estimated. These conditions are stated in terms of a general correspondence between loss functions and a class of functionals known as Ali-Silvey or $f$-divergence functionals. Whereas this correspondence was established by Blackwell [Proc. 2nd Berkeley Symp. Probab. Statist. 1 (1951) 93--102. Univ. California Press, Berkeley] for the 0--1 loss, we extend the correspondence to the broader class of surrogate loss functions that play a key role in the general theory of Bayes consistency for binary classification. Our result makes it possible to pick out the (strict) subset of surrogate loss functions that yield Bayes consistency for joint estimation of the discriminant function and the quantizer., Comment: Published in at http://dx.doi.org/10.1214/08-AOS595 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

We propose a new viewpoint-based simplification method for polygonal meshes, driven by several f-divergences such as Kullback-Leibler, Hellinger and Chi-Square. These distances are a measure of discrimination between probability distributions. The Kullback-Leibler distance between the projected and the actual area distributions of the polygons in the scene already has been used as a measure of viewpoint quality. In this paper, we use the variation in those viewpoint distances to determine the error introduced by an edge collapse. We apply the best half-edge collapse as a decimation criterion. The approximations produced by our method are close to the original model in terms of both visual and geometric criteria. Unlike many pure visibility-driven methods, our new approach does not completely remove hidden interiors in order to increase the visual quality of the simplified models. This makes our approach more suitable for applications which require exact geometry tolerance but also require high visual quality. © 2008 Elsevier Inc. All rights reserved.

Ein wichtiges Gebiet der Finanzmathematik ist die Bestimmung von Auszahlungsprofilen, die den erwarteten Nutzen eines Agenten unter einer Budgetrestriktion maximieren. Wir charakterisieren optimale Auszahlungsprofile für einen Agenten, der unsicher ist in Bezug auf das genaue Marktmodell. Der hier benutzte Dualitätsansatz führt zu einem Minimierungsproblem für bestimmte konvexe Funktionale über zwei Mengen von Wahrscheinlichkeitsmaßen, das wir zunächst lösen müssen. Schließlich führen wir noch eine zweite Restriktion ein, die das Risiko beschränkt, das der Agent eingehen darf. Wir gehen dabei wie folgt vor: Kapitel 1. Wir betrachten das Problem, die f-Divergenz f(P|Q) über zwei Mengen von Wahrscheinlichkeitsmaßen zu minimieren, wobei f eine konvexe Funktion ist. Wir zeigen, dass unter der Bedingung "f( undendlich ) / undendlich = undendlich" Minimierer existieren, falls die erste Menge abgeschlossen und die zweite schwach kompakt ist. Außerdem zeigen wir, dass unter der Bedingung "f( undendlich ) / undendlich = 0" ein Minimierer in einer erweiterten Klasse von Martingalmaßen existiert, falls die zweite Menge schwach kompakt ist. Kapitel 2. Die Existenzresultate aus dem ersten Kapitel implizieren die Existenz eines Auszahlungsprofils, das das robuste Nutzenfunktional inf E_Q[u(X)] über eine Menge von finanzierbaren Auszahlungen maximiert, wobei das Infimum über eine Menge von Modellmaßen betrachtet wird. Die entscheidende Idee besteht darin, die minimierenden Maße aus dem ersten Kapitel als gewisse "worst-case"-Maße zu identifizieren. Kapitel 3. Schließlich fordern wir, dass das Risiko der Auszahlungsprofile beschränkt ist. Wir lösen das robuste Problem in einem unvollständigen Marktmodell für Nutzenfunktionen, die nur auf der positiven Halbachse definiert sind. In einem Beispiel vergleichen wir das optimale Auszahlungsprofil unter der Risikorestriktion mit den optimalen Auszahlungen ohne eine solche Restriktion und unter einer Value-at-Risk-Nebenbedingung. Finding payoff profiles that maximize the expected utility of an agent under some budget constraint is a key issue in financial mathematics. We characterize optimal contingent claims for an agent who is uncertain about the market model. The dual approach that we use leads to a minimization problem for a certain convex functional over two sets of measures, which we first have to solve. Finally, we incorporate a second constraint that limits the risk that the agent is allowed to take. We proceed as follows: Chapter 1. Given a convex function f, we consider the problem of minimizing the f-divergence f(P|Q) over these two sets of measures. We show that, if the first set is closed and the second set is weakly compact, a minimizer exists if f( infinity ) / infinity = infinity. Furthermore, we show that if the second set of measures is weakly compact and f( infinifty ) / infinity = 0, then there is a minimizer in a class of extended martingale measures. Chapter 2. The existence results in Chapter 1 lead to the existence of a contingent claim which maximizes the robust utility functional inf E_Q[u(X)] over some set of affordable contingent claims, where the infimum is taken over a set of subjective or modell measures. The key idea is to identify the minimizing measures from the first chapter as certain worst case measures. Chapter 3. Finally, we require the risk of the contingent claims to be bounded. We solve the robust problem in an incomplete market for a utility function that is only defined on the positive halfline. In an example we compare the optimal claim under this risk constraint with the optimal claims without a risk constraint and under a value-at-risk constraint.

Aquest treball està enmarcat en el context de gràfics per computador partint de la intersecció de tres camps: rendering, teoria de la informació, i complexitat.Inicialment, el concepte de complexitat d'una escena es analitzat considerant tres perspectives des d'un punt de vista de la visibilitat geomètrica: complexitat en un punt interior, complexitat d'una animació, i complexitat d'una regió. L'enfoc principal d'aquesta tesi és l'exploració i desenvolupament de nous criteris de refinament pel problema de la il·luminació global. Mesures de la teoria de la informació basades en la entropia de Shannon i en la entropia generalitzada de Harvda-Charvát-Tsallis, conjuntament amb les f-divergències, són analitzades com a nuclis del refinement. Mostrem com ens aporten una rica varietat d'eficients i altament discriminatòries mesures que són aplicables al rendering en els seus enfocs de pixel-driven (ray-tracing) i object-space (radiositat jeràrquica).Primerament, basat en la entropia de Shannon, es defineixen un conjunt de mesures de qualitat i contrast del pixel. S'apliquen al supersampling en ray-tracing com a criteris de refinement, obtenint un algorisme nou de sampleig adaptatiu basat en entropia, amb un alt rati de qualitat versus cost. En segon lloc, basat en la entropia generalitzada de Harvda-Charvát-Tsallis, i en la informació mutua generalitzada, es defineixen tres nous criteris de refinament per la radiositat jeràrquica. En correspondencia amb tres enfocs clàssics, es presenten els oracles basats en la informació transportada, el suavitzat de la informació, i la informació mutua, amb resultats molt significatius per aquest darrer. Finalment, tres membres de la familia de les f-divergències de Csiszár's (divergències de Kullback-Leibler, chi-square, and Hellinger) son analitzats com a criteris de refinament mostrant bons resultats tant pel ray-tracing com per la radiositat jeràrquica., This work is framed within the context of computer graphics starting out from the intersection of three fields: rendering, information theory, and complexity.Initially, the concept of scene complexity is analysed considering three perspectives from a geometric visibility point of view: complexity at an interior point, complexity of an animation, and complexity of a region. The main focus of this dissertation is the exploration and development of new refinement criteria for the global illumination problem. Information-theoretic measures based on Shannon entropy and Harvda-Charvát-Tsallis generalised entropy, together with f-divergences, are analysed as kernels of refinement. We show how they give us a rich variety of efficient and highly discriminative measures which are applicable to rendering in its pixel-driven (ray-tracing) and object-space (hierarchical radiosity) approaches.Firstly, based on Shannon entropy, a set of pixel quality and pixel contrast measures are defined. They are applied to supersampling in ray-tracing as refinement criteria, obtaining a new entropy-based adaptive sampling algorithm with a high rate quality versus cost. Secondly, based on Harvda-Charvát-Tsallis generalised entropy, and generalised mutual information, three new refinement criteria are defined for hierarchical radiosity. In correspondence with three classic approaches, oracles based on transported information, information smoothness, and mutual information are presented, with very significant results for the latter. And finally, three members of the family of Csiszár's f-divergences (Kullback-Leibler, chi-square, and Hellinger divergences) are analysed as refinement criteria showing good results for both ray-tracing and hierarchical radiosity.