Search

Despite their remarkable performance, deep learning models still lack robustness guarantees, particularly in the presence of adversarial examples. This significant vulnerability raises concerns about their trustworthiness and hinders their deployment in critical domains that require certified levels of robustness. In this paper, we introduce an information geometric framework to establish precise robustness criteria for l 2 white-box attacks in a multi-class classification setting. We endow the output space with the Fisher information metric and derive criteria on the input–output Jacobian to ensure robustness. We show that model robustness can be achieved by constraining the model to be partially isometric around the training points. We evaluate our approach using MNIST and CIFAR-10 datasets against adversarial attacks, revealing its substantial improvements over defensive distillation and Jacobian regularization for medium-sized perturbations and its superior robustness performance to adversarial training for large perturbations, all while maintaining the desired accuracy. [ABSTRACT FROM AUTHOR]

The space of all probability measures having positive density function on a connected compact smooth manifold M, denoted by P(M)$\mathcal {P}(M)$, carries the Fisher information metric G. We define the geometric mean of probability measures by the aid of which we investigate information geometry of P(M)$\mathcal {P}(M)$, equipped with G. We show that a geodesic segment joining arbitrary probability measures μ1 and μ2 is expressed by using the normalized geometric mean of its endpoints. As an application, we show that any two points of P(M)$\mathcal {P}(M)$ can be joined by a unique geodesic. Moreover, we prove that the function ℓ defined by ℓ(μ1,μ2):=2arccos∫Mp1p2dλ$\ell \!\big (\mu _1, \mu _2\big):=2\arccos \int \nolimits _M \sqrt {p_1p_2}\,d\lambda$, μi=piλ$\mu _i=p_i \lambda$, i=1,2$i=1,2$, gives the Riemannian distance function on P(M)$\mathcal {P}(M)$. It is shown that geodesics are all minimal. [ABSTRACT FROM AUTHOR]

We consider evolutionary games with a continuous trait space where the replicator dynamics are restricted to the manifold of multivariate Gaussian distributions. We demonstrate that replicator dynamics are gradient flows with respect to the Fisher information metric. The potential function for these gradient flows is closely related to the mean fitness. Our findings extend previous results on natural gradient ascent in evolutionary games with a finite strategy set. Throughout the paper we pursue an information-geometric point of view on evolutionary games. This sheds a new light on the replicator dynamics as a learning process, realizing the compromise between maximization of the mean fitness and preservation of the diversity. • An information-geometric study of replicator dynamics. • Replicator dynamics follow natural gradient. • A novel result for evolutionary games with a continuous trait space. • A compromise between maximization of the mean fitness and preservation of diversity. [ABSTRACT FROM AUTHOR]

Background: Almost 16,000 human long non-coding RNA (lncRNA) genes have been identified in the GENCODE project. However, the function of most of them remains to be discovered. The function of lncRNAs and other novel genes can be predicted by identifying significantly enriched annotation terms in already annotated genes that are co-expressed with the lncRNAs. However, such approaches are sensitive to the methods that are used to estimate the level of co-expression. Results: We have tested and compared two well-known statistical metrics (Pearson and Spearman) and two geometrical metrics (Sobolev and Fisher) for identification of the co-expressed genes, using experimental expression data across 19 normal human tissues. We have also used a benchmarking approach based on semantic similarity to evaluate how well these methods are able to predict annotation terms, using a well-annotated set of protein-coding genes. Conclusion: This work shows that geometrical metrics, in particular in combination with the statistical metrics, will predict annotation terms more efficiently than traditional approaches. Tests on selected lncRNAs confirm that it is possible to predict the function of these genes given a reliable set of expression data. The software used for this investigation is freely available. [ABSTRACT FROM AUTHOR]

In many-body physics, renormalization techniques are used to extract aspects of a statistical or quantum state that are relevant at large scale, or for low energy experiments. Recent works have proposed that these features can be formally identified as those perturbations of the states whose distinguishability most resist coarse-graining. Here, we examine whether this same strategy can be used to identify important features of an unlabeled dataset. This approach indeed results in a technique very similar to kernel PCA (principal component analysis), but with a kernel function that is automatically adapted to the data, or "learned". We test this approach on handwritten digits, and find that the most relevant features are significantly better for classification than those obtained from a simple Gaussian kernel. [ABSTRACT FROM AUTHOR]

For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to α-divergence, which includes the special cases of Kullback-Leibler divergence, the Hellinger distance, and essentially χ2-divergence. The asymptotic expansion of the risk is given with respect to sample sizes up to order n-2. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model. [ABSTRACT FROM AUTHOR]

Tracking of moving objects in real-time scenes is a challenging research problem in computer vision. This is due to incessant live changes in the object features, background, occlusions, and illumination deviations occurring at different instances in the scene. With the objective of tracking visual objects in intricate videos, this paper presents a new color-independent tracking approach, the contributions of which are threefold. First, the illumination level of the sequences is maintained constant using fast discrete curvelet transform. Fisher information metric is calculated based on a cumulative score by comparing the template patches with a reference template at different timeframes. This metric is used for quantifying distances between the consecutive frame histogram distributions. Then, a novel iterative algorithm called conditionally adaptive multiple template update is proposed to regulate the object templates for handling dynamic occlusions effectively. The proposed method is evaluated on a set of extensive challenging benchmark datasets. Experimental results in terms of Center Location Error (CLE), Tracking Success Score (TSS), and Occlusion Success Score (OSS) show that the proposed method competes well with other relevant state-of-the-art tracking methods. [ABSTRACT FROM AUTHOR]

The Onsager-Machlup (OM) relaxation process, which provides a prototypical example of linear nonequilibrium thermodynamics, is characterized in terms of information geometry over the two-dimensional temperature-time parameter space. After deriving the probability distribution function for the OM process under the harmonic potential within a theoretical framework based on a transformation into imaginary-time quantum mechanics, the Fisher information metric is explicitly obtained. Differential geometry in the temperature-time space then gives the scalar curvature R = - 1 , thus specifying the OM process in a topological manner. [ABSTRACT FROM AUTHOR]

Recently the topic of financial network dynamics has gained renewed interest from researchers in the field of empirical systemic risk measurements. We refer to this type of network analysis as information flow networks analysis (IFNA). This paper proposes a new method that applies Fisher information metric to the evolutionary dynamics of financial networks using IFNA. Our paper is the first to apply the Fisher information metric to a set of financial time series. We introduce Evolution Index (EI) as a measure of systemic risk in financial networks. It is shown, for concrete networks with actual data of several stock markets, that the EI can be implemented as a measure of fitness of the stock market and as a leading indicator of systemic risk. [ABSTRACT FROM AUTHOR]

We describe four algorithms for neural network training, each adapted to different scalability constraints. These algorithms are mathematically principled and invariant under a number of transformations in data and network representation, from which performance is thus independent. These algorithms are obtained from the setting of differential geometry, and are based on either the natural gradient using the Fisher information matrix, or on Hessian methods, scaled down in a specific way to allow for scalability while keeping some of their key mathematical properties. [ABSTRACT FROM AUTHOR]

Using Busemann function of an Hadamard manifold X we define the barycenter map from the space 풫+(∂X, dθ) of probability measures having positive density on the ideal boundary ∂X to X. The space 풫+(∂X, dθ) admits geometrically a fiber space structure over X from Fisher information geometry. Following the arguments in [E. Douady and C. Earle, Conformally natural extension of homeomorphisms of the circle, Acta Math. 157 (1986) 23-48; G. Besson, G. Courtois and S. Gallot, Entropies et rigidités des espaces localement symétriques de coubure strictement négative, Geom. Funct. Anal. 5 (1995) 731-799; Minimal entropy and Mostow's rigidity theorems, Ergodic Theory Dynam. Systems 16 (1996) 623-649], we exhibit that under certain geometrical hypotheses a homeomorphism Φ of the ideal boundary ∂X induces, by the aid of push-forward, an isometry of X whose extension is Φ. [ABSTRACT FROM AUTHOR]

In this paper, we consider a deformed exponential family called F-exponential family. Then on a F-exponential family, we describe a dually flat structure called the X-geometry and we obtain that this X-geometry is the conformal flattening of the (F, G)-geometry for suitable choices of F and G. The potential functions and the dual coordinates are given for the dually flat structure. The dual coordinates are defined using the F-escort probability distributions. Thus one can say that the manifold of F-escort probability distributions is dually flat by the conformal flattening of the (F, G)-geometry defined on the manifold of usual probability distributions. Further a maximum likelihood estimator is defined using the F-escort probability distribution. [ABSTRACT FROM AUTHOR]

Abstract: We consider inference on the eigenvalues of the covariance matrix of a multivariate normal distribution. The family of multivariate normal distributions with a fixed mean is seen as a Riemannian manifold with Fisher information metric. Two submanifolds naturally arises: one is the submanifold given by the fixed eigenvectors of the covariance matrix; the other is the one given by the fixed eigenvalues. We analyze the geometrical structures of these manifolds such as metric, embedding curvature under e-connection or m-connection. Based on these results, we study (1) the bias of the sample eigenvalues, (2) the asymptotic variance of estimators, (3) the asymptotic information loss caused by neglecting the sample eigenvectors, (4) the derivation of a new estimator that is natural from a geometrical point of view. [Copyright &y& Elsevier]

Neural networks play a growing role in many scientific disciplines, including physics. Variational autoencoders (VAEs) are neural networks that are able to represent the essential information of a high dimensional data set in a low dimensional latent space, which have a probabilistic interpretation. In particular, the so-called encoder network, the first part of the VAE, which maps its input onto a position in latent space, additionally provides uncertainty information in terms of variance around this position. In this work, an extension to the autoencoder architecture is introduced, the FisherNet. In this architecture, the latent space uncertainty is not generated using an additional information channel in the encoder but derived from the decoder by means of the Fisher information metric. This architecture has advantages from a theoretical point of view as it provides a direct uncertainty quantification derived from the model and also accounts for uncertainty cross-correlations. We can show experimentally that the FisherNet produces more accurate data reconstructions than a comparable VAE and its learning performance also apparently scales better with the number of latent space dimensions. [ABSTRACT FROM AUTHOR]

Suppose we have n different types of self-replicating entity, with the population P i of the ith type changing at a rate equal to P i times the fitness f i of that type. Suppose the fitness f i is any continuous function of all the populations P 1 , ... , P n . Let p i be the fraction of replicators that are of the ith type. Then p = (p 1 , ... , p n) is a time-dependent probability distribution, and we prove that its speed as measured by the Fisher information metric equals the variance in fitness. In rough terms, this says that the speed at which information is updated through natural selection equals the variance in fitness. This result can be seen as a modified version of Fisher's fundamental theorem of natural selection. We compare it to Fisher's original result as interpreted by Price, Ewens and Edwards. [ABSTRACT FROM AUTHOR]

The main characteristics of the significant wave height in an area of increased interest, the north Atlantic ocean, are studied based on satellite records and corresponding simulations obtained from the numerical wave prediction model WAM. The two data sets are analyzed by means of a variety of statistical measures mainly focusing on the distributions that they form. Moreover, new techniques for the estimation and minimization of the discrepancies between the observed and modeled values are proposed based on ideas and methodologies from a relatively new branch of mathematics, information geometry. The results obtained prove that the modeled values overestimate the corresponding observations through the whole study period. On the other hand, 2-parameter Weibull distributions fit well the data in the study. However, one cannot use the same probability density function for describing the whole study area since the corresponding scale and shape parameters deviate significantly for points belonging to different regions. This variation should be taken into account in optimization or assimilation procedures, which is possible by means of information geometry techniques. [ABSTRACT FROM AUTHOR]

Abstract: A complete Riemannian manifold X with negative curvature satisfying for some constants , is naturally mapped in the space of probability measures on the ideal boundary ∂X by assigning the Poisson kernels. We show that this map is embedding and the pull-back metric of the Fisher information metric by this embedding coincides with the original metric of X up to constant provided X is a rank one symmetric space of non-compact type. Furthermore, we give a geometric meaning of the embedding. [Copyright &y& Elsevier]

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions. [ABSTRACT FROM AUTHOR]

Stochastic optimisation in Riemannian manifolds, especially the Riemannian stochastic gradient method, has attracted much recent attention. The present work applies stochastic optimisation to the task of recursive estimation of a statistical parameter which belongs to a Riemannian manifold. Roughly, this task amounts to stochastic minimisation of a statistical divergence function. The following problem is considered: how to obtain fast, asymptotically efficient, recursive estimates, using a Riemannian stochastic optimisation algorithm with decreasing step sizes. In solving this problem, several original results are introduced. First, without any convexity assumptions on the divergence function, we proved that, with an adequate choice of step sizes, the algorithm computes recursive estimates which achieve a fast non-asymptotic rate of convergence. Second, the asymptotic normality of these recursive estimates is proved by employing a novel linearisation technique. Third, it is proved that, when the Fisher information metric is used to guide the algorithm, these recursive estimates achieve an optimal asymptotic rate of convergence, in the sense that they become asymptotically efficient. These results, while relatively familiar in the Euclidean context, are here formulated and proved for the first time in the Riemannian context. In addition, they are illustrated with a numerical application to the recursive estimation of elliptically contoured distributions. [ABSTRACT FROM AUTHOR]

The Fisher information metric provides a smooth family of probability measures with a Riemannian manifold structure, which is an object in information geometry. The information geometry of the gamma manifold associated with the family of gamma distributions has been well studied. However, only a few results are known for the generalized gamma family that adds an extra shape parameter. The present article gives some new results about the generalized gamma manifold. This paper also introduces an application in medical imaging that is the classification of Alzheimer's disease population. In the medical field, over the past two decades, a growing number of quantitative image analysis techniques have been developed, including histogram analysis, which is widely used to quantify the diffuse pathological changes of some neurological diseases. This method presents several drawbacks. Indeed, all the information included in the histogram is not used and the histogram is an overly simplistic estimate of a probability distribution. Thus, in this study, we present how using information geometry and the generalized gamma manifold improved the performance of the classification of Alzheimer's disease population. [ABSTRACT FROM AUTHOR]

Abstract: Let be an Hadamard manifold with ideal boundary ∂X. We can then define the map associated with Poisson kernel on X, where is the space of probability measures on ∂X, together with the Fisher information metric G. We make geometrical investigation of homothetic property and minimality of this map with respect to the metrics g and G. The map φ is shown to be a minimal homothetic embedding for a rank one symmetric space of noncompact type as well as for a nonsymmetric Damek–Ricci space. The following is also obtained. If φ is assumed to be homothetic and minimal, then, turns out to be an asymptotically harmonic, visibility manifold with the Poisson kernel being expressed in terms of the Busemann function. [Copyright &y& Elsevier]