Search

There is much interest in the topic of partial information decomposition, both in developing new algorithms and in developing applications. An algorithm, based on standard results from information geometry, was recently proposed by Niu and Quinn (2019). They considered the case of three scalar random variables from an exponential family, including both discrete distributions and a trivariate Gaussian distribution. The purpose of this article is to extend their work to the general case of multivariate Gaussian systems having vector inputs and a vector output. By making use of standard results from information geometry, explicit expressions are derived for the components of the partial information decomposition for this system. These expressions depend on a real-valued parameter which is determined by performing a simple constrained convex optimisation. Furthermore, it is proved that the theoretical properties of non-negativity, self-redundancy, symmetry and monotonicity, which were proposed by Williams and Beer (2010), are valid for the decomposition I ig derived herein. Application of these results to real and simulated data show that the I ig algorithm does produce the results expected when clear expectations are available, although in some scenarios, it can overestimate the level of the synergy and shared information components of the decomposition, and correspondingly underestimate the levels of unique information. Comparisons of the I ig and I dep (Kay and Ince, 2018) methods show that they can both produce very similar results, but interesting differences are provided. The same may be said about comparisons between the I ig and I mmi (Barrett, 2015) methods. [ABSTRACT FROM AUTHOR]

In scientific problems, an appropriate statistical model often involves a large number of canonical parameters. Often times, the quantities of scientific interest are real-valued functions of these canonical parameters. Statistical inference for a specified function of the canonical parameters can be carried out via the Bayesian approach by simply using the posterior distribution of the specified function of the parameter of interest. Frequentist inference is usually based on the profile likelihood for the parameter of interest. When the likelihood function is analytical, computing the profile likelihood is simply a constrained optimization problem with many numerical algorithms available. However, for hierarchical models, computing the likelihood function and hence the profile likelihood function is difficult because of the high-dimensional integration involved. We describe a simple computational method to compute profile likelihood for any specified function of the parameters of a general hierarchical model using data doubling. We provide a mathematical proof for the validity of the method under regularity conditions that assure that the distribution of the maximum likelihood estimator of the canonical parameters is non-singular, multivariate, and Gaussian. [ABSTRACT FROM AUTHOR]

In the scenario of device-free localization under multiple effects, the accuracy of localization based on compressed sensing theory is severely affected. Most existing localization techniques directly ignore multiple path effects. However, it is not practical to ignore the multiple path effect due to its high signal strength, which can provide localization information. In this paper, we formulate the sensing matrix optimization problem in compressed sensing for device-free localization scenarios based on multiple reflections. To solve this problem, we model it as a constrained combinatorial optimization problem and propose a hybrid meta-heuristic algorithm. First, smart reflection surfaces and virtual node models are used to construct the desired communication links. Second, we iteratively improve the properties of the measurement matrix by using K-means clustering to obtain reasonable thresholds, and use a meta-heuristic algorithm to optimize the sensing matrix. Finally, the simulation results show that the proposed method efficiently optimizes the sensing matrix and achieves fast and high-precision localization while conserving communication resources. [ABSTRACT FROM AUTHOR]

Quantum linear system algorithms (QLSAs) have the potential to speed up algorithms that rely on solving linear systems. Interior point methods (IPMs) yield a fundamental family of polynomial-time algorithms for solving optimization problems. IPMs solve a Newton linear system at each iteration to compute the search direction; thus, QLSAs can potentially speed up IPMs. Due to the noise in contemporary quantum computers, quantum-assisted IPMs (QIPMs) only admit an inexact solution to the Newton linear system. Typically, an inexact search direction leads to an infeasible solution, so, to overcome this, we propose an inexact-feasible QIPM (IF-QIPM) for solving linearly constrained quadratic optimization problems. We also apply the algorithm to ℓ 1 -norm soft margin support vector machine (SVM) problems, and demonstrate that our algorithm enjoys a speedup in the dimension over existing approaches. This complexity bound is better than any existing classical or quantum algorithm that produces a classical solution. [ABSTRACT FROM AUTHOR]

This paper studies the classical problem of detecting the locations of signal occurrences in a one-dimensional noisy measurement. Assuming the signal occurrences do not overlap, we formulate the detection task as a constrained likelihood optimization problem and design a computationally efficient dynamic program that attains its optimal solution. Our proposed framework is scalable, simple to implement, and robust to model uncertainties. We show by extensive numerical experiments that our algorithm accurately estimates the locations in dense and noisy environments, and outperforms alternative methods. [ABSTRACT FROM AUTHOR]

Large-scale industrial processes usually adopt centralized control and optimization methods. However, with the growth of the scale of industrial processes leading to increasing computational complexity, the online optimization capability of the double-layer model predictive control algorithm is challenged, exacerbating the difficulty of the widespread implementation of this algorithm in the industry. This paper proposes a distributed double-layer model predictive control algorithm based on dual decomposition for multivariate constrained systems to reduce the computational complexity of process control. Firstly, to solve the problem that the original dual decomposition method does not apply to constrained systems, two improved dual decomposition model prediction control methods are proposed: the dual decomposition method based on the quadratic programming in the subsystem and the dual decomposition method based on constraint zones, respectively. It is proved that the latter will certainly converge to the constraint boundaries with appropriate convergence factors for the controlled variables. The online optimization ability of the proposed two methods is compared in discussion and simulation, concluding that the dual decomposition method based on the constraint zones exhibits superior online optimization ability. Further, a distributed double-layer model predictive control algorithm with dual decomposition based on constraint zones is proposed. Different from the objective function of the original dual decomposition model predictive control, the proposed algorithm's dynamic control-layer objective function simultaneously tracks the steady-state optimization values of the controlled and manipulated variables, giving the optimal solution formulation of the optimization problem consisting of this objective function and the constraints. The algorithm proposed in this paper achieves the control goals while significantly reducing the computational complexity and has research significance for promoting the industrial implementation of double-layer model predictive control. [ABSTRACT FROM AUTHOR]

In addition to various estimation algorithms, the target localization accuracy in wireless sensor networks (WSNs) can also be improved from the perspective of geometry optimization. Note that existing placement strategies are mainly aimed at unconstrained deployment regions, i.e., the positions of sensors are arbitrary. In this paper, considering factors such as terrain, communication, and security, the optimal range-based sensor geometries under circular deployment region and minimum safety distance constraints are proposed. The geometry optimization problem is modeled as a constrained optimization problem, with a D-optimality-based (maximizing the determinant of FIM matrix) scalar function as the objective function and the irregular feasible deployment regions as the constraints. We transform the constrained optimization problem into an equivalent form using the introduced maximum feasible angle and separation angle, and discuss the optimal geometries based on the relationship between the minimum safety distance and the maximum feasible angle. We first consider optimal geometries for two and three sensors in the localization system, and then use their findings to extend the study to scenarios with arbitrary numbers of sensors and arbitrarily shaped feasible regions. Numerical simulation results are included to verify the theoretical conclusions. [ABSTRACT FROM AUTHOR]

Ordinal regression methods are widely used to predict the ordered labels of data, among which support vector ordinal regression (SVOR) methods are popular because of their good generalization. In many realistic circumstances, data are collected by a distributed network. In order to protect privacy or due to some practical constraints, data cannot be transmitted to a center for processing. However, as far as we know, existing SVOR methods are all centralized. In the above situations, centralized methods are inapplicable, and distributed methods are more suitable choices. In this paper, we propose a distributed SVOR (dSVOR) algorithm. First, we formulate a constrained optimization problem for SVOR in distributed circumstances. Since there are some difficulties in solving the problem with classical methods, we used the random approximation method and the hinge loss function to transform the problem into a convex optimization problem with constraints. Then, we propose subgradient-based algorithm dSVOR to solve it. To illustrate the effectiveness, we theoretically analyze the consensus and convergence of the proposed method, and conduct experiments on both synthetic data and a real-world example. The experimental results show that the proposed dSVOR could achieve close performance to that of the corresponding centralized method, which needs all the data to be collected together. [ABSTRACT FROM AUTHOR]

This paper introduces a direct method derived from the global radial basis function (RBF) interpolation over arbitrary collocation nodes occurring in variational problems involving functionals that depend on functions of a number of independent variables. This technique parameterizes solutions with an arbitrary RBF and transforms the two-dimensional variational problem (2DVP) into a constrained optimization problem via arbitrary collocation nodes. The advantage of this method lies in its flexibility in selecting between different RBFs for the interpolation and parameterizing a wide range of arbitrary nodal points. Arbitrary collocation points for the center of the RBFs are applied in order to reduce the constrained variation problem into one of a constrained optimization. The Lagrange multiplier technique is used to transform the optimization problem into an algebraic equation system. Three numerical examples indicate the high efficiency and accuracy of the proposed technique. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the bifurcations of solutions to a class of degenerate constrained optimization problems. This study was motivated by the Information Bottleneck and Information Distortion problems, which have been used to successfully cluster data in many different applications. In the problems we discuss in this paper, the distortion function is not a linear function of the quantizer. This leads to a challenging annealing optimization problem, which we recast as a fixed-point dynamics problem of a gradient flow of a related dynamical system. The gradient system possesses an S N symmetry due to its invariance in relabeling representative classes. Its flow hence passes through a series of bifurcations with specific symmetry breaks. Here, we show that the dynamical system related to the Information Bottleneck problem has an additional spurious symmetry that requires more-challenging analysis of the symmetry-breaking bifurcation. For the Information Bottleneck, we determine that when bifurcations occur, they are only of pitchfork type, and we give conditions that determine the stability of the bifurcating branches. We relate the existence of subcritical bifurcations to the existence of first-order phase transitions in the corresponding distortion function as a function of the annealing parameter, and provide criteria with which to detect such transitions. [ABSTRACT FROM AUTHOR]

Only the smell perception rule is considered in the butterfly optimization algorithm (BOA), which is prone to falling into a local optimum. Compared with the original BOA, an extra operator, i.e., color perception rule, is incorporated into the proposed hybrid-flash butterfly optimization algorithm (HFBOA), which makes it more in line with the actual foraging characteristics of butterflies in nature. Besides, updating the strategy of the control parameters by the logistic mapping is used in the HFBOA for enhancing the global optimal ability. The performance of the proposed method was verified by twelve benchmark functions, where the comparison experiment results show that the HFBOA converges quicker and has better stability for numerical optimization problems, which are compared with six state-of-the-art optimization methods. Additionally, the proposed HFBOA is successfully applied to six engineering constrained optimization problems (i.e., tubular column design, tension/compression spring design, cantilever beam design, etc.). The simulation results reveal that the proposed approach demonstrates superior performance in solving complex real-world engineering constrained tasks. [ABSTRACT FROM AUTHOR]

Biconvex programming (or inequality constrained biconvex optimization) is an important model in solving many engineering optimization problems in areas like machine learning and signal and information processing. In this paper, the partial exactness of the partial optimum for the penalty function of biconvex programming is studied. The penalty function is partially exact if the partial Karush–Kuhn–Tucker (KKT) condition is true. The sufficient and necessary partially local stability condition used to determine whether the penalty function is partially exact for a partial optimum solution is also proven. Based on the penalty function, an algorithm is presented for finding a partial optimum solution to an inequality constrained biconvex optimization, and its convergence is proven under some conditions. [ABSTRACT FROM AUTHOR]

Information theory provides robust measures of multivariable interdependence, but classically does little to characterize the multivariable relationships it detects. The Partial Information Decomposition (PID) characterizes the mutual information between variables by decomposing it into unique, redundant, and synergistic components. This has been usefully applied, particularly in neuroscience, but there is currently no generally accepted method for its computation. Independently, the Information Delta framework characterizes non-pairwise dependencies in genetic datasets. This framework has developed an intuitive geometric interpretation for how discrete functions encode information, but lacks some important generalizations. This paper shows that the PID and Delta frameworks are largely equivalent. We equate their key expressions, allowing for results in one framework to apply towards open questions in the other. For example, we find that the approach of Bertschinger et al. is useful for the open Information Delta question of how to deal with linkage disequilibrium. We also show how PID solutions can be mapped onto the space of delta measures. Using Bertschinger et al. as an example solution, we identify a specific plane in delta-space on which this approach's optimization is constrained, and compute it for all possible three-variable discrete functions of a three-letter alphabet. This yields a clear geometric picture of how a given solution decomposes information. [ABSTRACT FROM AUTHOR]

Divergence functions play a relevant role in Information Geometry as they allow for the introduction of a Riemannian metric and a dual connection structure on a finite dimensional manifold of probability distributions. They also allow to define, in a canonical way, a symplectic structure on the square of the above manifold of probability distributions, a property that has received less attention in the literature until recent contributions. In this paper, we hint at a possible application: we study Lagrangian submanifolds of this symplectic structure and show that they are useful for describing the manifold of solutions of the Maximum Entropy principle. [ABSTRACT FROM AUTHOR]

The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y. It is defined as a constrained optimization problem that maximizes the information the representation has about the task, I (T ; Y) , while ensuring that a certain level of compression r is achieved (i.e., I (X ; T) ≤ r ). For practical reasons, the problem is usually solved by maximizing the IB Lagrangian (i.e., L IB (T ; β) = I (T ; Y) − β I (X ; T) ) for many values of β ∈ [ 0 , 1 ] . Then, the curve of maximal I (T ; Y) for a given I (X ; T) is drawn and a representation with the desired predictability and compression is selected. It is known when Y is a deterministic function of X, the IB curve cannot be explored and another Lagrangian has been proposed to tackle this problem: the squared IB Lagrangian: L sq − IB (T ; β sq) = I (T ; Y) − β sq I (X ; T) 2 . In this paper, we (i) present a general family of Lagrangians which allow for the exploration of the IB curve in all scenarios; (ii) provide the exact one-to-one mapping between the Lagrange multiplier and the desired compression rate r for known IB curve shapes; and (iii) show we can approximately obtain a specific compression level with the convex IB Lagrangian for both known and unknown IB curve shapes. This eliminates the burden of solving the optimization problem for many values of the Lagrange multiplier. That is, we prove that we can solve the original constrained problem with a single optimization. [ABSTRACT FROM AUTHOR]

Programs are under continuous attack for disclosing secret information, and defending against these attacks is becoming increasingly vital. An attractive approach for protection is to measure the amount of secret information that might leak to attackers. A fundamental issue in computing information leakage is that given a program and attackers with various knowledge of the secret information, what is the maximum amount of leakage of the program? This is called channel capacity. In this paper, two notions of capacity are defined for concurrent probabilistic programs using information theory. These definitions consider intermediate leakage and the scheduler effect. These capacities are computed by a constrained nonlinear optimization problem. Therefore, an evolutionary algorithm is proposed to compute the capacities. Single preference voting and dining cryptographers protocols are analyzed as case studies to show how the proposed approach can automatically compute the capacities. The results demonstrate that there are attackers who can learn the whole secret of both the single preference protocol and dining cryptographers protocol. The proposed evolutionary algorithm is a general approach for computing any type of capacity in any kind of program. [ABSTRACT FROM AUTHOR]

In this paper, the design of a data-rate constrained observer for a dynamical system is presented. This observer is designed to function both in discrete time and continuous time. The system is connected to a remote location via a communication channel which can transmit limited amounts of data per unit of time. The objective of the observer is to provide estimates of the state at the remote location through messages that are sent via the channel. The observer is designed such that it is robust toward losses in the communication channel. Upper bounds on the required communication rate to implement the observer are provided in terms of the upper box dimension of the state space and an upper bound on the largest singular value of the system's Jacobian. Results that provide an analytical bound on the required minimum communication rate are then presented. These bounds are obtained by using the Lyapunov dimension of the dynamical system rather than the upper box dimension in the rate. The observer is tested through simulations for the Lozi map and the Lorenz system. For the Lozi map, the Lyapunov dimension is computed. For both systems, the theoretical bounds on the communication rate are compared to the simulated rates. [ABSTRACT FROM AUTHOR]