Search

In this paper, we introduce a novel approach to multi-modal optimization by enhancing the recently developed kinetic-based optimization (KBO) method with genetic dynamics (GKBO). The proposed method targets objective functions with multiple global minima, addressing a critical need in fields like engineering design, machine learning, and bioinformatics. By incorpo rating leader-follower dynamics and localized interactions, the algorithm efficiently navigates high-dimensional search spaces to detect multiple optimal solutions. After providing a binary description, a mean-field approximation is derived, and different numerical experiments are conducted to validate the results.

We introduce CBXPy and ConsensusBasedX.jl, Python and Julia implementations of consensus-based interacting particle systems (CBX), which generalise consensus-based optimization methods (CBO) for global, derivative-free optimisation. The raison d'\^etre of our libraries is twofold: on the one hand, to offer high-performance implementations of CBX methods that the community can use directly, while on the other, providing a general interface that can accommodate and be extended to further variations of the CBX family. Python and Julia were selected as the leading high-level languages in terms of usage and performance, as well as their popularity among the scientific computing community. Both libraries have been developed with a common ethos, ensuring a similar API and core functionality, while leveraging the strengths of each language and writing idiomatic code., Comment: 7 pages, 3 figures

We give an overview of time-continuous pedestrian models with a focus on data-driven modelling. Starting from pioneer, reactive force-based models we move forward to modern, active pedestrian models with sophisticated collision-avoidance and anticipation techniques through optimisation problems. The overview focuses on the mathematical aspects of the models and their different components. We include methods used for data-based calibration of model parameters, hybrid approaches incorporating neural networks, and purely data-based models fitted by deep learning. The conclusion outlines some development perspectives that we expect to grow in the coming years., Comment: 26 pages; chapter accepted for publication in Crowd Dynamics (vol. 4)

We propose and analyse a variant of the recently introduced kinetic based optimization method that incorporates ideas like survival-of-the-fittest and mutation strategies well-known from genetic algorithms. Thus, we provide a first attempt to reach out from the class of consensus/kinetic-based algorithms towards genetic metaheuristics. Different generations of genetic algorithms are represented via two species identified with different labels, binary interactions are prescribed on the particle level and then we derive a mean-field approximation in order to analyse the method in terms of convergence. Numerical results underline the feasibility of the approach and show in particular that the genetic dynamics allows to improve the efficiency, of this class of global optimization methods in terms of computational cost.

We suggest a global perspective on dynamic network flow problems that takes advantage of the similarities to port-Hamiltonian dynamics. Dynamic minimum cost flow problems are formulated as open-loop optimal control problems for general port-Hamiltonian systems with possibly state-dependent system matrices. We prove well-posedness of these systems and characterize optimal controls by the first-order optimality system, which is the starting point for the derivation of an adjoint-based gradient descent algorithm. Our theoretical analysis is complemented by a proof of concept, where we apply the proposed algorithm to static minimum cost flow problems and dynamic minimum cost flow problems on a simple directed acyclic graph. We present numerical results to validate the approach.

We give a short overview of advantages and drawbacks of the classical formulation of minimum cost network flow problems and solution techniques, to motivate a reformulation of classical static minimum cost network flow problems as optimal control problems constrained by port-Hamiltonian systems (pHS). The first-order optimality system for the port-Hamiltonian system-constrained optimal control problem is formally derived. Then we propose a gradient-based algorithm to find optimal controls. The port-Hamiltonian system formulation naturally conserves flow and supports a wide array of further modeling options as, for example, node reservoirs, flow dependent costs, leaking pipes (dissipation) and coupled sub-networks (ports). They thus provide a versatile alternative to state-of-the art approaches towards dynamic network flow problems, which are often based on computationally costly time-expanded networks. We argue that this opens the door for a plethora of modeling options and solution approaches for network flow problems.

We consider an optimal control problem with tracking-type cost functional constrained by the Cattaneo equation, which is a well-known model for delayed heat transfer. In particular, we are interested the asymptotic behaviour of the optimal control problems for a vanishing delay time $\tau \rightarrow 0$. First, we show the convergence of solutions of the Cattaneo equation to the ones of the heat equation. Assuming the same right-hand side and compatible initial conditions for the equations, we prove a linear convergence rate. Moreover, we show linear convergence of the optimal states and optimal controls for the Cattaneo equation towards the ones for the heat equation. We present numerical results for both, the forward and the optimal control problem confirming these linear convergence rates.

We derive a minimal port-Hamiltonian formulation of a general class of interacting particle systems driven by alignment and potential-based force dynamics which include the Cucker-Smale model with potential interaction and the second order Kuramoto model. The port-Hamiltonian structure allows to characterize conserved quantities such as Casimir functions as well as the long-time behaviour using a LaSalle-type argument on the particle level. It is then shown that the port-Hamiltonian structure is preserved in the mean-field limit and an analogue of the LaSalle invariance principle is studied in the space of probability measures equipped with the 2-Wasserstein-metric. The results on the particle and mean-field limit yield a new perspective on uniform stability of general interacting particle systems. Moreover, as the minimal port-Hamiltonian formulation is closed we identify the ports of the subsystems which admit generalized mass-spring-damper structure modelling the binary interaction of two particles. Using the information of ports we discuss the coupling of difference species in a port-Hamiltonian preserving manner.

We present a gradient-based identification algorithm to identify the system matrices of a linear port-Hamiltonian system from given input-output time data. Aiming for a direct structure-preserving approach, we employ techniques from optimal control with ordinary differential equations and define a constrained optimization problem. The input-to-state stability is discussed which is the key step towards the existence of optimal controls. Further, we derive the first-order optimality system taking into account the port-Hamiltonian structure. Indeed, the proposed method preserves the skew-symmetry and positive (semi)-definiteness of the system matrices throughout the optimization iterations. Numerical results with perturbed and unperturbed synthetic data, as well as an example from the PHS benchmark collection demonstrate the feasibility of the approach.

We present a gradient-based calibration algorithm to identify a port-Hamiltonian system from given time-domain input-output data. The gradient is computed with the help of sensitivities and the algorithm is tailored such that the structure of the system matrices of the port-Hamiltonian system (skew-symmetry and positive semi-definiteness) is preserved in each iteration of the algorithm. As we only require input-output data, we need to calibrate the initial condition of the internal state of the port-Hamiltonian system as well. Numerical results with synthetic data show the feasibility of the approach., Comment: 8 pages, 3 figures

We propose a multi-swarm approach to approximate the Pareto front of general multi-objective optimization problems that is based on the Consensus-based Optimization method (CBO). The algorithm is motivated step by step beginning with a simple extension of CBO based on fixed scalarization weights. To overcome the issue of choosing the weights we propose an adaptive weight strategy in the second modelling step. The modelling process is concluded with the incorporation of a penalty strategy that avoids clusters along the Pareto front and a diffusion term that prevents collapsing swarms. Altogether the proposed $K$-swarm CBO algorithm is tailored for a diverse approximation of the Pareto front and, simultaneously, the efficient set of general non-convex multi-objective problems. The feasibility of the approach is justified by analytic results, including convergence proofs, and a performance comparison to the well-known non-dominated sorting genetic algorithm (NSGA2) and the recently proposed one-swarm approach for multi-objective problems involving Consensus-based Optimization.

Port-Hamiltonian systems (PHS) theory is a recent but already well-established modelling approach for non-linear physical systems. Some studies have shown lately that PHS frameworks are relevant for modelling and control of swarm and multi-agent systems. We identify in this contribution a general class of microscopic force-based pedestrian models that can be formulated as a port-Hamiltonian system. The pedestrian PHS has linear structure and dissipation components. Non-linear effects come from isotropic pedestrian interactions. Simulation results on a torus with disordered initial states show that the port-Hamiltonian pedestrian model can exhibit different types of dynamics. They range from relaxed speed models with no interaction, dynamical billiards, or crystallization dynamics to realistic pedestrian collective behaviors, including lane and strip formation for counter and crossing flow. The port-Hamiltonian framework is a natural multiscale description of pedestrian dynamics as the Hamiltonian turns out to be a generic order parameter that allows us to identify specific behaviours of the dynamics from a macroscopic viewpoint. Particular cases even enable through energy balance to determine the Hamiltonian behavior without requiring the tedious computation of the microscopic dynamics. Using PHS theory, we systematically identify a critical threshold value for the Hamiltonian, which relies only on exogenous input and can be physically interpreted., Comment: 22 pages, 14 figures

We propose an approach based on function evaluations and Bayesian inference to extract higher-order differential information of objective functions {from a given ensemble of particles}. Pointwise evaluation $\{V(x^i)\}_i$ of some potential $V$ in an ensemble $\{x^i\}_i$ contains implicit information about first or higher order derivatives, which can be made explicit with little computational effort (ensemble-based gradient inference -- EGI). We suggest to use this information for the improvement of established ensemble-based numerical methods for optimization and sampling such as Consensus-based optimization and Langevin-based samplers. Numerical studies indicate that the augmented algorithms are often superior to their gradient-free variants, in particular the augmented methods help the ensembles to escape their initial domain, to explore multimodal, non-Gaussian settings and to speed up the collapse at the end of optimization dynamics.} The code for the numerical examples in this manuscript can be found in the paper's Github repository (https://github.com/MercuryBench/ensemble-based-gradient.git).

We propose an extension of the anisotropic interaction model which allows for collision avoidance in pairwise interactions by a rotation of forces \cite{arXiv:1912.04234} by including the agents' body size. The influence of the body size on the self-organization of the agents in channel and crossing scenarios as well as the fundamental diagram is studied. Since the model is stated as a coupled system of ordinary differential equations, we are able to give a rigorous well-posedness analysis. Then we state a parameter calibration problem that involves data from real experiments. We prove the existence of a minimizer and derive the corresponding first-order optimality conditions. With the help of these conditions we propose a gradient descent algorithm based on mini-batches of the data set. We employ the proposed algorithm to fit the parameter of the collision avoidance and the strength parameters of the interaction forces to given real data from experiments. The results underpin the feasibility of the method.

We introduce a practical method for incorporating equality and inequality constraints in global optimization methods based on stochastic interacting particle systems, specifically consensus-based optimization (CBO) and ensemble Kalman inversion (EKI). Unlike other approaches in the literature, the method we propose does not constrain the dynamics to the feasible region of the state space at all times; the particles evolve in the full space, but are attracted towards the feasible set by means of a penalization term added to the objective function and, in the case of CBO, an additional relaxation drift. We study the properties of the method through the associated mean-field Fokker--Planck equation and demonstrate its performance in numerical experiments on several test problems.

The Heston model is a well-known two-dimensional financial model. Because the Heston model contains implicit parameters that cannot be determined directly from real market data, calibrating the parameters to real market data is challenging. In addition, some of the parameters in the model are non-linear, which makes it difficult to find the global minimum of the optimization problem within the calibration. In this paper, we present a first step towards a novel space mapping approach for parameter calibration of the Heston model. Since the space mapping approach requires an optimization algorithm, we focus on deriving a gradient descent algorithm. To this end, we determine the formal adjoint of the Heston PDE, which is then used to update the Heston parameters. Since the methods are similar, we consider a variation of constant and time-dependent parameter sets. Numerical results show that our calibration of the Heston PDE works well for the various challenges in the calibration process and meets the requirements for later incorporation into the space mapping approach. Since the model and the algorithm are well known, this work is formulated as a proof of concept.

We calibrate parameters of neural networks that model forces in interaction dynamics with the help of the Consensus-based global optimization method (CBO). We state the general framework of interaction particle systems driven by neural networks and test the proposed method with a real dataset from the ESIMAS traffic experiment. The resulting forces are compared to well-known physical interaction forces. Moreover, we compare the performance of the proposed calibration process to the one in [4] which uses a stochastic gradient descent algorithm.

In this chapter we give an overview of the consensus-based global optimization algorithm and its recent variants. We recall the formulation and analytical results of the original model, then we discuss variants using component-wise independent or common noise. In combination with mini-batch approaches those variants were tailored for machine learning applications. Moreover, it turns out that the analytical estimates are dimension independent, which is useful for high-dimensional problems. We discuss the relationship of consensus-based optimization with particle swarm optimization, a method widely used in the engineering community. Then we survey a variant of consensus-based optimization that is proposed for global optimization problems constrained to hyper-surfaces. We conclude the chapter with remarks on applications, preprints and open problems.

We propose a space mapping-based optimization algorithm for microscopic interacting particle dynamics which are inappropriate for direct optimization. This is of relevance for example in applications with bounded domains such that the microscopic optimization is difficult. The space mapping algorithm exploits the relationship of the microscopic description of the interacting particle system and the corresponding macroscopic description as partial differential equation in the "many particle limit". We validate the approach with the help of a toy problem that allows for direct optimization. Then we study the performance of the algorithm in two applications. An evacuation dynamic is considered and the transportation of goods on a conveyor belt is optimized. The numerical results underline the feasibility of the proposed approach., Comment: 26 pages,8 figures

We propose a neural network approach to model general interaction dynamics and an adjoint based stochastic gradient descent algorithm to calibrate its parameters. The parameter calibration problem is considered as optimal control problem that is investigated from a theoretical and numerical point of view. We prove the existence of optimal controls, derive the corresponding first order optimality system and formulate a stochastic gradient descent algorithm to identify parameters for given data sets. To validate the approach we use real data sets from traffic and crowd dynamics to fit the parameters. The results are compared to forces corresponding to well-known interaction models such as the Lighthill-Whitham-Richards model for traffic and the social force model for crowd motion.

We propose a mean-field optimal control problem for the parameter identification of a given pattern. The cost functional is based on the Wasserstein distance between the probability measures of the modeled and the desired patterns. The first-order optimality conditions corresponding to the optimal control problem are derived using a Lagrangian approach on the mean-field level. Based on these conditions we propose a gradient descent method to identify relevant parameters such as angle of rotation and force scaling which may be spatially inhomogeneous. We discretize the first-order optimality conditions in order to employ the algorithm on the particle level. Moreover, we prove a rate for the convergence of the controls as the number of particles used for the discretization tends to infinity. Numerical results for the spatially homogeneous case demonstrate the feasibility of the approach.

In this paper we propose a variant of a consensus-based global optimization (CBO) method that uses personal best information in order to compute the global minimum of a non-convex, locally Lipschitz continuous function. The proposed approach is motivated by the original particle swarming algorithms, in which particles adjust their position with respect to the personal best, the current global best, and some additive noise. The personal best information along an individual trajectory is included with the help of a weighted mean. This weighted mean can be computed very efficiently due to its accumulative structure. It enters the dynamics via an additional drift term. We illustrate the performance with a toy example, analyze the respective memory-dependent stochastic system and compare the performance with the original CBO with component-wise noise for several benchmark problems. The proposed method has a higher success rate for computational experiments with a small particle number and where the initial particle distribution is disadvantageous with respect to the global minimum.

We study an identification problem which estimates the parameters of the underlying random distribution for uncertain scalar conservation laws. The hyperbolic equations are discretized with the so-called discontinuous stochastic Galerkin method, i.e., using a spatial discontinuous Galerkin scheme and a Multielement stochastic Galerkin ansatz in the random space. We assume an uncertain flux or uncertain initial conditions and that a data set of an observed solution is given. The uncertainty is assumed to be uniformly distributed on an unknown interval and we focus on identifying the correct endpoints of this interval. The first-order optimality conditions from the discontinuous stochastic Galerkin discretization are computed on the time-continuous level. Then, we solve the resulting semi-discrete forward and backward schemes with the Runge Kutta method. To illustrate the feasibility of the approach, we apply the method to a stochastic advection and a stochastic equation of Burgers' type. The results show that the method is able to identify the distribution parameters of the random variable in the uncertain differential equation even if discontinuities are present.