Search

In recent years, there has been a notable evolution in various multidisciplinary design methodologies for dynamic systems. Among these approaches, a noteworthy concept is that of concurrent conceptual and control design or co-design. This approach involves the tuning of feedforward and/or feedback control strategies in conjunction with the conceptual design of the dynamic system. The primary aim is to discover integrated solutions that surpass those attainable through a disjointed or decoupled approach. This concurrent design paradigm exhibits particular promise in the context of hybrid unmanned aerial systems (UASs), such as tail-sitters, where the objectives of versatility (driven by control considerations) and efficiency (influenced by conceptual design) often present conflicting demands. Nevertheless, a persistent challenge lies in the potential disparity between the theoretical models that underpin the design process and the real-world operational environment, the so-called reality gap. Such disparities can lead to suboptimal performance when the designed system is deployed in reality. To address this issue, this paper introduces DAIMYO, a novel design architecture that incorporates a high-fidelity environment, which emulates real-world conditions, into the procedure in pursuit of a `first-time-right' design. The outcome of this innovative approach is a design procedure that yields versatile and efficient UAS designs capable of withstanding the challenges posed by the reality gap.

In this work, we establish an optimal control framework for airborne wind energy systems (AWESs) with flexible tethers. The AWES configuration, consisting of a six-degree-of-freedom aircraft, a flexible tether, and a winch, is formulated as an index-1 differential-algebraic system of equations (DAE). We achieve this by adopting a minimal coordinate representation that uses Euler angles to characterize the aircraft's attitude and employing a quasi-static approach for the tether. The presented method contrasts with other recent optimization studies that use an index-3 DAE approach. By doing so, our approach avoids related inconsistency condition problems. We use a homotopy strategy to solve the optimal control problem that ultimately generates optimal trajectories of the AWES with a flexible tether. We furthermore compare with a rigid tether model by investigating the resulting mechanical powers and tether forces. Simulation results demonstrate the efficacy of the presented methodology and the necessity to incorporate the flexibility of the tether when solving the optimal control problem.

This article proposes two new algorithms tailored to discrete-time deterministic finite-horizon nonlinear optimal control problems or so-called trajectory optimization problems. Both algorithms are inspired by a novel theoretical paradigm known as probabilistic optimal control, that reformulates optimal control as an equivalent probabilistic inference problem. This perspective allows to address the problem using the Expectation-Maximization algorithm. We show that the application of this algorithm results in a fixed point iteration of probabilistic policies that converge to the deterministic optimal policy. Two strategies for policy evaluation are discussed, using state-of-the-art uncertainty quantification methods resulting into two distinct algorithms. The algorithms are structurally closest related to the differential dynamic programming algorithm and related methods that use sigma-point methods to avoid direct gradient evaluations. The main advantage of our work is an improved balance between exploration and exploitation over the iterations, leading to improved numerical stability and accelerated convergence. These properties are demonstrated on different nonlinear systems.

We present a novel variational framework for performing inference in (neural) stochastic differential equations (SDEs) driven by Markov-approximate fractional Brownian motion (fBM). SDEs offer a versatile tool for modeling real-world continuous-time dynamic systems with inherent noise and randomness. Combining SDEs with the powerful inference capabilities of variational methods, enables the learning of representative function distributions through stochastic gradient descent. However, conventional SDEs typically assume the underlying noise to follow a Brownian motion (BM), which hinders their ability to capture long-term dependencies. In contrast, fractional Brownian motion (fBM) extends BM to encompass non-Markovian dynamics, but existing methods for inferring fBM parameters are either computationally demanding or statistically inefficient. In this paper, building upon the Markov approximation of fBM, we derive the evidence lower bound essential for efficient variational inference of posterior path measures, drawing from the well-established field of stochastic analysis. Additionally, we provide a closed-form expression to determine optimal approximation coefficients. Furthermore, we propose the use of neural networks to learn the drift, diffusion and control terms within our variational posterior, leading to the variational training of neural-SDEs. In this framework, we also optimize the Hurst index, governing the nature of our fractional noise. Beyond validation on synthetic data, we contribute a novel architecture for variational latent video prediction,-an approach that, to the best of our knowledge, enables the first variational neural-SDE application to video perception., Comment: 24 pages, under review

This work discusses the use of model predictive control (MPC) for the manipulation of a pusher-slider system. In particular we leverage the differential flatness of the pusher-slider in combination with a B-splines transcription to address the computational demand that is typically associated to real-time implementation of an MPC controller. We demonstrate the flatness based B-spline MPC controller in simulation and compare it to a standard MPC implementation approach using direct multiple shooting. We evaluate the computational advantage of the flatness based MPC empirically and document computational acceleration up to 65%.

In this work we show that the differential kinematics of slider-pusher systems are differentially flat assuming quasi-static behaviour and frictionless contact. Second we demonstrate that the state trajectories are invariant to time-differential transformations of the path parametrizing coordinate. For one this property allows to impose arbitrary velocity profiles on the slider without impacting the geometry of the state trajectory. This property implies that certain path planning problems may be decomposed approximately into a strictly geometric path planning and an auxiliary throughput speed optimization problem. Building on these insights we elaborate a numerical approach tailored to constrained time optimal collision free path planning and apply it to the slider-pusher system.

We introduce a framework for cooperative manipulation, applied on an underactuated manipulation problem. Two stationary robotic manipulators are required to cooperate in order to reposition an object within their shared work space. Control of multi-agent systems for manipulation tasks cannot rely on individual control strategies with little to no communication between the agents that serve the common objective through swarming. Instead a coordination strategy is required that queries subtasks to the individual agents. We formulate the problem in a Task And Motion Planning (TAMP) setting, while considering a decomposition strategy that allows us to treat the task and motion planning problems separately. We solve the supervisory planning problem offline using deep Reinforcement Learning techniques resulting into a supervisory policy capable of coordinating the two manipulators into a successful execution of the pick-and-place task. Additionally, a benefit of solving the task planning problem offline is the possibility of real-time (re)planning, demonstrating robustness in the event of subtask execution failure or on-the-fly task changes. The framework achieved zero-shot deployment on the real setup with a success rate that is higher than 90%.

The value of unknown parameters of multibody systems is crucial for prediction, monitoring, and control, sometimes estimated using a biased physics-based model leading to incorrect outcomes. Discovering motion equations of multibody systems from time-series data is challenging as they consist of complex rational functions, constants as function arguments, and diverse function terms, which are not trivial to guess. This study aims at developing an evolutionary symbolic sparse regression approach for the system identification of multibody systems. The procedure discovers equations of motion and system parameters appearing as either constant values in function arguments or coefficients of function expressions. A genetic programming algorithm is written to generate symbolic function expressions, in which a hard-thresholding regression method is embedded. In an evolutionary manner, the complex functional forms, constant arguments, and unknown coefficients are identified to eventually discover the governing equation of a given system. A fitness measure is presented to promote parsimony in distilled equations and reduction in fit-to-data error. Hybrid discrete-continuous dynamical systems are also investigated, for which an approach is suggested to determine both mode number and system submodels. The performance and efficiency of the suggested evolutionary symbolic sparse regression methodology are evaluated in a simulation environment. The capability of the developed approach is also demonstrated by studying several multibody systems. The procedure is efficient and gives the possibility to estimate system parameters and distill respective governing equations. This technique reduces the risk that the function dictionary does not cover all functionality required to unravel hidden physical laws and the need for prior knowledge of the mechanism of interest.

We present KeyCLD, a framework to learn Lagrangian dynamics from images. Learned keypoints represent semantic landmarks in images and can directly represent state dynamics. We show that interpreting this state as Cartesian coordinates, coupled with explicit holonomic constraints, allows expressing the dynamics with a constrained Lagrangian. KeyCLD is trained unsupervised end-to-end on sequences of images. Our method explicitly models the mass matrix, potential energy and the input matrix, thus allowing energy based control. We demonstrate learning of Lagrangian dynamics from images on the dm_control pendulum, cartpole and acrobot environments. KeyCLD can be learned on these systems, whether they are unactuated, underactuated or fully actuated. Trained models are able to produce long-term video predictions, showing that the dynamics are accurately learned. We compare with Lag-VAE, Lag-caVAE and HGN, and investigate the benefit of the Lagrangian prior and the constraint function. KeyCLD achieves the highest valid prediction time on all benchmarks. Additionally, a very straightforward energy shaping controller is successfully applied on the fully actuated systems. Please refer to our project page for code and additional results: https://rdaems.github.io/keycld/

In energy intensive industrial systems, an evaporative cooling process may introduce operational flexibility. Such flexibility refers to a systems ability to deviate from its scheduled energy consumption. Identifying the flexibility, and therefore, designing control that ensures efficient and reliable operation presents a great challenge due to the inherently complex dynamics of industrial systems. Recently, machine learning models have attracted attention for identifying flexibility, due to their ability to model complex nonlinear behavior. This research presents machine learning based methods that integrate system dynamics into the machine learning models (e.g., Neural Networks) for better adherence to physical constraints. We define and evaluate physics informed long-short term memory networks (PhyLSTM) and physics informed neural networks (PhyNN) for the identification of flexibility in the evaporative cooling process. These physics informed networks approximate the time-dependent relationship between control input and system response while enforcing the dynamics of the process in the neural network architecture. Our proposed PhyLSTM provides less than 2% system response estimation error, converges in less than half iterations compared to a baseline Neural Network (NN), and accurately estimates the defined flexibility metrics. We include a detailed analysis of the impact of training data size on the performance and optimization of our proposed models.

Due to the presence of reporting and settlement delay, claim data sets collected by non-life insurance companies are typically incomplete, facing right censored claim count and claim severity observations. Current practice in non-life insurance pricing tackles these right censored data via a two-step procedure. First, best estimates are computed for the number of claims that occurred in past exposure periods and the ultimate claim severities, using the incomplete, historical claim data. Second, pricing actuaries build predictive models to estimate technical, pure premiums for new contracts by treating these best estimates as actual observed outcomes, hereby neglecting their inherent uncertainty. We propose an alternative approach that brings valuable insights for both non-life pricing as well as reserving. As such we effectively bridge these two key actuarial tasks that have traditionally been discussed in silos. Hereto we develop a granular occurrence and development model for non-life claims that tackles reserving and at the same time resolves the inconsistency in traditional pricing techniques between actual observations and imputed best estimates. We illustrate our proposed model on an insurance as well as a reinsurance portfolio. The advantages of our proposed strategy are most compelling in the reinsurance illustration where large uncertainties in the best estimates originate from long reporting and settlement delays, low claim frequencies and heavy (even extreme) claim sizes.

Sample-based trajectory optimisers are a promising tool for the control of robotics with non-differentiable dynamics and cost functions. Contemporary approaches derive from a restricted subclass of stochastic optimal control where the optimal policy can be expressed in terms of an expectation over stochastic paths. By estimating the expectation with Monte Carlo sampling and reinterpreting the process as exploration noise, a stochastic search algorithm is obtained tailored to (deterministic) trajectory optimisation. For the purpose of future algorithmic development, it is essential to properly understand the underlying theoretical foundations that allow for a principled derivation of such methods. In this paper we make a connection between entropy regularisation in optimisation and deterministic optimal control. We then show that the optimal policy is given by a belief function rather than a deterministic function. The policy belief is governed by a Bayesian-type update where the likelihood can be expressed in terms of a conditional expectation over paths induced by a prior policy. Our theoretical investigation firmly roots sample based trajectory optimisation in the larger family of control as inference. It allows us to justify a number of heuristics that are common in the literature and motivate a number of new improvements that benefit convergence.

We propose a simple, practical and intuitive approach to improve the performance of a conventional controller in uncertain environments using deep reinforcement learning while maintaining safe operation. Our approach is motivated by the observation that conventional controllers in industrial motion control value robustness over adaptivity to deal with different operating conditions and are suboptimal as a consequence. Reinforcement learning on the other hand can optimize a control signal directly from input-output data and thus adapt to operational conditions, but lacks safety guarantees, impeding its use in industrial environments. To realize adaptive control using reinforcement learning in such conditions, we follow a residual learning methodology, where a reinforcement learning algorithm learns corrective adaptations to a base controller's output to increase optimality. We investigate how constraining the residual agent's actions enables to leverage the base controller's robustness to guarantee safe operation. We detail the algorithmic design and propose to constrain the residual actions relative to the base controller to increase the method's robustness. Building on Lyapunov stability theory, we prove stability for a broad class of mechatronic closed-loop systems. We validate our method experimentally on a slider-crank setup and investigate how the constraints affect the safety during learning and optimality after convergence.

Path Integral Control methods were developed for stochastic optimal control covering a wide class of finite horizon formulations with control affine nonlinear dynamics. Characteristic for this class is that the HJB equation is linear and consequently the value function can be expressed as a conditional expectation of the exponentially weighted cost-to-go evaluated over trajectories with uncontrolled system dynamics, hence the name. Subsequently it was shown that under the same assumptions Path Integral Control generalises to finite horizon risk sensitive stochastic optimal control problems. Here we study whether the HJB of infinite horizon formulations can be made linear as well. Our interest in infinite horizon formulations is motivated by the stationarity of the associated value function and their inherent dynamic stability seeking nature. Technically a stationary value function may ease the solution of the associated linear HJB. Second we argue this may offer an interesting starting point for off-linear Reinforcement Learning applications. We show formally that the discounted and average cost formulations are respectively intractable and tractable.

Functional connectivity quantifies the statistical dependencies between the activity of brain regions, measured using neuroimaging data such as functional MRI BOLD time series. The network representation of functional connectivity, called a Functional Connectome (FC), has been shown to contain an individual fingerprint allowing participants identification across consecutive testing sessions. Recently, researchers have focused on the extraction of these fingerprints, with potential applications in personalized medicine. Here, we show that a mathematical operation denominated degree-normalization can improve the extraction of FC fingerprints. Degree-normalization has the effect of reducing the excessive influence of strongly connected brain areas in the whole-brain network. We adopt the differential identifiability framework and apply it to both original and degree-normalized FCs of 409 individuals from the Human Connectome Project, in resting-state and 7 fMRI tasks. Our results indicate that degree-normalization systematically improves three fingerprinting metrics, namely differential identifiability, identification rate and matching rate. Moreover, the results related to the matching rate metric suggest that individual fingerprints are embedded in a low-dimensional space. The results suggest that low-dimensional functional fingerprints lie in part in weakly connected subnetworks of the brain, and that degree-normalization helps uncovering them. This work introduces a simple mathematical operation that could lead to significant improvements in future FCs fingerprinting studies., Comment: Main: 23 pages, 7 Figures, Supplementary: 3 pages, 3 Figures, 1 Table

In this article we present a generalised view on Path Integral Control (PIC) methods. PIC refers to a particular class of policy search methods that are closely tied to the setting of Linearly Solvable Optimal Control (LSOC), a restricted subclass of nonlinear Stochastic Optimal Control (SOC) problems. This class is unique in the sense that it can be solved explicitly to yield a formal optimal state trajectory distribution. In this contribution we first review the PIC theory and discuss related algorithms tailored to policy search in general. We are able to identify a generic design strategy that relies on the existence of an optimal state trajectory distribution and finds a parametric policy by minimizing the cross entropy between the optimal and a state trajectory distribution parametrized through its policy. Inspired by this observation we then aim to formulate a SOC problem that shares traits with the LSOC setting yet that covers a less restrictive class of problem formulations. We refer to this SOC problem as Entropy Regularized Trajectory Optimization. The problem is closely related to the Entropy Regularized Stochastic Optimal Control setting which is lately often addressed by the Reinforcement Learning (RL) community. We analyse the theoretical convergence behaviour of the theoretical state trajectory distribution sequence and draw connections with stochastic search methods tailored to classic optimization problems. Finally we derive explicit updates and compare the implied Entropy Regularized PIC with earlier work in the context of both PIC and RL for derivative-free trajectory optimization.

In this paper, an evolutionary-based sparse regression algorithm is proposed and applied onto experimental data collected from a Duffing oscillator setup and numerical simulation data. Our purpose is to identify the Coulomb friction terms as part of the ordinary differential equation of the system. Correct identification of this nonlinear system using sparse identification is hugely dependent on selecting the correct form of nonlinearity included in the function library. Consequently, in this work, the evolutionary-based sparse identification is replacing the need for user knowledge when constructing the library in sparse identification. Constructing the library based on the data-driven evolutionary approach is an effective way to extend the space of nonlinear functions, allowing for the sparse regression to be applied on an extensive space of functions. ,e results show that the method provides an effective algorithm for the purpose of unveiling the physical nature of the Duffing oscillator. In addition, the robustness of the identification algorithm is investigated for various levels of noise in simulation. ,e proposed method has possible applications to other nonlinear dynamic systems in mechatronics, robotics, and electronics.

Traditional non-life reserving models largely neglect the vast amount of information collected over the lifetime of a claim. This information includes covariates describing the policy, claim cause as well as the detailed history collected during a claim's development over time. We present the hierarchical reserving model as a modular framework for integrating a claim's history and claim-specific covariates into the development process. Hierarchical reserving models decompose the joint likelihood of the development process over time. Moreover, they are tailored to the portfolio at hand by adding a layer to the model for each of the events registered during the development of a claim (e.g. settlement, payment). Layers are modelled with statistical learning (e.g. generalized linear models) or machine learning methods (e.g. gradient boosting machines) and use claim-specific covariates. As a result of its flexibility, this framework incorporates many existing reserving models, ranging from aggregate models designed for run-off triangles to individual models using claim-specific covariates. This connection allows us to develop a data-driven strategy for choosing between aggregate and individual reserving; an important decision for reserving practitioners. We illustrate our method with a case study on a real insurance data set and deduce new insights in the covariates driving the development of claims. Moreover, we evaluate the method's performance on a large number of simulated portfolios representing several realistic development scenarios and demonstrate the flexibility and robustness of the hierarchical reserving model.

Dynamic models of mechatronic systems are abundantly used in the context of motion control and design of complex servo applications. In practice, these systems are often plagued by unknown interactions, which make the physics-based relations of the system dynamics only partially known. This paper presents a neural network augmented physics (NNAP) model as a combination of physics-inspired and neural layers. The neural layers are inserted in the model to compensate for the unmodeled interactions, without requiring direct measurements of these unknown phenomena. In contrast to traditional approaches, both the neural network and physical parameters are simultaneously optimized, solely by using state and control input measurements. The methodology is applied on experimental data of a slider-crank setup for which the state dependent load interactions are unknown. The NNAP model proves to be a stable and accurate modeling formalism for dynamic systems that ab initio can only be partially described by physical laws. Moreover, the results show that a recurrent implementation of the NNAP model enables improved robustness and accuracy of the system state predictions, compared to its feedforward counterpart. Besides capturing the system dynamics, the NNAP model provides a means to gain new insights by extracting the neural network from the converged NNAP model. In this way, we discovered accurate representations of the unknown spring force interaction and friction phenomena acting on the slider mechanism.

A delay between the occurrence and the reporting of events often has practical implications such as for the amount of capital to hold for insurance companies, or for taking preventive actions in case of infectious diseases. The accurate estimation of the number of incurred but not (yet) reported events forms an essential part of properly dealing with this phenomenon. We review the current practice for analysing such data and we present a flexible regression framework to jointly estimate the occurrence and reporting of events. By linking this setting to an incomplete data problem, estimation is performed via an expectation-maximization algorithm. The resulting method is elegant, easy to understand and implement, and provides refined insights in the nowcasts. The proposed methodology is applied to a European general liability portfolio in insurance.

A numerically efficient inverse method for parametric model uncertainty identification using maximum likelihood estimation is presented. The goal is to identify a probability model for a fixed number of model parameters based on a set of experiments. To perform maximum likelihood estimation, the output probability density function is required. Forward propagation of input uncertainty is established combining Polynomial Chaos and moment matching. High-order moments of the output distribution are estimated using the generalized Polynomial Chaos framework. Next, a maximum entropy parametric distribution is matched with the estimated moments. This method is numerically very attractive due to reduced forward sampling and deterministic nature of the propagation strategy. The methodology is applied on a wet clutch system for which certain model variables are considered as stochastic. The number of required model simulations to achieve the same accuracy as the brute force methodologies is decreased by one order of magnitude. The probability model identified with the high order estimates resulted into a true log-likelihood increase of about 4% since the accuracy of the estimated output probability density function could be improved up to 47%., Comment: 37 pages, 20 figures

This paper discusses a method enabling optimal control of nonlinear systems that are subject to parametric uncertainty. A stochastic optimal tracking problem is formulated that can be expressed in function of the first two stochastic moments of the state. The proposed formulation allows to penalize system performance and system robustness independently. The use of polynomial chaos expansions is investigated to arrive at a computationally tractable formulation expressing the stochastic moments in function of the polynomial expansion coefficients rigorously. It is then demonstrated how the stochastic optimal control problem can be reformulated as a deterministic optimal control problem in function of these coefficients. The proposed method is applied to find a robust control input for the start-up of an eccentrically loaded drive train that is inherently prone to bifurcation behaviour. A reference trajectory is chosen to deliberately provoke a bifurcation. The proposed framework is able to avoid the bifurcation behaviour regardlessly., Comment: 7 pages; 5 figures; ICSTCC 2018, 22nd International Conference on System Theory, Control and Computing. 10 - 12 October. Sinaia - Romania

We present a novel derivative-free interpolation based optimization algorithm. A trust-region method is used where a surrogate model is realized via an interpolation framework. The framework for interpolation is provided by Universal Kriging. A first contribution focuses on the development of an original sampling strategy. A valid model is guaranteed by maintaining a well-poised subset that exhibits the regular simplex geometry approximately. It follows that this strategy improves the scattering of points with respect to the state-of-the-art and, even importantly, assures that the surrogate model exhibits curvature. A second contribution focuses on the generalization of the spherical trust-region geometry to an ellipsoidal geometry, that to account for local anisotropy of the objective function and to improve the interpolation conditions as seen from the output space. The ensemble method is validated against its direct competitors on a set of multidimensional problems., Comment: 29 pages, 4 figures

In this paper we aim to apply an adaptation of the recently developed technique of sparse identification of nonlinear dynamical systems on a Duffing experimental setup with cubic feedback of the output. The Duffing oscillator described by nonlinear differential equation which demonstrates chaotic behavior and bifurcations, has received considerable attention in recent years as it arises in many real-world engineering applications. Therefore its identification is of interest for numerous practical problems. To adopt the existing identification method to this application, the optimization process which identifies the most important terms of the model has been modified. In addition, the impact of changing the amount of regularization parameter on the mean square error of the fit has been studied. Selection of the true model is done via balancing complexity and accuracy using Pareto front analysis. This study provides considerable insight into the employment of sparse identification method on the real-world setups and the results show that the developed algorithm is capable of finding the true nonlinear model of the considered application including a nonlinear friction term., Comment: 6 pages, 8 figures, conference paper

This paper considers the problem of predicting the number of events that have occurred in the past, but which are not yet observed due to a delay. Such delayed events are relevant in predicting the future cost of warranties, pricing maintenance contracts, determining the number of unreported claims in insurance and in modeling the outbreak of diseases. Disregarding these unobserved events results in a systematic underestimation of the event occurrence process. Our approach puts emphasis on modeling the time between the occurrence and observation of the event, the so-called observation delay. We propose a granular model for the heterogeneity in this observation delay based on the occurrence day of the event and on calendar day effects in the observation process, such as weekday and holiday effects. We illustrate this approach on a European general liability insurance data set where the occurrence of an accident is reported to the insurer with delay.