Search

As one of the mainstream approaches in system identification, subspace identification methods (SIMs) are known for their simple parameterization for MIMO systems and robust numerical properties. However, a comprehensive statistical analysis of SIMs remains an open problem. Amid renewed focus on identifying state-space models in the non-asymptotic regime, this work presents a finite sample analysis for a large class of open-loop SIMs. It establishes high-probability upper bounds for system matrices obtained via SIMs, and reveals that convergence rates for estimating Markov parameters and system matrices are $\mathcal{O}(1/\sqrt{N})$ up to logarithmic terms, in line with classical asymptotic results. Following the key steps of SIMs, we arrive at the above results by a three-step procedure. In Step 1, we begin with a parsimonious SIM (PARSIM) that uses least-squares regression to estimate multiple high-order ARX models in parallel. Leveraging a recent analysis of an individual ARX model, we obtain a union error bound for a bank of ARX models. Step 2 involves model reduction via weighted singular value decomposition (SVD), where we consider different data-dependent weighting matrices and use robustness results for SVD to obtain error bounds on extended controllability and observability matrices, respectively. The final Step 3 focuses on deriving error bounds for system matrices, where two different realization algorithms, the MOESP type and the Larimore type, are considered. Although our study initially focuses on PARSIM, the methodologies apply broadly across many variants of SIMs.

The sampling rate of input and output signals is known to play a critical role in the identification and control of dynamical systems. For slow-sampled continuous-time systems that do not satisfy the Nyquist-Shannon sampling condition for perfect signal reconstructability, careful consideration is required when identifying parametric and nonparametric models. In this letter, a comprehensive statistical analysis of estimators under slow sampling is performed. Necessary and sufficient conditions are obtained for unbiased estimates of the frequency response function beyond the Nyquist frequency, and it is shown that consistency of parametric estimators can be achieved even if input frequencies overlap after aliasing. Monte Carlo simulations confirm the theoretical properties., Comment: 6 pages, 2 figures

Markov parameters play a key role in system identification. There exists many algorithms where these parameters are estimated using least-squares in a first, pre-processing, step, including subspace identification and multi-step least-squares algorithms, such as Weighted Null-Space Fitting. Recently, there has been an increasing interest in non-asymptotic analysis of estimation algorithms. In this contribution we identify the Markov parameters using weighted least-squares and present non-asymptotic analysis for such estimator. To cover both stable and unstable systems, multiple trajectories are collected. We show that with the optimal weighting matrix, weighted least-squares gives a tighter error bound than ordinary least-squares for the case of non-uniformly distributed measurement errors. Moreover, as the optimal weighting matrix depends on the system's true parameters, we introduce two methods to consistently estimate the optimal weighting matrix, where the convergence rate of these estimates is also provided. Numerical experiments demonstrate improvements of weighted least-squares over ordinary least-squares in finite sample settings.

Subspace identification methods (SIMs) have proven very powerful for estimating linear state-space models. To overcome the deficiencies of classical SIMs, a significant number of algorithms has appeared over the last two decades, where most of them involve a common intermediate step, that is to estimate the range space of the extended observability matrix. In this contribution, an optimized version of the parallel and parsimonious SIM (PARSIM), PARSIM\textsubscript{opt}, is proposed by using weighted least-squares. It not only inherits all the benefits of PARSIM but also attains the best linear unbiased estimator for the above intermediate step. Furthermore, inspired by SIMs based on the predictor form, consistent estimates of the optimal weighting matrix for weighted least-squares are derived. Essential similarities, differences and simulated comparisons of some key SIMs related to our method are also presented.

While subspace identification methods (SIMs) are appealing due to their simple parameterization for MIMO systems and robust numerical realizations, a comprehensive statistical analysis of SIMs remains an open problem, especially in the non-asymptotic regime. In this work, we provide a finite sample analysis for a class of SIMs, which reveals that the convergence rates for estimating Markov parameters and system matrices are $\mathcal{O}(1/\sqrt{N})$, in line with classical asymptotic results. Based on the observation that the model format in classical SIMs becomes non-causal because of a projection step, we choose a parsimonious SIM that bypasses the projection step and strictly enforces a causal model to facilitate the analysis, where a bank of ARX models are estimated in parallel. Leveraging recent results from finite sample analysis of an individual ARX model, we obtain an overall error bound of an array of ARX models and proceed to derive error bounds for system matrices via robustness results for the singular value decomposition.

This paper addresses a kernel-based learning problem for a network of agents locally observing a latent multidimensional, nonlinear phenomenon in a noisy environment. We propose a learning algorithm that requires only mild a priori knowledge about the phenomenon under investigation and delivers a model with corresponding non-asymptotic high probability error bounds. Both non-asymptotic analysis of the method and numerical simulation results are presented and discussed in the paper.

Block coordinate descent is an optimization technique that is used for estimating multi-input single-output (MISO) continuous-time models, as well as single-input single output (SISO) models in additive form. Despite its widespread use in various optimization contexts, the statistical properties of block coordinate descent in continuous-time system identification have not been covered in the literature. The aim of this paper is to formally analyze the bias properties of the block coordinate descent approach for the identification of MISO and additive SISO systems. We characterize the asymptotic bias at each iteration, and provide sufficient conditions for the consistency of the estimator for each identification setting. The theoretical results are supported by simulation examples., Comment: 6 pages, 2 figures

Refined instrumental variable methods have been broadly used for identification of continuous-time systems in both open and closed-loop settings. However, the theoretical properties of these methods are still yet to be fully understood when operating in closed-loop. In this paper, we address the consistency of the simplified refined instrumental variable method for continuous-time systems (SRIVC) and its closed-loop variant CLSRIVC when they are applied on data that is generated from a feedback loop. In particular, we consider feedback loops consisting of continuous-time controllers, as well as the discrete-time control case. This paper proves that the SRIVC and CLSRIVC estimators are not generically consistent when there is a continuous-time controller in the loop, and that generic consistency can be achieved when the controller is implemented in discrete-time. Numerical simulations are presented to support the theoretical results., Comment: 14 pages, 5 figures

The maximum absolute correlation between regressors, which is called mutual coherence, plays an essential role in sparse estimation. A regressor matrix whose columns are highly correlated may result from optimal input design, since there is no constraint on the mutual coherence, making it difficult to handle sparse estimation. This paper aims to tackle this issue for fixed denominator models, which include Laguerre, Kautz, and generalized orthonormal basis function expansion models, for example. The paper proposes an optimal input design method where the achieved Fisher information matrix is fitted to the desired Fisher matrix, together with a coordinate transformation designed to make the regressors in the transformed coordinates have low mutual coherence. The method can be used together with any sparse estimation method and any desired Fisher matrix. A numerical study shows its potential for alleviating the problem of model order selection when used in conjunction with, for example, classical methods such as the Akaike Information Criterion., Comment: Accepted to the IEEE Transactions on Automatic Control

When identifying electrical, mechanical, or biological systems, parametric continuous-time identification methods can lead to interpretable and parsimonious models when the model structure aligns with the physical properties of the system. Traditional linear system identification may not consider the most parsimonious model when relying solely on unfactored transfer functions, which typically result from standard direct approaches. This paper presents a novel identification method that delivers additive models for both open and closed-loop setups. The estimators that are derived are shown to be generically consistent, and can admit the identification of marginally stable additive systems. Numerical simulations show the efficacy of the proposed approach, and its performance in identifying a modal representation of a flexible beam is verified using experimental data., Comment: 15 pages, 6 figures

A direct data-driven iterative algorithm is developed to accurately estimate the $H_\infty$ norm of a linear time-invariant system from continuous operation, i.e., without resetting the system. The main technical step involves a reversed-circulant matrix that can be evaluated in a model-free setting by performing experiments on the real system., Comment: 7 pages, 4 figures

Parameter estimation is an important sub-field in statistics and system identification. Various methods for parameter estimation have been proposed in the literature, among which the Two-Stage (TS) approach is particularly promising, due to its ease of implementation and reliable estimates. Among the different statistical frameworks used to derive TS estimators, the min-max framework is attractive due to its mild dependence on prior knowledge about the parameters to be estimated. However, the existing implementation of the minimax TS approach has currently limited applicability, due to its heavy computational load. In this paper, we overcome this difficulty by using a gradient boosting machine (GBM) in the second stage of TS approach. We call the resulting algorithm the Two-Stage Gradient Boosting Machine (TSGBM) estimator. Finally, we test our proposed TSGBM estimator on several numerical examples including models of dynamical systems., Comment: 6 pages

Many industrial processes require suitable controllers to meet their performance requirements. More often, a sophisticated digital twin is available, which is a highly complex model that is a virtual representation of a given physical process, whose parameters may not be properly tuned to capture the variations in the physical process. In this paper, we present a sim2real, direct data-driven controller tuning approach, where the digital twin is used to generate input-output data and suitable controllers for several perturbations in its parameters. State-of-the art neural-network architectures are then used to learn the controller tuning rule that maps input-output data onto the controller parameters, based on artificially generated data from perturbed versions of the digital twin. In this way, as far as we are aware, we tackle for the first time the problem of re-calibrating the controller by meta-learning the tuning rule directly from data, thus practically replacing the control engineer with a machine learning model. The benefits of this methodology are illustrated via numerical simulations for several choices of neural-network architectures., Comment: 6 pages

Knowing the features of a complex system that are highly relevant to a particular target variable is of fundamental interest in many areas of science. Existing approaches are often limited to linear settings, sometimes lack guarantees, and in most cases, do not scale to the problem at hand, in particular to images. We propose DRCFS, a doubly robust feature selection method for identifying the causal features even in nonlinear and high dimensional settings. We provide theoretical guarantees, illustrate necessary conditions for our assumptions, and perform extensive experiments across a wide range of simulated and semi-synthetic datasets. DRCFS significantly outperforms existing state-of-the-art methods, selecting robust features even in challenging highly non-linear and high-dimensional problems.

The Refined Instrumental Variable method for discrete-time systems (RIV) and its variant for continuous-time systems (RIVC) are popular methods for the identification of linear systems in open-loop. The continuous-time equivalent of the transfer function estimate given by the RIV method is commonly used as an initialization point for the RIVC estimator. In this paper, we prove that these estimators share the same converging points for finite sample size when the continuous-time model has relative degree zero or one. This relation does not hold for higher relative degrees. Then, we propose a modification of the RIV method whose continuous-time equivalent is equal to the RIVC estimator for any non-negative relative degree. The implications of the theoretical results are illustrated via a simulation example., Comment: 6 pages, 0 figures

We study the problem of diffusion-based network learning of a nonlinear phenomenon, $m$, from local agents' measurements collected in a noisy environment. For a decentralized network and information spreading merely between directly neighboring nodes, we propose a non-parametric learning algorithm, that avoids raw data exchange and requires only mild \textit{a priori} knowledge about $m$. Non-asymptotic estimation error bounds are derived for the proposed method. Its potential applications are illustrated through simulation experiments.

Robots have been increasingly better at doing tasks for humans by learning from their feedback, but still often suffer from model misalignment due to missing or incorrectly learned features. When the features the robot needs to learn to perform its task are missing or do not generalize well to new settings, the robot will not be able to learn the task the human wants and, even worse, may learn a completely different and undesired behavior. Prior work shows how the robot can detect when its representation is missing some feature and can, thus, ask the human to be taught about the new feature; however, these works do not differentiate between features that are completely missing and those that exist but do not generalize to new environments. In the latter case, the robot would detect misalignment and simply learn a new feature, leading to an arbitrarily growing feature representation that can, in turn, lead to spurious correlations and incorrect learning down the line. In this work, we propose separating the two sources of misalignment: we propose a framework for determining whether a feature the robot needs is incorrectly learned and does not generalize to new environment setups vs. is entirely missing from the robot's representation. Once we detect the source of error, we show how the human can initiate the realignment process for the model: if the feature is missing, we follow prior work for learning new features; however, if the feature exists but does not generalize, we use data augmentation to expand its training and, thus, complete the correction. We demonstrate the proposed approach in experiments with a simulated 7DoF robot manipulator and physical human corrections., Comment: 8 pages, 4 figures

This paper concerns the identification of continuous-time systems in state-space form that are subject to Lebesgue sampling. Contrary to equidistant (Riemann) sampling, Lebesgue sampling consists of taking measurements of a continuous-time signal whenever it crosses fixed and regularly partitioned thresholds. The knowledge of the intersample behavior of the output data is exploited in this work to derive an expectation-maximization (EM) algorithm for parameter estimation of the state-space and noise covariance matrices. For this purpose, we use the incremental discrete-time equivalent of the system, which leads to EM iterations of the continuous-time state-space matrices that can be computed by standard filtering and smoothing procedures. The effectiveness of the identification method is tested via Monte Carlo simulations., Comment: 6 pages, 3 figures

The identification of electrical, mechanical, and biological systems using data can benefit greatly from prior knowledge extracted from physical modeling. Parametric continuous-time identification methods can naturally incorporate this knowledge, which leads to interpretable and parsimonious models. However, some applications lead to model structures that lack parsimonious descriptions using unfactored transfer functions, which are commonly used in standard direct approaches for continuous-time system identification. In this paper we characterize this parsimony problem, and develop a block-coordinate descent algorithm that delivers parsimonious models by sequentially estimating an additive decomposition of the transfer function of interest. Numerical simulations show the efficacy of the proposed approach., Comment: 6 pages, 3 figures

The contribution of this paper is a generalized formulation of correctional learning using optimal transport, which is about how to optimally transport one mass distribution to another. Correctional learning is a framework developed to enhance the accuracy of parameter estimation processes by means of a teacher-student approach. In this framework, an expert agent, referred to as the teacher, modifies the data used by a learning agent, known as the student, to improve its estimation process. The objective of the teacher is to alter the data such that the student's estimation error is minimized, subject to a fixed intervention budget. Compared to existing formulations of correctional learning, our novel optimal transport approach provides several benefits. It allows for the estimation of more complex characteristics as well as the consideration of multiple intervention policies for the teacher. We evaluate our approach on two theoretical examples, and on a human-robot interaction application in which the teacher's role is to improve the robots performance in an inverse reinforcement learning setting.

Parameter estimation in statistics and system identification relies on data that may contain sensitive information. To protect this sensitive information, the notion of \emph{differential privacy} (DP) has been proposed, which enforces confidentiality by introducing randomization in the estimates. Standard algorithms for differentially private estimation are based on adding an appropriate amount of noise to the output of a traditional point estimation method. This leads to an accuracy-privacy trade off, as adding more noise reduces the accuracy while increasing privacy. In this paper, we propose a new Unified Bayes Private Point (UBaPP) approach to Bayes point estimation of the unknown parameters of a data generating mechanism under a DP constraint, that achieves a better accuracy-privacy trade off than traditional approaches. We verify the performance of our approach on a simple numerical example.

In this paper we establish that every (deterministic) non-autonomous, discrete-time, causal, time invariant system has a state-space representation, and discuss its minimality., Comment: Accepted to the IEEE Transactions on Automatic Control

One of the most important problems in system identification and statistics is how to estimate the unknown parameters of a given model. Optimization methods and specialized procedures, such as Empirical Minimization (EM) can be used in case the likelihood function can be computed. For situations where one can only simulate from a parametric model, but the likelihood is difficult or impossible to evaluate, a technique known as the Two-Stage (TS) Approach can be applied to obtain reliable parametric estimates. Unfortunately, there is currently a lack of theoretical justification for TS. In this paper, we propose a statistical decision-theoretical derivation of TS, which leads to Bayesian and Minimax estimators. We also show how to apply the TS approach on models for independent and identically distributed samples, by computing quantiles of the data as a first step, and using a linear function as the second stage. The proposed method is illustrated via numerical simulations., Comment: 7 pages, 6 figures, 1 table

A classical learning setting typically concerns an agent/student who collects data, or observations, from a system in order to estimate a certain property of interest. Correctional learning is a type of cooperative teacher-student framework where a teacher, who has partial knowledge about the system, has the ability to observe and alter (correct) the observations received by the student in order to improve the accuracy of its estimate. In this paper, we show how the variance of the estimate of the student can be reduced with the help of the teacher. We formulate the corresponding online problem - where the teacher has to decide, at each time instant, whether or not to change the observations due to a limited budget - as a Markov decision process, from which the optimal policy is derived using dynamic programming. We validate the framework in numerical experiments, and compare the optimal online policy with the one from the batch setting., Comment: 8 pages, 7 figures

We study the problem of regret minimization in a multi-armed bandit setup where the agent is allowed to play multiple arms at each round by spreading the resources usually allocated to only one arm. At each iteration the agent selects a normalized power profile and receives a Gaussian vector as outcome, where the unknown variance of each sample is inversely proportional to the power allocated to that arm. The reward corresponds to a linear combination of the power profile and the outcomes, resembling a linear bandit. By spreading the power, the agent can choose to collect information much faster than in a traditional multi-armed bandit at the price of reducing the accuracy of the samples. This setup is fundamentally different from that of a linear bandit -- the regret is known to scale as $\Theta(\sqrt{T})$ for linear bandits, while in this setup the agent receives a much more detailed feedback, for which we derive a tight $\log(T)$ problem-dependent lower-bound. We propose a Thompson-Sampling-based strategy, called Weighted Thompson Sampling (\WTS), that designs the power profile as its posterior belief of each arm being the best arm, and show that its upper bound matches the derived logarithmic lower bound. Finally, we apply this strategy to a problem of control and system identification, where the goal is to estimate the maximum gain (also called $\mathcal{H}_\infty$-norm) of a linear dynamical system based on batches of input-output samples., Comment: 9 content pages, 3 references pages, 22 appendix pages, 4 figures, 34 total pages

The Consistency of the Closed-Loop Simplified Refined Instrumental Variable method for Continuous-time system (CLSRIVC) is analysed based on sampled data. It is proven that the CLSRIVC estimator is not consistent when a continuous-time controller is used in the closed-loop.

In continuous-time system identification, the intersample behavior of the input signal is known to play a crucial role in the performance of estimation methods. One common input behavior assumption is that the spectrum of the input is band-limited. The sinc interpolation property of these input signals yields equivalent discrete-time representations that are non-causal. This observation, often overlooked in the literature, is exploited in this work to study non-parametric frequency response estimators of linear continuous-time systems. We study the properties of non-causal least-square estimators for continuous-time system identification, and propose a kernel-based non-causal regularized least-squares approach for estimating the band-limited equivalent impulse response. The proposed methods are tested via extensive numerical simulations., Comment: 6 pages, 4 figures

We consider a cooperative system identification scenario in which an expert agent (teacher) knows a correct, or at least a good, model of the system and aims to assist a learner-agent (student), but cannot directly transfer its knowledge to the student. For example, the teacher's knowledge of the system might be abstract or the teacher and student might be employing different model classes, which renders the teacher's parameters uninformative to the student. In this paper, we propose correctional learning as an approach to the above problem: Suppose that in order to assist the student, the teacher can intercept the observations collected from the system and modify them to maximize the amount of information the student receives about the system. We formulate a general solution as an optimization problem, which for a multinomial system instantiates itself as an integer program. Furthermore, we obtain finite-sample results on the improvement that the assistance from the teacher results in (as measured by the reduction in the variance of the estimator) for a binomial system., Comment: 8 pages, 4 figures

For many years, the Simplified Refined Instrumental Variable method for Continuous-time systems (SRIVC) has been widely used for identification. The intersample behaviour of the input plays an important role in this method, and it has been shown recently that the SRIVC estimator is not consistent if an incorrect assumption on the intersample behaviour is considered. In this paper, we present an extension of the SRIVC algorithm that is able to deal with continuous-time multisine signals, which cannot be interpolated exactly through hold reconstructions. The proposed estimator is generically consistent for any input reconstructed through zero or first-order-hold devices, and we show that it is generically consistent for continuous-time multisine inputs as well. The statistical performance of the proposed estimator is compared to the standard SRIVC estimator through extensive simulations., Comment: 12 pages, 3 figures

We consider a counter-adversarial sequential decision-making problem where an agent computes its private belief (posterior distribution) of the current state of the world, by filtering private information. According to its private belief, the agent performs an action, which is observed by an adversarial agent. We have recently shown how the adversarial agent can reconstruct the private belief of the decision-making agent via inverse optimization. The main contribution of this paper is a method to obfuscate the private belief of the agent from the adversary, by performing a suboptimal action. The proposed method optimizes the trade-off between obfuscating the private belief and limiting the increase in cost accrued due to taking a suboptimal action. We propose a probabilistic relaxation to obtain a linear optimization problem for solving the trade-off. In numerical examples, we show that the proposed methods enable the agent to obfuscate its private belief without compromising its cost budget., Comment: 7 pages, 7 figures

In this paper, we derive the asymptotic Cram\'er-Rao lower bound for the continuous-time output error model structure and provide an analysis of the statistical efficiency of the Simplified Refined Instrumental Variable method for Continuous-time systems (SRIVC) based on sampled data.It is shown that the asymptotic Cram\'er-Rao lower bound is independent of the intersample behaviour of the noise-free system output and hence only depends on the intersample behaviour of the system input. We have also shown that, at the converging point of the SRIVC algorithm, the estimates do not depend on the intersample behaviour of the measured output. It is then proven that the SRIVC estimator is asymptotically efficient for the output error model structure under mild conditions. Monte Carlo simulations are performed to verify the asymptotic Cram\'er-Rao lower bound and the asymptotic covariance of the SRIVC estimates., Comment: 11 pages, 2 figures. Preprint submitted to Automatica

Bayesian filtering deals with computing the posterior distribution of the state of a stochastic dynamic system given noisy observations. In this paper, motivated by applications in counter-adversarial systems, we consider the following inverse filtering problem: Given a sequence of posterior distributions from a Bayesian filter, what can be inferred about the transition kernel of the state, the observation likelihoods of the sensor and the measured observations? For finite-state Markov chains observed in noise (hidden Markov models), we show that a least-squares fit for estimating the parameters and observations amounts to a combinatorial optimization problem with non-convex objective. Instead, by exploiting the algebraic structure of the corresponding Bayesian filter, we propose an algorithm based on convex optimization for reconstructing the transition kernel, the observation likelihoods and the observations. We discuss and derive conditions for identifiability. As an application of our results, we illustrate the design of counter-adversarial systems: By observing the actions of an autonomous enemy, we estimate the accuracy of its sensors and the observations it has received. The proposed algorithms are evaluated in numerical examples.

In this paper, we study non-asymptotic deviation bounds of the least squares estimator in Gaussian AR($n$) processes. By relying on martingale concentration inequalities and a tail-bound for $\chi^2$ distributed variables, we provide a concentration bound for the sample covariance matrix of the process output. With this, we present a problem-dependent finite-time bound on the deviation probability of any fixed linear combination of the estimated parameters of the AR$(n)$ process. We discuss extensions and limitations of our approach., Comment: 15 pages

We address the new problem of estimating a piece-wise constant signal with the purpose of detecting its change points and the levels of clusters. Our approach is to model it as a nonparametric penalized least square model selection on a family of models indexed over the collection of partitions of the design points and propose a computationally efficient algorithm to approximately solve it. Statistically, minimizing such a penalized criterion yields an approximation to the maximum a posteriori probability (MAP) estimator. The criterion is then analyzed and an oracle inequality is derived using a Gaussian concentration inequality. The oracle inequality is used to derive on one hand conditions for consistency and on the other hand an adaptive upper bound on the expected square risk of the estimator, which statistically motivates our approximation. Finally, we apply our algorithm to simulated data to experimentally validate the statistical guarantees and illustrate its behavior., Comment: 37 page, 4 figures, Proceedings of the 35th International Conference on Machine Learning (ICML), PMLR 80:3433-3442, 2018

We consider a finite impulse response system with centered independent sub-Gaussian design covariates and noise components that are not necessarily identically distributed. We derive non-asymptotic near-optimal estimation and prediction bounds for the least-squares estimator of the parameters. Our results are based on two concentration inequalities on the norm of sums of dependent covariate vectors and on the singular values of their covariance operator that are of independent value on their own and where the dependence arises from the time shift structure of the time series. These results generalize the known bounds for the independent case., Comment: 23 pages

In this paper, we study finite-sample properties of the least squares estimator in first order autoregressive processes. By leveraging a result from decoupling theory, we derive upper bounds on the probability that the estimate deviates by at least a positive $\varepsilon$ from its true value. Our results consider both stable and unstable processes. Afterwards, we obtain problem-dependent non-asymptotic bounds on the variance of this estimator, valid for sample sizes greater than or equal to seven. Via simulations we analyze the conservatism of our bounds, and show that they reliably capture the true behavior of the quantities of interest., Comment: 9 pages, 2 figures

We consider fixed-interval smoothing problems for counter-adversarial autonomous systems. An adversary deploys an autonomous filtering and control system that i) measures our current state via a noisy sensor, ii) computes a posterior estimate (belief) and iii) takes an action that we can observe. Based on such observed actions and our knowledge of our state sequence, we aim to estimate the adversary's past and current beliefs -- this forms a foundation for predicting, and counteracting against, future actions. We derive the optimal smoother for the adversary's beliefs (we treat the problem in a Bayesian framework). Moreover, we demonstrate how the smoother can be computed for discrete systems even though the corresponding backward variables do not admit a finite-dimensional characterization. Finally, we illustrate our results in numerical simulations.

In this paper, we analyse the consistency of the Simplified Refined Instrumental Variable method for Continuous-time systems (SRIVC). It is well known that the intersample behaviour of the input signal influences the quality and accuracy of the results when estimating and simulating continuous-time models. Here, we present a comprehensive analysis on the consistency of the SRIVC estimator while taking into account the intersample behaviour of the input signal. The main result of the paper shows that, under some mild conditions, the SRIVC estimator is generically consistent. We also describe some conditions when consistency is not achieved, which is important from a practical standpoint. The theoretical results are supported by simulation examples.

The indirect approach to continuous-time system identification consists in estimating continuous-time models by first determining an appropriate discrete-time model. For a zero-order hold sampling mechanism, this approach usually leads to a transfer function estimate with relative degree 1, independent of the relative degree of the strictly proper real system. In this paper, a refinement of these methods is developed. Inspired by indirect PEM, we propose a method that enforces a fixed relative degree in the continuous-time transfer function estimate, and show that the resulting estimator is consistent and asymptotically efficient. Extensive numerical simulations are put forward to show the performance of this estimator when contrasted with other indirect and direct methods for continuous-time system identification., Comment: 6 pages, 2 figures

In identification of dynamical systems, the prediction error method using a quadratic cost function provides asymptotically efficient estimates under Gaussian noise and additional mild assumptions, but in general it requires solving a non-convex optimization problem. An alternative class of methods uses a non-parametric model as intermediate step to obtain the model of interest. Weighted null-space fitting (WNSF) belongs to this class. It is a weighted least-squares method consisting of three steps. In the first step, a high-order ARX model is estimated. In a second least-squares step, this high-order estimate is reduced to a parametric estimate. In the third step, weighted least squares is used to reduce the variance of the estimates. The method is flexible in parametrization and suitable for both open- and closed-loop data. In this paper, we show that WNSF provides estimates with the same asymptotic properties as PEM with a quadratic cost function when the model orders are chosen according to the true system. Also, simulation studies indicate that WNSF may be competitive with state-of-the-art methods.

Standard system identification methods often provide inconsistent estimates with closed-loop data. With the prediction error method (PEM), this issue is solved by using a noise model that is flexible enough to capture the noise spectrum. However, a too flexible noise model (i.e., too many parameters) increases the model complexity, which can cause additional numerical problems for PEM. In this paper, we consider the weighted null-space fitting (WNSF) method. With this method, the system is first modeled using a non-parametric ARX model, which is then reduced to a parametric model of interest using weighted least squares. In the reduction step, a parametric noise model does not need to be estimated if it is not of interest. Because the flexibility of the noise model is increased with the sample size, this will still provide consistent estimates in closed loop and asymptotically efficient estimates in open loop. In this paper, we prove these results, and we derive the asymptotic covariance for the estimation error obtained in closed loop, which is optimal for an infinite-order noise model. For this purpose, we also derive a new technical result for geometric variance analysis, instrumental to our end. Finally, we perform a simulation study to illustrate the benefits of the method when the noise model cannot be parametrized by a low-order model.

Trial-varying disturbances are a key concern in Iterative Learning Control (ILC) and may lead to inefficient and expensive implementations and severe performance deterioration. The aim of this paper is to develop a general framework for optimization-based ILC that allows for enforcing additional structure, including sparsity. The proposed method enforces sparsity in a generalized setting through convex relaxations using $\ell_1$ norms. The proposed ILC framework is applied to the optimization of sampling sequences for resource efficient implementation, trial-varying disturbance attenuation, and basis function selection. The framework has a large potential in control applications such as mechatronics, as is confirmed through an application on a wafer stage., Comment: 12 pages, 14 figures

This paper discusses algorithms for solving Markov decision processes (MDPs) that have monotone optimal policies. We propose a two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by exploiting the monotone property. The first stage is a linear program formulated in terms of the joint state-action probabilities. The second stage is a regularized problem formulated in terms of the conditional probabilities of actions given states. The regularization uses techniques from nearly-isotonic regression. While a variety of iterative method can be used in the first formulation of the problem, we show in numerical simulations that, in particular, the alternating method of multipliers (ADMM) can be significantly accelerated using the regularization step., Comment: This work has been accepted for presentation at the 20th World Congress of the International Federation of Automatic Control, 9-14 July 2017

In this paper, we develop an upper bound for the SPARSEVA (SPARSe Estimation based on a VAlidation criterion) estimation error in a general scheme, i.e., when the cost function is strongly convex and the regularized norm is decomposable for a pair of subspaces. We show how this general bound can be applied to a sparse regression problem to obtain an upper bound for the traditional SPARSEVA problem. Numerical results are used to illustrate the effectiveness of the suggested bound.