Search

Conformal prediction (CP) is a distribution-free framework for achieving probabilistic guarantees on black-box models. CP is generally applied to a model post-training. Recent research efforts, on the other hand, have focused on optimizing CP efficiency during training. We formalize this concept as the problem of conformal risk minimization (CRM). In this direction, conformal training (ConfTr) by Stutz et al.(2022) is a technique that seeks to minimize the expected prediction set size of a model by simulating CP in-between training updates. Despite its potential, we identify a strong source of sample inefficiency in ConfTr that leads to overly noisy estimated gradients, introducing training instability and limiting practical use. To address this challenge, we propose variance-reduced conformal training (VR-ConfTr), a CRM method that incorporates a variance reduction technique in the gradient estimation of the ConfTr objective function. Through extensive experiments on various benchmark datasets, we demonstrate that VR-ConfTr consistently achieves faster convergence and smaller prediction sets compared to baselines.

With the pervasiveness of Stochastic Shortest-Path (SSP) problems in high-risk industries, such as last-mile autonomous delivery and supply chain management, robust planning algorithms are crucial for ensuring successful task completion while mitigating hazardous outcomes. Mainstream chance-constrained incremental sampling techniques for solving SSP problems tend to be overly conservative and typically do not consider the likelihood of undesirable tail events. We propose an alternative risk-aware approach inspired by the asymptotically-optimal Rapidly-Exploring Random Trees (RRT*) planning algorithm, which selects nodes along path segments with minimal Conditional Value-at-Risk (CVaR). Our motivation rests on the step-wise coherence of the CVaR risk measure and the optimal substructure of the SSP problem. Thus, optimizing with respect to the CVaR at each sampling iteration necessarily leads to an optimal path in the limit of the sample size. We validate our approach via numerical path planning experiments in a two-dimensional grid world with obstacles and stochastic path-segment lengths. Our simulation results show that incorporating risk into the tree growth process yields paths with lengths that are significantly less sensitive to variations in the noise parameter, or equivalently, paths that are more robust to environmental uncertainty. Algorithmic analyses reveal similar query time and memory space complexity to the baseline RRT* procedure, with only a marginal increase in processing time. This increase is offset by significantly lower noise sensitivity and reduced planner failure rates., Comment: Accepted for presentation at the 2024 IEEE Conference on Decision and Control (CDC)

A state-feedback watermarking signal design for the detection of replay attacks in linear systems is proposed. The control input is augmented with a random time-delayed term of the system state estimate, in order to secure the system against attacks of replay type. We outline the basic analysis of the closed-loop response of the state-feedback watermarking in a LQG controlled system. Our theoretical results are applied on a temperature process control example. While the proposed secure control scheme requires very involved analysis, it, nevertheless, holds promise of being superior to conventional, feed-forward, watermarking schemes, in both its ability to detect attacks as well as the secured system performance., Comment: 6 pages, 3 figures, manuscript to appear in the 2024 American Control Conference

Real-world scenarios are characterized by timing uncertainties, e.g., delays, and disturbances. Algorithms with temporal robustness are crucial in guaranteeing the successful execution of tasks and missions in such scenarios. We study time-robust path planning for synthesizing robots' trajectories that adhere to spatial-temporal specifications expressed in Signal Temporal Logic (STL). In contrast to prior approaches that rely on {discretize}d trajectories with fixed time steps, we leverage Piece-Wise Linear (PWL) signals for the synthesis. PWL signals represent a trajectory through a sequence of time-stamped waypoints. This allows us to encode the STL formula into a Mixed-Integer Linear Program (MILP) with fewer variables. This reduction is more pronounced for specifications with a long planning horizon. To that end, we define time-robustness for PWL signals. Subsequently, we propose quantitative semantics for PWL signals according to the recursive syntax of STL and prove their soundness. We then propose an encoding strategy to transform our semantics into a MILP. Our simulations showcase the soundness and the performance of our algorithm.

Reinforcement Learning (RL) has shown exceptional performance across various applications, enabling autonomous agents to learn optimal policies through interaction with their environments. However, traditional RL frameworks often face challenges in terms of iteration complexity and robustness. Risk-sensitive RL, which balances expected return and risk, has been explored for its potential to yield probabilistically robust policies, yet its iteration complexity analysis remains underexplored. In this study, we conduct a thorough iteration complexity analysis for the risk-sensitive policy gradient method, focusing on the REINFORCE algorithm and employing the exponential utility function. We obtain an iteration complexity of $\cO(\epsilon^{-2})$ to reach an $\epsilon$-approximate first-order stationary point (FOSP). We investigate whether risk-sensitive algorithms can potentially achieve better iteration complexity compared to their risk-neutral counterparts. Our theoretical analysis demonstrates that risk-sensitive REINFORCE can potentially have a reduced number of iterations required for convergence. This leads to improved iteration complexity, as employing the exponential utility does not entail additional computation per iteration. We characterize the conditions under which risk-sensitive algorithms can potentially achieve better iteration complexity. Our simulation results also validate that risk-averse cases can converge and stabilize more quickly after $41\%$ of the episodes compared to their risk-neutral counterparts.

Envisioned application areas for reinforcement learning (RL) include autonomous driving, precision agriculture, and finance, which all require RL agents to make decisions in the real world. A significant challenge hindering the adoption of RL methods in these domains is the non-robustness of conventional algorithms. In this paper, we argue that a fundamental issue contributing to this lack of robustness lies in the focus on the expected value of the return as the sole ``correct'' optimization objective. The expected value is the average over the statistical ensemble of infinitely many trajectories. For non-ergodic returns, this average differs from the average over a single but infinitely long trajectory. Consequently, optimizing the expected value can lead to policies that yield exceptionally high returns with probability zero but almost surely result in catastrophic outcomes. This problem can be circumvented by transforming the time series of collected returns into one with ergodic increments. This transformation enables learning robust policies by optimizing the long-term return for individual agents rather than the average across infinitely many trajectories. We propose an algorithm for learning ergodicity transformations from data and demonstrate its effectiveness in an instructive, non-ergodic environment and on standard RL benchmarks.

Humans have the ability to deviate from their natural behavior when necessary, which is a cognitive process called response inhibition. Similar approaches have independently received increasing attention in recent years for ensuring the safety of control. Realized using control barrier functions or predictive safety filters, these approaches can effectively ensure the satisfaction of state constraints through an online adaptation of nominal control laws, e.g., obtained through reinforcement learning. While the focus of these realizations of inhibitory control has been on risk-neutral formulations, human studies have shown a tight link between response inhibition and risk attitude. Inspired by this insight, we propose a flexible, risk-sensitive method for inhibitory control. Our method is based on a risk-aware condition for value functions, which guarantees the satisfaction of state constraints. We propose a method for learning these value functions using common techniques from reinforcement learning and derive sufficient conditions for its success. By enforcing the derived safety conditions online using the learned value function, risk-sensitive inhibitory control is effectively achieved. The effectiveness of the developed control scheme is demonstrated in simulations., Comment: The 62nd IEEE Conference on Decision and Control, Dec. 13-15, 2023, Singapore

There is an emerging interest in generating robust counterfactual explanations that would remain valid if the model is updated or changed even slightly. Towards finding robust counterfactuals, existing literature often assumes that the original model $m$ and the new model $M$ are bounded in the parameter space, i.e., $\|\text{Params}(M){-}\text{Params}(m)\|{<}\Delta$. However, models can often change significantly in the parameter space with little to no change in their predictions or accuracy on the given dataset. In this work, we introduce a mathematical abstraction termed $\textit{naturally-occurring}$ model change, which allows for arbitrary changes in the parameter space such that the change in predictions on points that lie on the data manifold is limited. Next, we propose a measure -- that we call $\textit{Stability}$ -- to quantify the robustness of counterfactuals to potential model changes for differentiable models, e.g., neural networks. Our main contribution is to show that counterfactuals with sufficiently high value of $\textit{Stability}$ as defined by our measure will remain valid after potential $\textit{naturally-occurring}$ model changes with high probability (leveraging concentration bounds for Lipschitz function of independent Gaussians). Since our quantification depends on the local Lipschitz constant around a data point which is not always available, we also examine practical relaxations of our proposed measure and demonstrate experimentally how they can be incorporated to find robust counterfactuals for neural networks that are close, realistic, and remain valid after potential model changes. This work also has interesting connections with model multiplicity, also known as, the Rashomon effect., Comment: International Conference on Machine Learning (ICML), 2023

While reinforcement learning has shown experimental success in a number of applications, it is known to be sensitive to noise and perturbations in the parameters of the system, leading to high variance in the total reward amongst different episodes in slightly different environments. To introduce robustness, as well as sample efficiency, risk-sensitive reinforcement learning methods are being thoroughly studied. In this work, we provide a definition of robust reinforcement learning policies and formulate a risk-sensitive reinforcement learning problem to approximate them, by solving an optimization problem with respect to a modified objective based on exponential criteria. In particular, we study a model-free risk-sensitive variation of the widely-used Monte Carlo Policy Gradient algorithm and introduce a novel risk-sensitive online Actor-Critic algorithm based on solving a multiplicative Bellman equation using stochastic approximation updates. Analytical results suggest that the use of exponential criteria generalizes commonly used ad-hoc regularization approaches, improves sample efficiency, and introduces robustness with respect to perturbations in the model parameters and the environment. The implementation, performance, and robustness properties of the proposed methods are evaluated in simulated experiments.

Electron dynamics, financial markets and nuclear fission reactors, though seemingly unrelated, all produce observable characteristics evolving with time. Within this broad scope, departures from normal temporal behavior range from academically interesting to potentially catastrophic. New algorithms for time series anomaly detection (TAD) are therefore certainly in demand. With the advent of newly accessible quantum processing units (QPUs), exploring a quantum approach to TAD is now relevant and is the topic of this work. Our approach - Quantum Variational Rewinding, or, QVR - trains a family of parameterized unitary time-devolution operators to cluster normal time series instances encoded within quantum states. Unseen time series are assigned an anomaly score based upon their distance from the cluster center, which, beyond a given threshold, classifies anomalous behavior. After a first demonstration with a simple and didactic case, QVR is used to study the real problem of identifying anomalous behavior in cryptocurrency market data. Finally, multivariate time series from the cryptocurrency use case are studied using IBM's Falcon r5.11H family of superconducting transmon QPUs, where anomaly score errors resulting from hardware noise are shown to be reducible by as much as 20% using advanced error mitigation techniques., Comment: Main article: 11 pages and 4 figures. Supplemental material: 11 pages, 6 figures and 2 tables

This work discusses a novel framework for simultaneous synthesis of optimal watermarking signal and robust controllers in cyber-physical systems to minimize the loss in performance due to added watermarking signal and to maximize the detection rate of the attack. A general dynamic controller is designed to improve system performance with respect to the $\mathcal H_2$ norm, while a watermarking signal is added to improve security performance concerning the detection rate of replay attacks. The attack model considered in the paper is a replay attack, a natural attack mode when the dynamics of the system is unknown to the attacker. The paper first generalizes the existing result on the detection rate of $\chi^2$ detector from a static-LQR controller to a general dynamic controller. The design improvements on both robustness and security fronts are obtained by iteratively solving the convex subsets of the formulated non-convex problem in terms of the controller and watermarking signal. A semi-definite programming optimization is formulated using Linear Matrix Inequality (LMI) results to solve the larger system-level design optimization problem. We highlight the effectiveness of our method over a simplified three-tank chemical system.

In this paper, we propose a numerical scheme of the predictor-corrector type for solving nonlinear fractional initial value problems, the chosen fractional derivative is called the Atangana-Baleanu derivative defined in Caputo sense (ABC). This proposed method is based on Lagrangian quadratic polynomials to approximate the nonlinearity implied in the Volterra integral which is obtained by reducing the given fractional differential equation via the properties of the ABC-fractional derivative. Through this technique, we get corrector formula with high accuracy which is implicit as well as predictor formula which is explicit and has the same precision order as the corrective formula. On the other hand, the so-called memory term is computed only once for both prediction and correction phases, which indicates the low cost of the proposed method. Also, the error bound of the proposed numerical scheme is offered. Furthermore, numerical experiments are presented in order to assess the accuracy of the new method on two differential equations. Moreover, a case study is considered where the proposed predictor-corrector scheme is used to obtained approximate solutions of ABC-fractional generalized SI (susceptible-infectious) epidemic model for the purpose of analyzing dynamics of the suggested system as well as demonstrating the effectiveness of the new method to solve systems dealing with real-world problems.

Investigations on diverse particulate matter can be carried out by non-invasive and non-destructive analytical means of light scattering technique. Exploration of small particles using this tool must address the concept of refractive index accurately. Interpretation of light scattering results for quantification of morphology of biological particles like viruses and bacterial cells can only be validated by exact evidence of its refractive index. Size and shape quantification of such particles by collecting information from the scattered light, was reported earlier by many researchers but parameter like refractive index was considered as a standard value employed from literature or selected as a median value of biological particles which are alike. A strong analysis is needed on inclusion of appropriate value of refractive index for revelation of light scattering results. This important perception is otherwise not considered strongly in similar works reported earlier. We primarily took an attempt to find the influence of refractive index in light scattering studies with allusion to biological particles like Staphylococcus aureus, Escherichia coli, Corona virus (SARS COV 2) etc., Comment: 14 pages

A sofic shift is a shift space consisting of bi-infinite labels of paths from a labelled graph. Being a dynamical system, the distribution of its closed orbits may indicate the complexity of the space. For this purpose, prime orbit and Mertens' orbit counting functions are introduced as a way to describe the growth of the closed orbits. The asymptotic behaviours of these counting functions can be implied from the analiticity of the Artin-Mazur zeta function of the space. Despite having a closed-form expression, the zeta function is expressed implicitly in terms of several signed subset matrices, and this makes the study on its analyticity to be seemingly difficult. In this paper, we will prove the asymptotic behaviours of the counting functions for a sofic shift via its zeta function. This involves investigating the properties of the said matrices. Suprisingly, the proof is rather short and only uses well-known facts about a sofic shift, especially on its minimal right-resolving presentation., Comment: 11 pages