Search

Objective. Developing a new neuromodulation method for epilepsy treatment requires a large amount of time and resources to find effective stimulation parameters and often fails due to inter-subject variability in stimulation effect. As an alternative, we present a novel data-driven surrogate approach which can optimize the neuromodulation efficiently by investigating the stimulation effect on surrogate neural states. Approach. Medial septum (MS) optogenetic stimulation was applied for modulating electrophysiological activities of the hippocampus in a rat temporal lobe epilepsy model. For the new approach, we implemented machine learning techniques to describe the pathological neural states and to optimize the stimulation parameters. Specifically, first, we found neural state surrogates to estimate a seizure susceptibility based on hippocampal local field potentials. Second, we modulated the neural state surrogates in a desired way with the subject-specific optimal stimulation parameters found by in vivo Bayesian optimization. Finally, we tested whether modulating the neural state surrogates affected seizure frequency. Main results. We found two neural state surrogates: The first was hippocampal theta power by considering its well-known relationship with epilepsy, and the second was the output of pre-ictal state model (PriSM) which was built by characterizing the hippocampal activity during pre-ictal period. The optimal stimulation parameters found by Bayesian optimization outperformed the other parameters in terms of modulating the surrogates toward anti-seizure neural state. When treatment efficacy was tested, the subject-specific optimal parameters for increasing theta power were more effective to suppress seizures than fixed stimulation parameter (7Hz). However, modulation of the other neural state surrogate, PriSM, did not suppress seizures. Significance. The surrogate approach can save enormous time and resources to find subject-specific optimal stimulation parameters which can effectively modulate neural states and further improve therapeutic effectiveness. This approach also can be used for improving neuromodulation treatment of other neurological or psychiatric diseases.

The aim of this work is to present a sampling-based algorithm designed to solve a certain class of stochastic optimal control problems, utilizing forward and backward stochastic differential equations (FBSDEs). Specifically, we address the class of problems in which the running cost of the performance index involves an L 1 -type minimization problem in terms of the control effort. Such problems are typically called minimum fuel problems in optimal control literature. By means of a nonlinear version of the Feynman–Kac lemma, we obtain a probabilistic representation of the solution to the nonlinear Hamilton–Jacobi–Bellman equation, expressed in the form of a system of decoupled FBSDEs. This system of FBSDEs can be solved by employing linear regression techniques.

The aim of this work is to present a sampling-based algorithm designed to solve various classes of stochastic differential games. The foundation of the proposed approach lies in the formulation of the game solution in terms of a decoupled pair of forward and backward stochastic differential equations (FBSDEs). In light of the nonlinear version of the Feynman–Kac lemma, probabilistic representations of solutions to the nonlinear Hamilton–Jacobi–Isaacs equations that arise for each class are obtained. These representations are in form of decoupled systems of FBSDEs, which may be solved numerically.

The aim of this work is to present a novel sampling-based numerical scheme designed to solve a certain class of stochastic optimal control problems, utilizing forward and backward stochastic differential equations (FBSDEs). By means of a nonlinear version of the Feynman–Kac lemma, we obtain a probabilistic representation of the solution to the nonlinear Hamilton–Jacobi–Bellman equation, expressed in the form of a system of decoupled FBSDEs. This system of FBSDEs can be solved by employing linear regression techniques. The proposed framework relaxes some of the restrictive conditions present in recent sampling based methods within the Linearly Solvable Optimal Control framework, and furthermore addresses problems in which the time horizon is not prespecified. To enhance the efficiency of the proposed scheme when treating more complex nonlinear systems, we then derive an iterative algorithm based on Girsanov’s theorem on the change of measure, which features importance sampling. This scheme is shown to be capable of learning the optimal control without requiring an initial guess.

This study presents a model predictive control approach for seizure reduction in a computational model of epilepsy. The differential dynamic programming (DDP) algorithm is implemented in a model predictive fashion to optimize a controller for suppressing seizures in a chaotic oscillator model. The chaotic oscillator model uses proportional-integral (PI) controller to represent the internal control mechanism that maintains stable neural activity in a healthy brain. In the pathological case, the gains of this PI controller are reduced, preventing sufficient feedback to suppress correlation increase between normal and pathological brain dynamics. This increase in correlation leads to synchronization of oscillator dynamics leading to the destabilization of neural activity and epileptic behavior. The pathological case of the chaotic oscillator model is formulated as an optimal control problem, which we solve using the dynamic programming principle. We propose using model predictive control with differential dynamic programming optimization as a possible method for controlling epileptic seizures in known models of epilepsy.

In this work we present a sampling-based algorithm designed to solve game-theoretic control problems and risk-sensitive stochastic optimal control problems. The cornerstone of the proposed approach is the formulation of the problem in terms of forward and backward stochastic differential equations (FBSDEs). By means of a nonlinear version of the Feynman-Kac lemma, we obtain a probabilistic representation of the solution to the nonlinear Hamilton-Jacobi-Isaacs equation, expressed in the form of a decoupled system of FBSDEs. This system of FBSDEs can then be simulated by employing linear regression techniques. Utilizing the connection between stochastic differential games and risk-sensitive optimal control, we demonstrate that the proposed algorithm is also applicable to the latter class of problems. Simulation results validate the algorithm.

We consider the following differential game of pursuit and evasion involving two unmanned aerial vehicle agents (UAVs): an evading UAV (“evader”), which has limited maneuverability, and a pursuing UAV (“pursuer”), which is assumed to be completely agile. The two UAVs move on the Euclidean plane with different but constant speeds. Whereas the pursuer can change the orientation of its velocity vector arbitrarily fast, that is, she is a “pedestrian” a la Isaacs, the evader cannot make turns having a radius smaller than a specified minimum turning radius. This problem can be seen as a reversed roles Homicidal Chauffeur game, hence the name “Suicidal Pedestrian” differential game. The aim of this paper is to derive the optimal strategies of the two players and characterize the initial conditions that lead to capture if the pursuer acts optimally, along with the states that guarantee evasion regardless of the pursuer’s strategy. Both proximity-capture and point-capture are considered. This is a rare instance where a complete solution of a differential game can be obtained in closed-form. Utilizing the connection to Zermelo’s Navigation Problem, the two-player solution is generalized under some assumptions for the case of multiple pursuer UAVs. The results are directly applicable to collision avoidance, in the sense that safety regions are delineated in which no collision is possible, even in the worst case of malicious intent by one of the pursuers.

In this paper we present a novel sampling-based numerical scheme designed to solve a certain class of stochastic optimal control problems, utilizing forward and backward stochastic differential equations (FBSDEs). By means of a nonlinear version of the Feynman-Kac lemma, we obtain a probabilistic representation of the solution to the nonlinear Hamilton-Jacobi-Bellman equation, expressed in the form of a decoupled system of FBSDEs. This system of FBSDEs can then be simulated by employing linear regression techniques. To enhance the efficiency of the proposed scheme when treating more complex nonlinear systems, we then derive an iterative modification based on Girsanov's theorem on the change of measure, which features importance sampling. The modified scheme is capable of learning the optimal control without requiring an initial guess. We present simulations that validate the algorithm and demonstrate its efficiency in treating nonlinear dynamics.