Search

The operation of the three-phase standalone inverter is affected by many factors, such as heavy changes in load, unbalanced loads, nonlinear loads, system uncertainties and external disturbances. These are critical weaknesses for the control system of the three-phase inverter to reach robust optimal performance. This paper proposes an adaptive optimal sliding mode control (AOSMC) scheme for a three-phase nonlinear uncertain inverter. This AOSMC strategy solves the problems of nonlinear optimization by adaptive dynamic programming, one of the techniques of reinforcement learning, and overcomes the uncertainties and disturbance effects by a disturbance observer–based sliding mode controller. This algorithm uses only one neural network to approximate the critic; therefore, the burden of computation is significantly reduced. Both the weight matrix of the critic network and the disturbance observer are asymptotically stable. The overall system is guaranteed to be ultimately uniformly bounded stable via Lyapunov stable theory. The simulation is conducted to validate the insensitivity of the proposed AOSMC algorithm to working conditions. Also, the competitive results are presented to demonstrate the improvement of the proposed AOSMC scheme in comparison to some other existing controllers. [ABSTRACT FROM AUTHOR]

In this paper, an optimisation problem for the monotone mean-variance (MMV) criterion is considered from the perspective of the insurance company. The MMV criterion is an amended version of the classical mean-variance (MV) criterion which guarantees the monotonicity of the utility function. With this criterion we study the optimal investment and reinsurance problem which is formulated as a zero-sum game between the insurance company and an imaginary player. We apply the dynamic programming principle to obtain the corresponding Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation. As the main conclusion of this paper, by solving the HJBI equation explicitly, the closed forms of the optimal strategy and the value function are obtained. Moreover, the MMV efficient frontier is also provided. At the end of the paper, a numerical example is presented. [ABSTRACT FROM AUTHOR]

Water demand continuously increases in urban zones, and water scarcity is frequently associated with a lack or a reduction of water availability at water sources and maintenance problems, such as leakage and pipe aging. These facts inevitably lead to challenging water distribution system (WDS) management. In this scenario, intermittent operation emerges as an alternative to system operation. This is not the most desirable solution from a social perspective because many consumers cannot be supplied as desired for days. To tackle problems like this, many works have investigated how to help decision makers improve water system efficiency during the last decades. Nevertheless, few works have considered combining several structural interventions, such as pipe replacement, installation of new pump stations, fixing leaks, and installing and controlling pump stations and valves. One reason is that alternatives for recovering the hydraulic capacity in decision-making processes are computationally burdensome, mathematically complex, and, sometimes, even physically incompatible. Considering the problem stated by the Battle of Intermittent Water Supply, this work proposes a methodology for optimal operation and recovery of a WDS. The Battle problem is presented in two stages organized during different years: Year 0 and the following 5 years. For Year 0, only operational optimization is allowed. Consequently, optimal operation of pumps and valves is proposed for this initial year to maximize the number of nodes being supplied. For the rest of the years, because implementing structural changes is allowed within a defined budget, the proposal suggests applying a search space reduction process based on a cost-benefit trade-off and the hydraulic relevance of each structural alternative evaluated individually in terms of the nine indicators proposed in the Battle statement. Those alternatives that better improve the indicators are then considered in a multiobjective optimization setting. For every year, a set of structural changes is selected, followed by related changes in the operational setup. The alternatives are selected year by year and evaluated considering the past selected alternatives to assess the effects during the five evaluation years. This is done in a dynamic programming process, ensuring that a near optimal is achieved by the end of the last, fifth, year. [ABSTRACT FROM AUTHOR]

In this article, a multi‐event‐triggered (MET) near‐optimal tracking control algorithm is developed for a discrete‐time (DT) nonlinear non‐zero‐sum game (NZSG). Firstly, the iterative dual heuristic dynamic programming (DHP) approach is utilized to obtain the optimal tracking control set. Subsequently, distinct independent triggering conditions are formulated for control inputs to improve the utilization efficiency of resources and ensure that the independence between them and the closed‐loop system has an excellent control performance. Meanwhile, input‐to‐state stability (ISS) is proven for the MET tracking plant. Additionally, neural networks (NNs) are designed in the iterative DHP algorithm to construct the model module, the critic module and the action modules, with aims to identify the unknown system, approximate costate function and estimate optimal tracking control strategy set, respectively. Finally, two numerical experimental simulations are conducted to verify the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

In this paper, the H∞$$ {H}_{\infty } $$ optimal output tracking control problem for Markov jump systems is investigated, where the two cases with known or completely unknown transition probabilities are both considered. Based on game theory and H∞$$ {H}_{\infty } $$ performance, quadratic cost is considered, where a discount parameter is introduced into the quadratic cost in order to track unstable systems and eliminate the assumption that the noise energy is bounded. The game coupled algebraic Riccati equation and the corresponding controller are presented by dynamic programming. The stochastic stability of the tracking error system is further investigated. Moreover, iterative and reinforcement learning‐based algorithms are proposed for solving the H∞$$ {H}_{\infty } $$ optimal tracking controller with known or completely unknown transition probabilities, respectively. Finally, some numerical simulations on a DC motor are performed to validate the effectiveness of the proposed results. [ABSTRACT FROM AUTHOR]

The advancement of autonomous capabilities in maritime navigation has gained significant attention, with a trajectory moving from decision support systems to full autonomy. This push towards autonomy has led to extensive research focusing on collision avoidance, a critical aspect of safe navigation. Among the various possible approaches, dynamic programming is a promising tool for optimizing collision avoidance maneuvers. This paper presents a DP formulation for the collision avoidance of autonomous vessels. We set up the problem framework, formulate it as a multi-stage decision process, define cost functions and constraints focusing on the actual requirements a marine maneuver must comply with, and propose a solution algorithm leveraging parallel computing. Additionally, we present a greedy approximation to reduce algorithm complexity. We put the proposed algorithms to the test in realistic navigation scenarios and also develop an extensive test on a large set of randomly generated scenarios, comparing them with the RRT* algorithm using performance metrics proposed in the literature. The results show the potential benefits of an autonomous navigation or decision support framework. [ABSTRACT FROM AUTHOR]

AbstractWe consider the problem of clustering with K-means and Gaussian mixture models with a constraint on the separation between the centers in the context of real-valued data. We first propose a dynamic programming approach to solving the K-means problem with a separation constraint on the centers, building on Wang and Song (2011). In the context of fitting a Gaussian mixture model, we then propose an EM algorithm that incorporates such a constraint. A separation constraint can help regularize the output of a clustering algorithm, and we provide both simulated and real data examples to illustrate this point. [ABSTRACT FROM AUTHOR]

In this paper, a novel parametrized guaranteed cost control (PGCC) method with prescribed performance control (PPC) is proposed for nonlinear systems with unknown bounded uncertainties and external disturbances described by Euler‐Lagrange (EL) equations. The proposed method is capable of ensuring a robust optimal performance in the presence of dynamics constraints without the specific information of nonlinear perturbation. As a consequence, the conditions in the form of disturbances and uncertainties are also relaxed. The augmented model is converted by transforming the uncertainties and disturbances in the original system to a mismatched term using asymmetric PPC. By designing and modifying the novel guaranteed cost function to account for the maximum value of the nonlinear perturbation, an infinite‐horizon PGCC problem is proposed by the unconstrained stationary optimal control problem. The relevant proposed linear parametrized Hamilton‐Jacobian‐Bellman (PHJB) equation is approximated to solve by a critic‐only neural network (NN) with efficient computationally. Uniformly ultimately bounded stability is guaranteed via a Lyapunov‐based stability analysis. Finally, numerical simulation results demonstrate the effectiveness of the proposed control scheme. [ABSTRACT FROM AUTHOR]

Background: Given a sequencing read, the broad goal of read mapping is to find the location(s) in the reference genome that have a "similar sequence". Traditionally, "similar sequence" was defined as having a high alignment score and read mappers were viewed as heuristic solutions to this well-defined problem. For sketch-based mappers, however, there has not been a problem formulation to capture what problem an exact sketch-based mapping algorithm should solve. Moreover, there is no sketch-based method that can find all possible mapping positions for a read above a certain score threshold. Results: In this paper, we formulate the problem of read mapping at the level of sequence sketches. We give an exact dynamic programming algorithm that finds all hits above a given similarity threshold. It runs in O (| t | + | p | + ℓ 2) time and O (ℓ log ℓ) space, where |t| is the number of k -mers inside the sketch of the reference, |p| is the number of k -mers inside the read's sketch and ℓ is the number of times that k -mers from the pattern sketch occur in the sketch of the text. We evaluate our algorithm's performance in mapping long reads to the T2T assembly of human chromosome Y, where ampliconic regions make it desirable to find all good mapping positions. For an equivalent level of precision as minimap2, the recall of our algorithm is 0.88, compared to only 0.76 of minimap2. [ABSTRACT FROM AUTHOR]

The standard approach for constructing a phylogenetic tree from a set of sequences consists of two key stages. First, a multiple sequence alignment (MSA) of the sequences is computed. The aligned data are then used to reconstruct the phylogenetic tree. The accuracy of the resulting tree heavily relies on the quality of the MSA. The quality of the popularly used progressive sequence alignment depends on a guide tree, which determines the order of aligning sequences. Most MSA methods use pairwise comparisons to generate a distance matrix and reconstruct the guide tree. However, when dealing with highly diverged sequences, constructing a good guide tree is challenging. In this work, we propose an alternative approach using three-way dynamic programming alignment to generate the distance matrix and the guide tree. This three-way alignment incorporates information from additional sequences to compute evolutionary distances more accurately. Using simulated datasets on two symmetric and asymmetric trees, we compared MAFFT with its default guide tree with MAFFT with a guide tree produced using the three-way alignment. We found that (1) the three-way alignment can reconstruct better guide trees than those from the most accurate options of MAFFT, and (2) the better guide tree, on average, leads to more accurate phylogenetic reconstruction. However, the improvement over the L-INS-i option of MAFFT is small, attesting to the excellence of the alignment quality of MAFFT. Surprisingly, the two criteria for choosing the best MSA (phylogenetic accuracy and sum-of-pair score) conflict with each other. [ABSTRACT FROM AUTHOR]

We present Synesth, the most comprehensive and flexible tool for tree reconciliation that allows for events on syntenies (i.e., on sets of multiple genes), including duplications, transfers, fissions, and transient events going through unsampled species. This model allows for building histories that explicate the inconsistencies between a synteny tree and its associated species tree. We examine the combinatorial properties of this extended reconciliation model and study various associated parsimony problems. First, the infinite set of explicatory histories is reduced to a finite but exponential set of Pareto-optimal histories (in terms of counts of each event type), then to a polynomial set of Pareto-optimal event count vectors, and this eventually ends with minimum event cost histories given an event cost function. An inductive characterization of the solution space using different algebras for each granularity leads to efficient dynamic programming algorithms, ultimately ending with an O (m n) time complexity algorithm for computing the cost of a minimum-cost history (m and n: number of nodes in the input synteny and species trees). This time complexity matches that of the fastest known algorithms for classical gene reconciliation with transfers. We show how Synesth can be applied to infer Pareto-optimal evolutionary scenarios for CRISPR-Cas systems in a set of bacterial genomes. [ABSTRACT FROM AUTHOR]

Resource Constraint Project Scheduling Problems with Discounted Cash Flows (RCPSPDC) focuses on maximizing the net present value by summing the discounted cash flows of project activities. An extension of this problem is the Payment at Event Occurrences (PEO) scheme, where the client makes multiple payments to the contractor upon completion of predefined activities, with additional final settlement at project completion. Numerous approximation methods such as metaheuristics have been proposed to solve this NP-hard problem. However, these methods suffer from parameter control and/or the computational cost of correcting infeasible solutions. Alternatively, approximate dynamic programming (ADP) sequentially generates a schedule based on strategies computed via Monte Carlo (MC) simulations. This saves the computations required for solution corrections, but its performance is highly dependent on its strategy. In this study, we propose the hybridization of ADP with three different metaheuristics to take advantage of their combined strengths, resulting in six different models. The Estimation of Distribution Algorithm (EDA) and Ant Colony Optimization (ACO) were used to recommend policies for ADP. A Discrete cCuckoo Search (DCS) further improved the schedules generated by ADP. Our experimental analysis performed on the j30, j60, and j90 datasets of PSPLIB has shown that ADP–DCS is better than ADP alone. Implementing the EDA and ACO as prioritization strategies for Monte Carlo simulations greatly improved the solutions with high statistical significance. In addition, models with the EDA showed better performance than those with ACO and random priority, especially when the number of events increased. [ABSTRACT FROM AUTHOR]

In this work, we study the optimal control of stochastic Burgers equation perturbed by Gaussian and Lévy-type noises with distributed control process acting on the state equation. We use dynamic programming approach for the feedback synthesis to obtain an infinite-dimensional second-order Hamilton–Jacobi–Bellman (HJB) equation consisting of an integro-differential operator with Lévy measure associated with the stochastic control problem. Using the regularizing properties of the transition semigroup corresponding to the stochastic Burgers equation and compactness arguments, we solve the HJB equation and the resultant feedback control problem. [ABSTRACT FROM AUTHOR]

The costs of foraging can be high while also carrying significant risks, especially for consumers feeding at the top of the food chain.To mitigate these risks, many predators supplement active hunting with scavenging and kleptoparasitic behaviours, in some cases specializing in these alternative modes of predation.The factors that drive differential utilization of these tactics from species to species are not well understood.Here, we use an energetics approach to investigate the survival advantages of hunting, scavenging and kleptoparasitism as a function of predator, prey and potential competitor body sizes for terrestrial mammalian carnivores.The results of our framework reveal that predator tactics become more diverse closer to starvation, while the deployment of scavenging and kleptoparasitism is strongly constrained by the ratio of predator to prey body size.Our model accurately predicts a behavioural transition away from hunting towards alternative modes of predation with increasing prey size for predators spanning an order of magnitude in body size, closely matching observational data across a range of species.We then show that this behavioural boundary follows an allometric power‐law scaling relationship where the predator size scales with an exponent nearing 3/4 with prey size, meaning that this behavioural switch occurs at relatively larger threshold prey body size for larger carnivores.We suggest that our approach may provide a holistic framework for guiding future observational efforts exploring the diverse array of predator foraging behaviours. [ABSTRACT FROM AUTHOR]

In this work, exergy management strategies (ExMSs) for hybrid electric ground vehicles (HEVs) are developed. The main advantage of using the exergetic framework is the possibility of pursuing unconventional optimization goals that are inaccessible to the standard energy management strategy (EMS). For instance, in military applications, the critical goal of preventing thermal imaging detection from adversary units does not seem achievable with the conventional EMS. On the other hand, the exergy-based framework can be adopted to reduce the vehicle thermal emissions through the minimization of exergy terms related to heat exchange. Moreover, the overall efficiency of the vehicle can be increased through the minimization of the exergy destruction, a quantity that is not quantifiable by energy-based methods. In this paper, the exergetic model of a series hybrid electric military truck and the exergetic model of the electric induction generator are developed and used to formulate and solve two novel exergy management strategies aiming to minimize genset exergy destruction and thermal emissions, respectively. The optimal solutions to the EMS and ExMSs control problems are obtained through Dynamic Programming over two driving missions. The results show that ExMS for the minimization of exergy destruction achieves similar results to the standard EMS, while the ExMS for the minimization of thermal emissions obtains significantly lower thermal emissions compared to the EMS, effectively reducing the thermal imaging detection risk. [ABSTRACT FROM AUTHOR]

This paper addresses the limited adaptability and the computational burden of energy management systems (EMSs) for hybrid electric vehicles (HEVs) implemented via dynamic programming (DP)-based approaches. First, a deterministic dynamic programming (DDP) framework is presented to solve HEV EMS problems subject to a specific driving cycle. To address this limitation, an improved DDP approach, integrating the actual travelled position of the vehicle into the control law, is proposed. This way, a given DDP-based EMS can be applied to all the driving cycles, yet still measured on the same road. Stochastic dynamic programming (SDP)-based EMSs are also developed and prove to be more adaptive to driving scenarios completely different from the ones used for their computation. Real-world driving cycles are employed in all the presented cases, while a reduced HEV powertrain model is used to alleviate the typical DP computational burden. [ABSTRACT FROM AUTHOR]

In this paper, an online event-triggered self-learning scheme based on adaptive dynamic programming (ADP) is developed to address tracking control design for nonlinear systems with constrained input and uncertain disturbance. Firstly, the value function with non-quadratic function is defined for the augmented nominal system, and the constrained robust tracking problem is equivalent to the optimal control for solving the tracking event-triggered Hamilton–Jacobi–Bellman (ETHJB) equation. Then, a single-critic network is developed to obtain the value function and control law related to the solution of the tracking ETHJB equation, greatly reducing approximation errors and computational costs. To alleviate the requirement for the entire state sampling, we propose a triggering rule that ensures system stability while limiting control updates. Theoretical proof demonstrates that the tracking state of the closed-loop system and the weight approximation error of the neural network are uniformly ultimately bounded (UUB). Finally, two examples are provided to validate the availability of the proposed scheme. [ABSTRACT FROM AUTHOR]

Tensegrity structures, which consist of tension cables and compression rods, are widely used in various fields. It is particularly important to find the tensegrity with definite geometric configuration for the following research and application. This paper presents a traversal topology-finding method for tensegrity structures based on dynamic programming algorithm and ground structure method. After the designer has given the cable topology, the dynamic programming is adopted to realize the traversal selection of the rod. In order to reduce the traversal space and realize the constraint of rod length type, the rod is classified according to length. Compared with existing topology-finding methods, the algorithm can not only output all feasible topologies but also incorporate structural constraints, such as rod length type and prestressed stability, into the calculation process. Four tensegrity numerical examples illustrate the feasibility and effectiveness of the topology-finding method. [ABSTRACT FROM AUTHOR]

—Using hydrogen production as energy storage can realize large-scale storage and transportation of energy, which is conducive to flexible scheduling of offshore wind power resources. However, compared with the battery energy storage system, the energy management strategy (EMS) of the dual-storage offshore wind power system with hydrogen production is more complex and nonlinear due to the large number of state variables and control variables. In order to realize the EMS of the system to maximize the benefits and minimize the aging cost, a nonlinear convex model is first established, and then an iterative solution method combining convex programming (CP) and dynamic programming (DP) is proposed. Among them, CP solves the global optimization problem of power distribution, and DP determines the start and stop of the electrolyzer. Finally, the feasibility of the proposed method is verified by simulation analysis. [Display omitted] • Conversion among different working states of the electrolyzer is considered, including cold start and warm start. • Degradation models of the electrolyzer and battery storage are considered for their state of health preservation. • An iterative algorithm combing dynamic programming and convex programming is proposed to search the optimal power splits and the electrolyzer state. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) has emerged as a dynamic and transformative paradigm in artificial intelligence, offering the promise of intelligent decision-making in complex and dynamic environments. This unique feature enables RL to address sequential decision-making problems with simultaneous sampling, evaluation, and feedback. As a result, RL techniques have become suitable candidates for developing powerful solutions in various domains. In this study, we present a comprehensive and systematic review of RL algorithms and applications. This review commences with an exploration of the foundations of RL and proceeds to examine each algorithm in detail, concluding with a comparative analysis of RL algorithms based on several criteria. This review then extends to two key applications of RL: robotics and healthcare. In robotics manipulation, RL enhances precision and adaptability in tasks such as object grasping and autonomous learning. In healthcare, this review turns its focus to the realm of cell growth problems, clarifying how RL has provided a data-driven approach for optimizing the growth of cell cultures and the development of therapeutic solutions. This review offers a comprehensive overview, shedding light on the evolving landscape of RL and its potential in two diverse yet interconnected fields. [ABSTRACT FROM AUTHOR]

In this study, we applied the iterative procedure (IP) method to search for families of highly diverged dispersed repeats in the genome of Cyanidioschyzon merolae, which contains over 16 million bases. The algorithm included the construction of position weight matrices (PWMs) for repeat families and the identification of more dispersed repeats based on the PWMs using dynamic programming. The results showed that the C. merolae genome contained 20 repeat families comprising a total of 33,938 dispersed repeats, which is significantly more than has been previously found using other methods. The repeats varied in length from 108 to 600 bp (522.54 bp in average) and occupied more than 72% of the C. merolae genome, whereas previously identified repeats, including tandem repeats, have been shown to constitute only about 28%. The high genomic content of dispersed repeats and their location in the coding regions suggest a significant role in the regulation of the functional activity of the genome. [ABSTRACT FROM AUTHOR]

Considering overshoot and chatter caused by the unknown interference, this article studies the adaptive robust optimal controls of continuous‐time (CT) multi‐input systems with an approximate dynamic programming (ADP) based Q‐function scheme. An adaptive integral reinforcement learning (IRL) scheme is proposed to study the optimal solutions of Q‐functions. First, multi‐input value functions are presented, and Nash equilibrium is analyzed. A complex Hamilton–Jacobi–Issacs (HJI) equation is constructed with the multi‐input system and the zero‐sum‐game‐based value function. It is a challenging task to solve the HJI equation for nonlinear system. Thus, A transformation of the HJI equation is constructed as a Q‐function. The neural network (NN) is applied to learn the solution of the transformed Q‐functions based on the adaptive IRL scheme. Moreover, an error information is added to the Q‐function for the issue of insufficient initial incentives to relax the persistent excitation (PE) condition. Simultaneously, an IRL signal of the critic networks is introduced to study the saddle‐point intractable solution, such that the system drift and NN derivatives in the HJI equation are relaxed. The convergence of weight parameters is proved, and the closed‐loop stability of the multi‐system with the proposed IRL Q‐function scheme is analyzed. Finally, a two‐engine driven F‐16 aircraft plant and a nonlinear system are presented to verify the effectiveness of the proposed adaptive IRL Q‐function scheme. [ABSTRACT FROM AUTHOR]

In this article, an adaptive critic scheme with a novel performance index function is developed to solve the tracking control problem, which eliminates the tracking error and possesses the adjustable convergence rate in the offline learning process. Under some conditions, the convergence and monotonicity of the accelerated value function sequence can be guaranteed. Combining the advantages of the adjustable and general value iteration schemes, an integrated algorithm is proposed with a fast guaranteed convergence, which involves two stages, namely the acceleration stage and the convergence stage. Moreover, an effective approach is given to adaptively determine the acceleration interval. With this operation, the fast convergence of the new value iteration scheme can be fully utilized. Finally, compared with the general value iteration, the numerical results are presented to verify the fast convergence and the tracking performance of the developed adaptive critic design. [ABSTRACT FROM AUTHOR]

This article pursues the robust consensus for linear discrete‐time multi‐agent systems with unknown dynamics and random uncertain channels. Stochastic multiplicative uncertainties depending on relative states of agents inevitably appear in the signal transmission channels. To handle these uncertainties, the distributed robust controllers are designed. Before designing this robust controller, the optimal control problem of a single‐agent system is considered first. Based on the technology of adaptive dynamic programming, a model‐free off‐policy algorithm is proposed to solve the optimal controller for every agent with fully unknown system dynamics. According to the theory of mean square stability, it is revealed that the distributed robust controller can be obtained by using the optimal controller of the single‐agent system and other related parameters without relying on the dynamic information of the systems. Besides, the relationship between uncertainty and the robust controller is described by a sufficient condition. Finally, to demonstrate the effectiveness of the theoretical analysis, a numerical example is presented. [ABSTRACT FROM AUTHOR]

This article investigates the problem of optimal observer‐based sliding mode control (SMC) of connected vehicles subject to deception attacks and disturbances with adaptive dynamic programming (ADP) method. For a group of vehicles with unknown nonlinear dynamics term and disturbance, this article aims to give a control methodology to achieve secure tracking of the desired spacing, velocity and acceleration. A neural network (NN) and an observer are constructed to estimate the unknown nonlinear term and the states, respectively. Then, a SMC scheme incorporating NN approximation is developed and an off‐policy ADP method is used to implement the optimal control of sliding mode dynamics. The proposed method can ensure individual stability and string stability of the set of vehicles. Numerical simulations are conducted to demonstrate the validity of the proposed controller. [ABSTRACT FROM AUTHOR]

This paper studies the optimal reinsurance and investment problem with multiple risky assets and correlation risk. The claim process is described by a Brownian motion with drift. The insurer is allowed to invest in a risk-free asset and multiple risky assets and the instantaneous return rate of each risky asset follows the Ornstein-Uhlenbeck (O-U) model. Moreover, the correlation between risk model and the risky assets' price is taken into account. We first consider the optimal investment problem for the insurer. Subsequently, we assume that the insurer can purchase proportional reinsurance and invest in the financial market. In both cases, the insurer's objective is to maximize the expected exponential utility of the terminal wealth. By applying stochastic control approach, we derive the optimal reinsurance and investment strategies and the corresponding value functions explicitly. Finally, numerical simulations are presented to illustrate the effects of model parameters on the optimal reinsurance and investment strategies. [ABSTRACT FROM AUTHOR]

We propose a new version of the tug-of-war game and a corresponding dynamic programming principle related to the p-Laplacian with 1 < p < 2. For this version, the asymptotic Hölder continuity of solutions can be directly derived from recent Krylov–Safonov type regularity results in the singular case. Moreover, existence of a measurable solution can be obtained without using boundary corrections. We also establish a comparison principle. [ABSTRACT FROM AUTHOR]

A groundbreaking study employs interval arithmetic to address the challenging multi-objective interval traveling salesperson problem. Customizing methods like a nearest neighbor, branch and bound, two-way heuristics, and dynamic programming effectively resolve this complex problem. Preserving interval values without the need for classical form conversion is a significant advantage. Researchers validated this approach through extensive experiments, consistently demonstrating superior outcomes compared to existing methods. These algorithmic approaches were optimized for Python 3.11 64-bit to enhance processing speed and efficiency. [ABSTRACT FROM AUTHOR]

Vegetables have a short period of freshness, and therefore, the purchase of vegetables has to be carefully matched with sales, especially in the "small production and big market" setting prevalent in China. Therefore, it is worthwhile to develop a systematic and comprehensive mathematical model of replenishment plans and pricing strategies for each category of vegetables and individual products. In this paper, we analyze the following three questions: Question One: What is the distribution law and relationship between the sales volume of vegetable categories and single products? Question Two: What is the relationship between total sales volume and cost-plus pricing of vegetable categories? And is it possible to provide the daily total replenishment and pricing strategy of each vegetable category for the following week to maximize supermarket profit? Question Three: How can we incorporate the market demand for single vegetable products into a profit-maximizing program for supermarkets? Is it possible to further formulate the replenishment plan requirements for single products? To answer the first question, we created pivot tables to analyze occupancy. We found that mosaic leaves, peppers, and edible mushrooms accounted for a larger proportion of occupacy, while cauliflowers, aquatic rhizomes, and eggplants accounted for a smaller proportion. For the single items, lettuce, cabbage, green pepper, screw pepper, enoki mushroom, and shiitake mushroom accounted for a large proportion of their respective categories. We used the Pearson correlation coefficient and the Mfuzz package based on fuzzy c-means (FCM) algorithm to analyze the correlation between vegetable categories and single products. We found that there was a strong correlation between vegetable categories. Moreover, the sale of vegetable items belonging to the same category exhibited the same patterns of change over time. In order to address the second question, we established the LightGBM sales forecasting model. Combined with previous sales data, we forecasted and planned an efficient daily replenishment volume for each vegetable category in the coming week. In addition, we developed a pricing strategy for vegetable categories to maximize supermarket profits. For the third question, we built a dynamic programming model combining an optimal replenishment volume with a product pricing strategy for single items, which let the supermarket maximize its expected profits. [ABSTRACT FROM AUTHOR]

To reduce the learning time and space occupation, this study presents a novel model‐free algorithm for obtaining the Nash equilibrium solution of continuous‐time nonlinear non‐zero‐sum games. Based on the integral reinforcement learning method, a new integral HJ equation that can quickly and cooperatively determine the Nash equilibrium strategies of all players is proposed. By leveraging the neural network approximation and gradient descent method, simultaneous continuous‐time adaptive tuning laws are provided for both critic and actor neural network weights. These laws facilitate the estimation of the optimal value function and optimal policy without requiring knowledge or identification of the system's dynamics. The closed‐loop system stability and convergence of weights are guaranteed through the Lyapunov analysis. Additionally, the algorithm is enhanced to reduce the number of auxiliary NNs used in the critic. The simulation results for a two‐player non‐zero‐sum game validate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The energy efficiency of intelligent networked connected electric vehicle (EV) is directly related to its velocity. Aiming at the influence of real-time traffic flow information on road speed interval, a two-layer speed planning method is proposed. The upper layer extracts the road speed interval according to the traffic flow information, and based on cellular automata and confidence interval theory, traffic information rules are introduced, and a road speed interval extraction method considering traffic density information is established. The lower layer is used to obtain energy-optimal cruising velocity profile. Taking the road speed interval as the variable boundary constraint, a dynamic programming algorithm that changes the state quantity boundary in real time is designed, which realizes the efficient acquisition of the energy-optimized velocity trajectory. To verify the effectiveness of proposed approach, the simulation model is formulated based on using collected real traffic information. The simulation results demonstrate that, compared with the conventional constant speed cruising strategy and dynamic programming (DP) strategy based on road speed interval, the strategy proposed in this study not only improves energy efficiency and reduces computing time significantly, but also can predict the traffic conditions ahead to avoid large fluctuations in velocity. Besides, the biggest significance of this study is the designed economic velocity planning algorithm based on real-time traffic density information improves the adaptability of intelligent networked connected EV control strategy to actual traffic conditions, and extends the optimization dimension of eco-driving. [ABSTRACT FROM AUTHOR]

Collective animal behaviour is a subfield of behavioural ecology, making extensive use of its tools of observation, experimental manipulation and model building. However, a fundamental behavioural ecology approach, the application of optimality theory, has been comparatively neglected in collective animal behaviour. This article seeks to address this imbalance, by outlining an evolutionary theory framework for the discipline. The application of optimality theory to collective animal behaviour requires a number of questions to be addressed. First, what is the correct quantity to optimize? This can be achieved via a combination of considering the organisms' life history, alongside tools such as statistical decision theory and stochastic dynamic programming. Second, what mechanism is appropriate for optimal behaviour? This involves ensuring that models are self-consistent rather than assuming parameter values. Third, at what level of selection does optimization act? Selection acts on the individual except in very particular circumstances, yet collective animal behaviour phenomena are group level, thus introducing a risk of confusing at what level adaptive properties emerge. This article presents examples under each of the three questions, as well as discussing mismatches between theory and observation. In doing so, it is hoped that collective animal behaviour fully inherits the tools and philosophy of its parent discipline of behavioural ecology. • We undertake a review of collective animal behaviour using Tinbergen's four questions. • We thus identify a lack of optimality theory in collective animal behaviour. • We propose three new questions for optimality theory in collective animal behaviour. • Using these questions we review the literature for successes and missed opportunities. [ABSTRACT FROM AUTHOR]

This paper develops a novel guaranteed cost control (GCC) approach under the event-triggered mechanism for completely unknown systems using integral reinforcement learning (IRL) algorithm. First, based on the adaptive dynamic programming (ADP) method, the GCC problem is addressed by transforming it into the optimal control problem. Second, without using the accurate information of system dynamics, a model-free data-based GCC approach is designed via IRL algorithm. Moreover, for the purpose of reducing the waste of communication resources, a GCC algorithm is presented under the event-triggered mechanism for completely unknown system by utilizing the explorized IRL algorithm. The critic–actor–disturbance neural networks (NNs) are applied to approximate near optimal solution. In addition, the weight estimations of NNs are tuned synchronously according to the designed novel triggering condition. Furthermore, the stability analysis of the controlled system is given by utilizing the Lyapunov principle. Finally, the simulation results are presented to verify the feasibility of the designed approach. [ABSTRACT FROM AUTHOR]

In this paper we extend dynamic programming techniques to the study of discrete-time infinite horizon optimal control problems on compact control invariant sets with state-independent best asymptotic average cost. To this end we analyse the interplay of dissipativity and optimal control, and propose novel recursive approaches for the solution of so called shifted Bellman Equations. [ABSTRACT FROM AUTHOR]

It is important to make improvements for the optimum operation of water pumping stations in order to reduce energy costs while meeting water demand continuously. In this article, a new algorithm (OSDPA-optimal simplified dynamic programming algorithm) has been developed by optimizing the previously developed SDPA (simplified dynamic programming algorithm) approach. The SDPA approach is most effective with straightforward systems, whereas the OSDPA approach is versatile, suitable for optimizing both simple and multi-stage, complex pumping systems. A water supply system with a multi-stage structure for Sakarya Water and Sewerage Administration (SASKİ) in Turkey has been used as a case study. The results show that the OSDPA approach can be quite successful in shifting the loads in the time period (peak) where the unit price is expensive to the time periods (night and day) where the unit price is cheaper. When compared with the conventional water level control (CWLC) approach, while SDPA saves about 18% for pump C in the second stage, about 22% for pump B, and about 19% for pump A in the first stage; OSDPA saved 39% for C pump, 41% for B pump and 24% for A pump. [ABSTRACT FROM AUTHOR]

In this article, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the stochastic spread of infectious diseases, such as COVID-19. Although the Markov Decision Process (MDP) offers a mathematical framework for identifying optimal actions, such as vaccination and transmission-reducing intervention, to combat disease spread calculated using the SEIR model. However, uncertainties in these scenarios demand a more robust approach that is less reliant on error-prone assumptions. The primary objective of our study is to introduce a new DRMDP framework that allows for an ambiguous distribution of transition dynamics. Specifically, we consider the worst-case distribution of these transition probabilities within a decision-dependent ambiguity set. To overcome the computational complexities associated with policy determination, we propose an efficient Real-Time Dynamic Programming (RTDP) algorithm that is capable of computing optimal policies based on the reformulated DRMDP model in an accurate, timely, and scalable manner. Comparative analysis against the classic MDP model demonstrates that the DRMDP achieves a lower proportion of infections and susceptibilities at a reduced cost. [ABSTRACT FROM AUTHOR]

The problem of computing the longest common subsequence of two sequences (LCS for short) is a classical and fundamental problem in computer science. In this article, we study four variants of LCS : the Repetition-Bounded Longest Common Subsequence problem (RBLCS), the Multiset-Restricted Common Subsequence problem (MRCS), the Two-Side-Filled Longest Common Subsequence problem (2FLCS), and the One-Side-Filled Longest Common Subsequence problem (1FLCS). Although the original LCS can be solved in polynomial time, all these four variants are known to be NP-hard. Recently, an exact, O (1. 4422 5 n) -time, dynamic programming (DP) based algorithm for RBLCS was proposed, where the two input sequences have lengths n and p o l y (n). Here, we first establish that each of MRCS , 1FLCS , and 2FLCS is polynomially equivalent to RBLCS. Then, we design a refined DP-based algorithm for RBLCS that runs in O (1. 4142 2 n) time, which implies that MRCS , 1FLCS , and 2FLCS can also be solved in O (1. 4142 2 n) time. Finally, we give a polynomial-time 2-approximation algorithm for 2FLCS. [ABSTRACT FROM AUTHOR]

Dynamic programming equations for mean field control problems with a separable structure are Eikonal type equations on the Wasserstein space. Standard differentiation using linear derivatives yield a direct extension of the classical viscosity theory. We use Fourier representation of the Sobolev norms on the space of measures, together with the standard techniques from the finite dimensional theory to prove a comparison result among semi-continuous sub and super solutions, obtaining a unique characterization of the value function. [ABSTRACT FROM AUTHOR]

Changes in demand, various hydrological inputs, and environmental stressors are among the issues that reservoir managers and policymakers face on a regular basis. These concerns have sparked interest in applying different techniques to determine reservoir operation policy decisions. As the resolution of the analysis increases, it becomes more difficult to effectively represent a real-world system using traditional methods such as dynamic programming and stochastic dynamic programming for determining the best reservoir operation policy. One of the challenges is the "curse of dimensionality," which means the number of samples needed to estimate an arbitrary function with a given level of accuracy grows exponentially with respect to the number of input variables (i.e., dimensionality) of the function. Deep reinforcement learning (DRL) is an intelligent approach to overcome the curses of stochastic optimization problems for reservoir operation policy decisions. To our knowledge, this study is the first attempt that examines various novel DRL continuous-action policy gradient methods, including deep deterministic policy gradients, twin delayed DDPG (TD3), and two different versions of Soft Actor-Critic (SAC18 and SAC19) for optimizing reservoir operation policy. In this study, multiple DRL techniques were implemented to find an optimal operation policy for Folsom Reservoir in California. The reservoir system supplies agricultural, municipal, hydropower, and environmental flow demands and flood control operations to the City of Sacramento. Analysis suggests that the TD3 and SAC are robust to meet the Folsom Reservoir's demands and optimize reservoir operation policies. Experiments on continuous-action spaces of reservoir policy decisions demonstrated that the DRL techniques can efficiently learn strategic policies in spaces and can overcome the curse of dimensionality and modeling. [ABSTRACT FROM AUTHOR]

The weighted Tower of Hanoi problem is a new generalization of the classical Tower of Hanoi problem, where a move of a disc between two pegs i and j is weighted by a positive real w i j . This new problem generalizes the concept of finding the minimum number of moves to solve the Tower of Hanoi, to find a sequence of moves with the minimum total weight. We present an optimal dynamic algorithm to solve the weighted Tower of Hanoi problem, as well as some of its properties. We also show the link of this generalization to the Tower of Hanoi variants with restricted disc moves, and how this new problem can generalize the concept of restricting a disc move. [ABSTRACT FROM AUTHOR]

In this research, we consider a two-machine job-shop scheduling problem with one joint job where a joint job is defined as a job whose operations are to be processed by different machines. The machine repetition and transportation times between machines are also considered. This problem is associated with real-world applications in the fields of production planning and supply chains. We first demonstrate that this problem is NP-hard in the strong sense. We then propose polynomial-time algorithms based on dynamic programming for cases with a fixed number of jobs. We also propose lemmas to reduce the number of states in dynamic programming without loss of optimality, so that the time complexity is improved. Other methodologies, including preprocessing and the two-pointers method, are also embedded to improve the algorithms. The proposed algorithms have better time complexities than those of variants of a well-known algorithm when applied to the cases of our problem. [ABSTRACT FROM AUTHOR]

The article investigates a networked optimal control problem with one remote controller and N$$ N $$ local controllers, where each local controller partially observes the local state and sends the observed information to remote controller through an unreliable channel and yet the channels from remote controller to local controllers are perfect. The objective of all the controllers is to cooperate to minimize a common cost function. In order to overcome the difficulties caused by partial observation of state, sufficient statistic of historical data is firstly investigated for a nonlinear non‐Gaussian partial observation model, which takes the same role as that of state in the full observation model. Then, a dynamic programming framework is given for solving the optimal control problem based on the sufficient statistic and common information approach. Furthermore, the explicit closed‐form solution is obtained for a linear‐quadratic‐Gaussian optimal control problem. Finally, a numerical example is given to show the effectiveness of the obtained results of this article. [ABSTRACT FROM AUTHOR]

This article studies the two‐layer leader‐follower optimal affine formation maneuver (AFM) control problem for multiple unmanned surface vessels (USV). The basic idea of the developed two‐layer control framework is to realize the time‐varying formation for the leader group in advance, and then let the follower USVs agilely respond to the target formation affinely localized by leaders. To realize this goal, a time‐varying formation reference system is first synthesized to describe leaders' target formation in the affine map of the nominal configuration. The reference system is set up by virtual vessel velocity command and unrotated geometric command, which make it possesses an intuitive application structure for operators to plan AFM commands in real‐time sensed surrounding environments. Then, by means of optimized backstepping design and the adaptive dynamic programming technique, actor‐only reinforcement learning controllers are established for USVs in both two layers to realize the optimal target formation tracking objective. For the proposed optimal controllers, the persistent excitation condition that is required by common optimal control schemes is also removed, which endows the presented scheme with lower computational complexity. Finally, theoretical analysis and simulation results illustrate the closed‐loop stability and the effectiveness of the presented scheme. [ABSTRACT FROM AUTHOR]

Motivation: Many bioinformatics problems can be approached as optimization or controlled sampling tasks, and solved exactly and efficiently using Dynamic Programming (DP). However, such exact methods are typically tailored towards specific settings, complex to develop, and hard to implement and adapt to problem variations. Methods: We introduce the Infrared framework to overcome such hindrances for a large class of problems. Its underlying paradigm is tailored toward problems that can be declaratively formalized as sparse feature networks, a generalization of constraint networks. Classic Boolean constraints specify a search space, consisting of putative solutions whose evaluation is performed through a combination of features. Problems are then solved using generic cluster tree elimination algorithms over a tree decomposition of the feature network. Their overall complexities are linear on the number of variables, and only exponential in the treewidth of the feature network. For sparse feature networks, associated with low to moderate treewidths, these algorithms allow to find optimal solutions, or generate controlled samples, with practical empirical efficiency. Results: Implementing these methods, the Infrared software allows Python programmers to rapidly develop exact optimization and sampling applications based on a tree decomposition-based efficient processing. Instead of directly coding specialized algorithms, problems are declaratively modeled as sets of variables over finite domains, whose dependencies are captured by constraints and functions. Such models are then automatically solved by generic DP algorithms. To illustrate the applicability of Infrared in bioinformatics and guide new users, we model and discuss variants of bioinformatics applications. We provide reimplementations and extensions of methods for RNA design, RNA sequence-structure alignment, parsimony-driven inference of ancestral traits in phylogenetic trees/networks, and design of coding sequences. Moreover, we demonstrate multidimensional Boltzmann sampling. These applications of the framework—together with our novel results—underline the practical relevance of Infrared. Remarkably, the achieved complexities are typically equivalent to the ones of specialized algorithms and implementations. Availability: Infrared is available at https://amibio.gitlabpages.inria.fr/Infrared with extensive documentation, including various usage examples and API reference; it can be installed using Conda or from source. [ABSTRACT FROM AUTHOR]

In response to the challenges posed by the low energy utilization of single-power pure electric vehicles and the limited lifespan of power batteries, this study focuses on the development of a compound power system. This study constructs a composite power system, analyzes the coupling characteristics of multiple systems, and investigates the energy management and optimal control mechanisms. Firstly, a power transmission scheme is designed for a hybrid electric vehicle. Then, a multi-state model is established to assess the electric vehicle's performance under complex working conditions and explore how these conditions impact system coupling. Next, load power is redistributed using the Haar wavelet theory. The super capacitor is employed to stabilize chaotic and transient components in the required power, with low-frequency components serving as input variables for the controller. Further, power distribution is determined through the application of fuzzy logic theory. Input parameters include the system's power requirements, power battery status, and super capacitor state of charge. The result is the output of a composite power supply distribution factor. To fully exploit the composite power supply's potential and optimize the overall system performance, a global optimization control strategy using the dynamic programming algorithm is explored. The optimization objective is to minimize power loss within the composite power system, and the optimal control is calculated through interpolation using the interp function. Finally, a comparative simulation experiment is conducted under UDDS cycle conditions. The results show that the composite power system improved the battery discharge efficiency and reduced the number of discharge cycles and discharge current of the power battery. Under the cyclic working condition of 1369 s, the state of charge of the power battery in the hybrid power system decreases from 0.9 to 0.69, representing a 12.5% increase compared to the single power system. The peak current of the power battery in the hybrid power system decreases by approximately 20 A compared with that in the single power system. Based on dynamic programming optimization, the state of charge of the power battery decreases from 0.9 to 0.724. Compared with that of the single power system, the power consumption of the proposed system increases by 25%, that of the hybrid power fuzzy control system increases by 14.2%, and that of the vehicle decreases by 14.7% after dynamic programming optimization. The multimode energy shunt relationship is solved through efficient and reasonable energy management and optimization strategies. The performance and advantages of the composite energy storage system are fully utilized. This approach provides a new idea for the energy storage scheme of new energy vehicles. [ABSTRACT FROM AUTHOR]

Energy management strategies (EMSs) are one of the key technologies for the development of plug-in hybrid electric buses (PHEBs). This paper addresses the issue of optimal energy distribution for PHEBs under significant variations in passenger load at different bus stations, which cannot be solved by a single equivalent factor equivalent fuel consumption minimization energy management strategy (ECMS). An adaptive equivalent factor equivalent fuel consumption minimization energy management strategy (A-ECMS) considering passenger load is proposed. First, the powertrain system of the PHEB is modeled, and the accuracy of the model is verified in a simulation environment. Then, the reference SOC descent trajectory of the battery is obtained using a dynamic programming (DP) algorithm, the recursive least squares (RLS) method is employed to identify the passenger load, and the influence of different loads on the state of charge (SOC) trajectory under a single equivalent factor is analyzed. Finally, a genetic algorithm (GA) is used to establish the correspondence between passenger load, bus station, and equivalent factor, enabling the actual SOC to follow the reference SOC descent trajectory, thereby achieving optimal energy distribution. Simulation results demonstrate that the A-ECMS reduces fuel consumption of the PHEB per 100 km by 2.59% and 10.10% compared to the ECMS and rule-based EMS, respectively, validating the effectiveness of the proposed strategy. [ABSTRACT FROM AUTHOR]

Fuel-cell hybrid trucks (FCHTs), as primary force in fuel-cell vehicles, currently lack comprehensive development of rule-based energy management strategies (EMSs). The current optimization rule of the dynamic programming (DP) algorithm exhibits poor adaptability to changes in operating conditions and battery state of charge (SOC). Therefore, the work proposed real-time rule EMS of DP-optimized FCHTs. First, DP was taken offline to determine the optimal relationship between fuel cell power and demand power under driving conditions and SOCs. Secondly, real-time online recognition was achieved through driving pattern recognition (DPR) controller. A backpropagation neural network, optimized by the northern goshawk algorithm, was used for constructing DPR. Finally, the MATLAB/Simulink simulation showed that the proposed strategy exhibited superior DPR accuracy. Compared to rule-based strategies, significantly lower hydrogen consumption reduced vehicle-operating costs. • A DP-optimized real-time energy management strategy was designed. • The multivariate DP offline of the optimal rule control sequence library was constructed. • Driving conditions and the initial SOC were defined as multivariates. • A driving pattern recognizer was constructed using the northern goshawk optimization and back-propagation neural network. [ABSTRACT FROM AUTHOR]

Pseudoknots are key structure motifs of RNA and pseudoknotted RNAs play important roles in a variety of biological processes. Here, we present KnotFold, an accurate approach to the prediction of RNA secondary structure including pseudoknots. The key elements of KnotFold include a learned potential function and a minimum-cost flow algorithm to find the secondary structure with the lowest potential. KnotFold learns the potential from the RNAs with known structures using an attention-based neural network, thus avoiding the inaccuracy of hand-crafted energy functions. The specially designed minimum-cost flow algorithm used by KnotFold considers all possible combinations of base pairs and selects from them the optimal combination. The algorithm breaks the restriction of nested base pairs required by the widely used dynamic programming algorithms, thus enabling the identification of pseudoknots. Using 1,009 pseudoknotted RNAs as representatives, we demonstrate the successful application of KnotFold in predicting RNA secondary structures including pseudoknots with accuracy higher than the state-of-the-art approaches. We anticipate that KnotFold, with its superior accuracy, will greatly facilitate the understanding of RNA structures and functionalities. Presenting KnotFold, an application for accurately predicting RNA secondary structures, including pseudoknots. [ABSTRACT FROM AUTHOR]

The hydrogen combustion engine (H 2 ICE) is known to be able to burn H 2 , producing no CO 2 emissions and extremely low engine-out NO x eo emissions. In this work, the potential to reduce the NO x eo emissions through the implementation of electric hybridization of an H 2 ICE-equipped passenger car (H 2 -HEV), combined with a dedicated energy management system (EMS) is discussed. Achieving a low H 2 consumption and low NO x eo emissions are conflicting objectives, the trade-off of which depends on the EMS and can be represented as a Pareto front. The dynamic programming algorithm is used to calculate the Pareto-optimal EMS calibrations for various driving missions. Through the utilization of a dedicated energy management calibration, H 2 -HEVs exhibit the potential to decrease the NO x eo emissions by more than 90%, while, decreasing the H 2 consumption by over 16%, compared to a comparable non-hybridized H 2 -vehicle. The present paper represents the initial potential analysis, suggesting that H 2 -HEVs are a viable option towards a CO 2 -free mobility with extremely low NO x eo emissions. • H 2 -HEVs with a dedicated energy management can achieve extremely low NO x eo • Mixed H 2 -HEVs have a superior H 2 -NO x eo trade-off than series and parallel H 2 -HEVs • Mixed H 2 -HEVs lower the NO x eo emissions by 90% and decrease the H 2 consumption by 16% • H 2 -HEV's optimal EMS calibrations for the entire H 2 - NO x eo Pareto front [ABSTRACT FROM AUTHOR]