Search

In recent years, a gradient of the $n$ -step temporal-difference [TD( $\lambda$ )] learning has been developed to present an advanced adaptive dynamic programming (ADP) algorithm, called value-gradient learning [VGL( $\lambda$ )]. In this paper, we improve the VGL( $\lambda$ ) architecture, which is called the “single adaptive actor network [SNVGL( $\lambda$ )]” because it has only a single approximator function network (critic) instead of dual networks (critic and actor) as in VGL( $\lambda$ ). Therefore, SNVGL( $\lambda$ ) has lower computational requirements when compared to VGL( $\lambda$ ). Moreover, in this paper, a recurrent hybrid neuro-fuzzy (RNF) and a first-order Takagi–Sugeno RNF (TSRNF) are derived and implemented to build the critic and actor networks. Furthermore, we develop the novel study of the theoretical convergence proofs for both VGL( $\lambda$ ) and SNVGL( $\lambda$ ) under certain conditions. In this paper, mobile robot simulation model (model based) is used to solve the optimal control problem for affine nonlinear discrete-time systems. Mobile robot is exposed various noise levels to verify the performance and to validate the theoretical analysis.

In problems with complex dynamics and challenging state spaces, the dual heuristic programming (DHP) algorithm has been shown theoretically and experimentally to perform well. This was recently extended by an approach called value gradient learning (VGL). VGL was inspired by a version of temporal difference (TD) learning that uses eligibility traces. The eligibility traces create an exponential decay of older observations with a decay parameter ( $\lambda $ ). This approach is known as TD( $\lambda $ ), and its DHP extension is known as VGL( $\lambda $ ), where VGL(0) is identical to DHP. VGL has presented convergence and other desirable properties, but it is primarily useful for batch learning. Online learning requires an eligibility-trace-work-space matrix, which is not required for the batch learning version of VGL. Since online learning is desirable for many applications, it is important to remove this computational and memory impediment. This paper introduces a dual-critic version of VGL, called $N$ -step VGL (NSVGL), that does not need the eligibility-trace-work-space matrix, thereby allowing online learning. Furthermore, this combination of critic networks allows an NSVGL algorithm to learn faster. The first critic is similar to DHP, which is adapted based on TD(0) learning, while the second critic is adapted based on a gradient of $n$ -step TD( $\lambda $ ) learning. Both networks are combined to train an actor network. The combination of feedback signals from both critic networks provides an optimal decision faster than traditional adaptive dynamic programming (ADP) via mixing current information and event history. Convergence proofs are provided. Gradients of one- and $n$ -step value functions are monotonically nondecreasing and converge to the optimum. Two simulation case studies are presented for NSVGL to show their superior performance.

Biclustering is a powerful tool for exploratory data analysis in domains such as social networking, data reduction, and differential gene expression studies. Topological learning identifies connected regions that are difficult to find using other traditional clustering methods and produces a graphical representation. Therefore, to improve the quality of biclustering and module extraction, this work combines the adaptive resonance theory (ART)-based methods of biclustering ARTMAP (BARTMAP) and topological ART (TopoART), to produce TopoBARTMAP. The latter inherits the ability to detect topological associations while performing data reduction. The capabilities of TopoBARTMAP were benchmarked using 35 real world cancer datasets and contrasted with other (bi)clustering methods, where it showed a statistically significant improvement over the other assessed methods on ordered and shuffled data experiments. In experiments with 12 synthetic datasets, the method was observed to perform better at identifying constant, scale, shift, and shift scale type biclusters. The produced graphical representation was refined to represent gene bicluster associations and was assessed on the NCBI GSE89116 dataset containing expression levels of 39,326 probes sampled over 38 observations.

When features in a high dimension dataset are organized hierarchically, there is an inherent opportunity to reduce dimensionality. Since more specific concepts are subsumed by more general concepts, subsumption can be applied successively to reduce dimensionality. We tested whether sub-sumption could reduce the dimensionality of a disease dataset without impairing classification accuracy. We started with a dataset that had 168 neurological patients, 14 diagnoses, and 293 unique features. We applied subsumption repeatedly to create eight successively smaller datasets, ranging from 293 dimensions in the largest dataset to 11 dimensions in the smallest dataset. We tested a MLP classifier on all eight datasets. Precision, recall, accuracy, and validation declined only at the lowest dimensionality. Our preliminary results suggest that when features in a high dimension dataset are derived from a hierarchical ontology, subsumption is a viable strategy to reduce dimensionality.Clinical relevance- Datasets derived from electronic health records are often of high dimensionality. If features in the dataset are based on concepts from a hierarchical ontology, subsumption can reduce dimensionality.

This article investigates the local stability and local convergence of a class of neural network (NN) controllers with error integrals as inputs for reference tracking. It is formally proved that if the input of the NN controller consists exclusively of error terms, the control system shows a non-zero steady-state error for any constant reference except for one specific point, for both single-layer and multi-layer NN controllers. It is further proved that adding error integrals to the input of the (single- and multi-layers) NN controller is one sufficient way to remove the steady-state error for any constant reference. Due to the nonlinearity of the NN controllers, the NN control systems are linearized at the equilibrium points. We provide proof that if all the eigenvalues of the linearized NN control system have negative real parts, local asymptotic stability and local exponential convergence are guaranteed. Two case studies were explored to verify the theoretical results: a single-layer NN controller in a 1-D system and a four-layer NN controller in a 2-D system applied to renewable energy integration. Simulations demonstrate that when NN controllers and the corresponding generalized proportional-integral (PI) controllers have the same eigenvalues, all control systems exhibit almost the same responses in a small neighborhood of their respective equilibrium points.

In this article, we consider an iterative adaptive dynamic programming (ADP) algorithm within the Hamiltonian-driven framework to solve the Hamilton-Jacobi-Bellman (HJB) equation for the infinite-horizon optimal control problem in continuous time for nonlinear systems. First, a novel function, ``min-Hamiltonian,'' is defined to capture the fundamental properties of the classical Hamiltonian. It is shown that both the HJB equation and the policy iteration (PI) algorithm can be formulated in terms of the min-Hamiltonian within the Hamiltonian-driven framework. Moreover, we develop an iterative ADP algorithm that takes into consideration the approximation errors during the policy evaluation step. We then derive a sufficient condition on the iterative value gradient to guarantee closed-loop stability of the equilibrium point as well as convergence to the optimal value. A model-free extension based on an off-policy reinforcement learning (RL) technique is also provided. Finally, numerical results illustrate the efficacy of the proposed framework.

We present a framework for an explainable and statistically validated ensemble clustering model applied to Traumatic Brain Injury (TBI). The objective of our analysis is to identify patient injury severity subgroups and key phenotypes that delineate these subgroups using varied clinical and computed tomography data. Explainable and statistically-validated models are essential because a data-driven identification of subgroups is an inherently multidisciplinary undertaking. In our case, this procedure yielded six distinct patient subgroups with respect to mechanism of injury, severity of presentation, anatomy, psychometric, and functional outcome. This framework for ensemble cluster analysis fully integrates statistical methods at several stages of analysis to enhance the quality and the explainability of results. This methodology is applicable to other clinical data sets that exhibit significant heterogeneity as well as other diverse data science applications in biomedicine and elsewhere.

With the rapid development of science and technology, in order to realize intelligent control for human–machine hybrid systems, research scholars have studied a large number of learning approaches. However, the problem that we need to further consider is ensuring the control stability while realizing the optimal performance of human–machine hybrid systems.

This paper presents a model-free solution to the robust stabilization problem of discrete-time linear dynamical systems with bounded and mismatched uncertainty. An optimal controller design method is derived to solve the robust control problem, which results in solving an algebraic Riccati equation (ARE). It is shown that the optimal controller obtained by solving the ARE can robustly stabilize the uncertain system. To develop a model-free solution to the translated ARE, off-policy reinforcement learning (RL) is employed to solve the problem in hand without the requirement of system dynamics. In addition, the comparisons between on- and off-policy RL methods are presented regarding the robustness to probing noise and the dependence on system dynamics. Finally, a simulation example is carried out to validate the efficacy of the presented off-policy RL approach.

A moral crisis has swept through the United States dividing social, political, and religious organizations with corrupt and ineffectual leadership. However, the present moral crisis has its roots in the technological and cultural shifts of the last half century. The goal of interfaith dialogue is not merely to exchange pleasantries, but to build a mutual collaboration addressing the moral and ethical issues with a unified voice. Interfaith dialogue has the potential to pull us out of our individualism and, in focusing on our relationships, create a new sensibility about being human. And were that new sensibility understood to reflect some of the fundamentals of science, it would strengthen the ability of religions to pursue a more healing inclusivity and reveal a rich unity among religious faiths, stretching down from our personal relationships with each other to the divine and the very fabric of reality.

Distributed energy storage is beneficial to energy consumers and system operators, but it is expensive to install and requires careful management. To improve the viability of demand-side energy storage, this study presents a new method of obtaining optimal control actions for a demand-side energy storage system in the presence of distributed generation. The optimal usage policy is determined by framing the selection of charge and discharge actions as a stochastic optimization problem, which is conveniently represented as a Markov decision process. The proposed method uses short-term forecasts of loads and local generation to represent the time-dependence and nonstationarity of the net load profile. In contrast to previous studies, the forecast information is used to restrict the problem state space, reducing the computational complexity of the policy calculation. The method is tested using historic load and generation data with real time-of-use rate schedules and achieves substantial reductions in energy costs over similar existing methods.

A continuous-time ${K}$ -winners-take-all (KWTA) neural model that can identify the largest ${K}$ of ${N}$ inputs, where command signal $1 \le {K} is described. The model is given by a differential equation where the spike train is a sum of delta functions. A functional block-diagram of the model includes ${N}$ feed-forward hard-limiting neurons and one feedback neuron, used to handle input dynamics. The existence and uniqueness of the model steady states are analyzed, the convergence analysis of the state variable trajectories to the KWTA operation is proven, the convergence time and number of spikes required are derived, as well as the processing of time-varying inputs and perturbations of the model nonlinearities are analyzed. The main advantage of the model is that it is not subject to the intrinsic convergence of speed limitations of comparable designs. The model also has an arbitrary finite resolution determined by a given parameter, low complexity, and initial condition independence. Applications of the model for parallel sorting and parallel rank-order filtering are presented. Theoretical results are derived and illustrated with computer-simulated examples that demonstrate the model’s performance.