Search

Time series forecasting models that use the past information of exogenous or endogenous sequences to forecast future series play an important role in the real world because most real-world time series datasets are rich in time-dependent information. Most conventional prediction models for time series datasets are time-consuming and fraught with complex limitations because they usually fail to adequately exploit the latent spatial dependence between pairs of variables. As a successful variant of recurrent neural networks, the long short-term memory network (LSTM) has been demonstrated to have stronger nonlinear dynamics to store sequential data than traditional machine learning models. Nevertheless, the common shallow LSTM architecture has limited capacity to fully extract the transient characteristics of long interval sequential datasets. In this study, a novel deep autoregression feature augmented bidirectional LSTM network (DAFA-BiLSTM) is proposed as a new deep BiLSTM architecture for time series prediction. Initially, the input vectors are fed into a vector autoregression (VA) transformation module to represent the time-delayed linear and nonlinear properties of the input signals in an unsupervised way. Then, the learned nonlinear combination vectors of VA are progressively fed into different layers of BiLSTM and the output of the previous BiLSTM module is also concatenated with the time-delayed linear vectors of the VA as an augmented feature to form new additional input signals for the next adjacent BiLSTM layer. Extensive real-world time series applications are addressed to demonstrate the superiority and robustness of the proposed DAFA-BiLSTM. Comparative experimental results and statistical analysis show that the proposed DAFA-BiLSTM has good adaptive performance as well as robustness even in noisy environment.

Highlights • In order to reestablish the random initialization input weights of ESN, a single layer ESN with a global reversible AE (GRAE) algorithm is proposed to ensure that the outputs of GRAE a...

In this paper, a new time-space network model is proposed for addressing the time-dependent rural postman problem (TDRPP) of a single vehicle. The proposed model follows the idea of arc-path alternation to form a feasible and complete route. Based on the proposed model, the time dependency of the TDRPP is better described to capture its dynamic process, compared to the existing methods using a piecewise constant function with limited intervals. Furthermore, the property of first-in-first-out (FIFO) can be satisfied with the time spent on each arc. We investigate the FIFO property for the considered time-dependent network and key optimality property for the TDRPP. Based on this property, a dedicated genetic algorithm (GA) is proposed to efficiently solve the considered TDRPP that suffers from computational intractability for large-scale cases. Comprehensive simulation experiments are conducted for various time-dependent networks to show the effectiveness of the proposed GA.

How to improve the behavioral decision-making ability and adaptability to unknown environments is of great importance for an agent. The traditional decision-making methods usually suffer from long training time, due to the large amount of training samples, or low adaptability to the unknown environments, or lack of the continuous learning capacities, etc. In response to these problems, this work proposes a novel motivated developmental network (MDN) to improve the decision-making ability of the agent. During the environment exploration, if the agent encounters an unknown environment, new layers and neurons are dynamically inserted to the MDN, according to the task requirements. Through the interaction between internal neurons and the inserted new neurons, the agent can autonomously develop and learn in the unknown environments without training data, but the behavioral decision-making at this stage is random. To further improve the agent’s decision-making ability, in the off-task process, through the gated self-organization mechanism, the agent can selectively recall the specific knowledge in its “brain”, and the MDN will transfer and generate a large amount of new data, according to the recalled knowledge, then the new layers and neurons will be inserted to memorize the new knowledge. Hence the knowledge base of the MDN becomes more and more complete, thereby improving its decision-making ability and adaptability to the new environment. To demonstrate the performance of the MDN model, a mobile robot navigation in different environments are executed. The experimental results illustrate that the agent can not only autonomously learn in static environments, but also has better decision-making ability in unknown dynamic environment, i.e., better adaptability to the new environment. Comparison with other algorithms further demonstrate the potential of the proposed MDN model.

In the intensive computing scenario with the application of micro-service framework, it is too costly to allocate resources centrally to the micro-service instance on the server side, and it cannot adapt to the changes of the environment in real time. Therefore, allocating resource for micro-service instance distributed on the terminal becomes an effective way to solve the problem mentioned above. A client-based micro-service discovery algorithm with attractor selection model is proposed in this paper, which has the characters of simple and intelligent evolution. At first, this paper models micro-service discovery problem, defines optimization object and constraints; then, it re-defines the parameters of attractor selection and proposes the client-based micro-service discovery algorithm with attractor selection model; and at last, it analyzes the performance of the proposed algorithm with the compared ones in the really productive environment. The experiment results show that the proposed algorithm reduces almost 32 % resource fragments and 70% running time than the compared ones.

Echo State Network (ESN) is a specific form of recurrent neural network, which displays very rich dynamics owing to its reservoir based hidden neurons. In the issue, ESN is viewed as a powerful approach to model real-valued time series processes. Nevertheless, ESN has been criticized for its manually experienced or brute-force searching parameters, such as initial input weights and reservoir layer weights, i.e., the conventional randomly generated ESN is unlikely to be optimal because the reservoir layer weights and input layer weights are created randomly. Simple Cycle Reservoir Network (SCRN), which constitutes a type of conclusively constructed input and internal layer weights, can yield performance comparable with conventional ESN. A Redundant Unit Pruning Auto-Encoder (RUP-AE) algorithm is proposed to optimize the input layer weights of SCRN and for resolving the dilemma of ill-conditioned output weights matrix in SCRN, through an unsupervised pre-training process. Initially, the output weights matrix of SCRN is pre-trained by pseudo-inverse algorithm through training data. Then, the pre-trained output weights matrix is pruned by a Redundant Unit Pruning (RUP) algorithm. Finally, the pruned output weights matrix of SCRN is injected to the input weights matrix to ensure the specificity of the auto-encoder. Three tasks, namely nonlinear time series system identification task, real-valued time series benchmark, and standard chaotic time series benchmark, are applied to demonstrate the advantage and superiority of RUP-AE. Extensive experimental results show that our RUP-AE is effective in improving the performance of SCRN. Meanwhile, RUP-AE is able to resolve the dilemma of ill-conditioned output weights matrix in SCRN.

Obtaining appropriate parameters of photovoltaic models based on measured current-voltage data is crucial for the evaluation, control, and optimization of photovoltaic systems. Although many techniques have been developed to solve this problem, it is still challenging to identify the model parameters accurately and reliably. To improve parameters identification of different photovoltaic models, a multiple learning backtracking search algorithm (MLBSA) is proposed in this paper. In MLBSA, some individuals learn from the current population information and historical population information simultaneously, which aims to maintain population diversity and enhance the exploration ability. While other individuals learn from the best individual of current population to improve the convergence speed and thus enhance the exploitation ability. In addition, an elite strategy based on chaotic local search is developed to further refine the quality of current population. The proposed MLBSA is employed to solve the parameters identification problems of different photovoltaic models, i.e., single diode, double diode, and photovoltaic module. Comprehensive experimental results and analyses demonstrate that MLBSA outperforms other state-of-the-art algorithms in terms of accuracy, reliability, and computational efficiency.

Parameters identification of photovoltaic (PV) models based on measured current-voltage characteristic curves is significant for the simulation, evaluation, and control of PV systems. To accurately and reliably identify the parameters of different PV models, an improved JAYA (IJAYA) optimization algorithm is proposed in the paper. In IJAYA, a self-adaptive weight is introduced to adjust the tendency of approaching the best solution and avoiding the worst solution at different search stages, which enables the algorithm to approach the promising area at the early stage and implement the local search at the later stage. Furthermore, an experience-based learning strategy is developed and employed randomly to maintain the population diversity and enhance the exploration ability. A chaotic elite learning method is proposed to refine the quality of the best solution in each generation. The proposed IJAYA is used to solve the parameters identification problems of different PV models, i.e., single diode, double diode, and PV module. Comprehensive experiment results and analyses indicate that IJAYA can obtain a highly competitive performance compared with other state-of-the-state algorithms, especially in terms of accuracy and reliability.

Echo state networks (ESNs) have become one of the most effective dynamic neural networks because of its excellent fitting performance in real-valued time series modeling tasks and simple training processes. The original ESN concept used randomly fixed created reservoirs, and this concept is considered to be one of its main advantages. However, ESNs have been criticized for its randomly created connectivity and weight parameters. Determining the appropriate weight parameters for a given task is an important problem. An optimization method based on mutual information (MI) is proposed in this study to optimize the input scaling parameters and the structure of ESN to address the aforementioned problem and improve the performance of ESN. The MI optimization method mainly consists of two parts: First, the scaling parameters of multiple inputs are adjusted based on the MI between the network inputs and outputs. Second, some output weight connections are pruned for optimization based on the MI between reservoir states. Furthermore, three MI-ESN models are proposed for a fed-batch penicillin fermentation process. Our experimental outcomes reveal that the obtained MI-ESN models outperform the ESN models without optimization and other traditional neural networks.

Phytochemical studies led to the isolation of a new phenylpropanoid glucoside, named purpuroside (

Echo State Network (ESN) is a specific class of recurrent neural networks, which displays very rich dynamics owing to its reservoir based hidden neurons. ESN has been viewed as a powerful approach to model real-valued time series processes. In order to integrate with deep learning theory, Deep Belief Echo State Network (DBESN) is employed to address the slow convergence in Deep Belief Network (DBN). In DBESN, the DBN part is employed for feature learning in an unsupervised fashion and the ESN part is utilized as a regression layer of DBN. However, the ESN input layer is still not working in an unsupervised status in DBESN. Moreover, ESN’s input dimension increases dramatically because of the DBN layer in DBESN. Namely, the DBN layer in DBESN makes the ESN more difficult to construct the input scaling parameters. For purpose of constructing an optimal input weights matrix and input scaling parameters in the ESN layer of DBESN, a novel Sensitivity Analysis Input Scaling Auto-Encoder (SAIS-AE) algorithm is employed in this paper through an unsupervised pre-training process. Initially, the output weights matrix of ESN layer is pre-trained by total input data set. Then, the pre-trained output weights matrix is injected into the input weights matrix of the ESN layer to ensure the specificity of AE. Finally, the input scaling parameters of ESN layer are tuned based on a sensitivity analysis algorithm. Two multivariable sequence tasks and one univariate sequence benchmark are applied to demonstrate the advantage and superiority of SAIS-AE. Extensive experimental results show that our SAIS-AE-DBESN model can effectively improve the performance of DBESN.

Algal blooms are one of the most prevalent global problems. Studying the Chlorophyll-a Chl-a predicting model helps to control algal blooms. Predicting the behavior of algae is difficult because of the complex physical, chemical, and biological processes involved. Artificial neural network ANN models have been determined to be useful and efficient, especially for such problems for which the characteristics of the processes are difficult to describe using numerical models. An indoor simulated environment is designed for algal cultivation to analyze the temporal change in the algae biomass of Taihu Lake during summer. A Chl-a prediction model based on a nonlinear autoregressive neural network with exogenous inputs NARX that can detect and consider within the time dependency is proposed. The NARX model is compared to a static neural network and a dynamic neural network: feedforward neural network FNN and Elman recurrent neural network ERNN. The performance of the proposed NARX model was examined with experimental data collected over 3 months in 2010. The results showed that the NARX model outperformed the other ANN models and significantly enhance the accuracy of Chl-a prediction.

The echo state network (ESN) is a novel and powerful method for the temporal processing of recurrent neural networks. It has tremendous potential for solving a variety of problems, especially real-valued, time-series modeling tasks. However, its complicated topologies and random reservoirs are difficult to implement in practice. For instance, the reservoir must be large enough to capture all data features given that the reservoir is generated randomly. To reduce network complexity and to improve generalization ability, we present a novel optimized ESN (O-ESN) based on binary particle swarm optimization (BPSO). Because the optimization of output weights connection structures is a feature selection problem and PSO has been used as a promising method for feature selection problems, BPSO is employed to determine the optimal connection structures for output weights in the O-ESN. First, we establish and train an ESN with sufficient internal units using training data. The connection structure of output weights, i.e., connection or disconnection, is then optimized through BPSO with validation data. Finally, the performance of the O-ESN is evaluated through test data. This performance is demonstrated in three different types of problems, namely, a system identification and two time-series benchmark tasks. Results show that the O-ESN outperforms the classical feature selection method, least angle regression (LAR) method in that its architecture is simpler than that of LAR.

Reservoir Computing (RC) is an effective approach to design and train recurrent neural networks, which is successfully and widely applied in real-valued time series modeling tasks. However, RC has been criticized for not being principled enough, namely the reservoir which is unlikely to be optimal because the reservoir connectivity and weight structure are created randomly. A new Simple Cycle Reservoir Network (SCRN) with deterministically constructed connectivity and weight structure can yield performance competitive with standard Echo State Network (ESN). In order to determine the proper size of the reservoir and improve generalization ability of SCRN, a Sensitive Iterated Pruning Algorithm (SIPA), in which a larger than necessary reservoir is employed firstly and then its size is reduced by pruning out the least sensitive internal units, is proposed to optimize the reservoir size and weights of SCRN. A system identification and two time-series benchmark tasks are applied to demonstrate the feasibility and superiority of SIPA. The results show that the SIPA method significantly outperforms a Least Angle Regression (LAR) method and SIPA is able to improve the generalization performance of SCRN. Besides, two well known reservoir characterizations, i.e. pseudo-Lyapunov exponent of the reservoir dynamics and Memory Capacity, and the impact of SIPA on two characterizations are investigated.

Gelsemium elegans (GE) is a kind of well-known toxic plant. It can be detoxified by Mussaenda pubescens (MP), but the detoxification mechanism is still unclear. Thus, a detoxification herbal formula (GM) comprising GE and MP was derived. The Caco-2 cells monolayer model was used to evaluate GM effects on transporting six kinds of indole alkaloids of GE. The bidirectional transport studies demonstrated that absorbance percentage of indole alkaloids in GE increased linearly over time. But in GM, Papp (AP→BL) values of the most toxic members, gelsenicine, humantenidine, and gelsevirine, were lower than that of Papp (BL→AP) (P < 0.05). The prominent analgesic effect members, gelsemine and koumine, were approximately 1.00 in γ values. Nowhere was this increasing efflux more pronounced than in the case of indole alkaloids with N-O structure. In the presence of verapamil, the γ values of humantenidine, gelsenicine, gelsevirine, and humantenine were decreased by 43.69, 41.42, 36.00, and 8.90 percent, respectively. The γ values in presence of ciclosporin were homologous with a decrease of 42.32, 40.59, 34.00, and 15.07 percent. It suggested that the efflux transport was affected by transporters. Taken together, due to the efflux transporters participation, the increasing efflux of indole alkaloids from GM was found in Caco-2 cells.

Gelsemium elegans (GE) is a kind of well-known toxic plant. It can be detoxified by Mussaenda pubescens (MP), but the detoxification mechanism is still unclear. Thus, a detoxification herbal formula (GM) comprising GE and MP was derived. The Caco-2 cells monolayer model was used to evaluate GM effects on transporting six kinds of indole alkaloids of GE. The bidirectional transport studies demonstrated that absorbance percentage of indole alkaloids in GE increased linearly over time. But in GM, Papp (AP→BL) values of the most toxic members, gelsenicine, humantenidine, and gelsevirine, were lower than that of Papp (BL→AP) (P<0.05). The prominent analgesic effect members, gelsemine and koumine, were approximately 1.00 in γ values. Nowhere was this increasing efflux more pronounced than in the case of indole alkaloids with N-O structure. In the presence of verapamil, the γ values of humantenidine, gelsenicine, gelsevirine, and humantenine were decreased by 43.69, 41.42, 36.00, and 8.90 percent, respectively. The γ values in presence of ciclosporin were homologous with a decrease of 42.32, 40.59, 34.00, and 15.07 percent. It suggested that the efflux transport was affected by transporters. Taken together, due to the efflux transporters participation, the increasing efflux of indole alkaloids from GM was found in Caco-2 cells.

Although reservoir computing (RC) is an effective approach to designing and training recurrent neural networks, the optimization of neural network systems involves a number of manual, tweaking or brute-force searching parameters, such as network size, input scaling parameters, and spectral radius. To create an optimal echo state network (ESN), we propose a modified RC that combines sensitivity analysis input scaling regulation (SAISR) and redundant unit pruning algorithm (RUPA). SAISR is first employed to obtain the optimal input scaling parameters. In SAISR, an ESN without tuning is established, and then its input scaling parameters are tuned based on the Sobol’ sensitivity analysis. Second, RUPA is employed to prune out the redundant readout connections. A fed-batch penicillin cultivation process is chosen to demonstrate the applicability of the modified RC. The results show that the input scaling parameter has a more important influence than other parameters in ESN, and SAISR-ESN outperforms ESN withou...

Echo state networks (ESNs) have become one of the most effective recurrent neural networks because of its good prediction performance in real-valued time-series modeling tasks and simple training processes. The original ESN concept uses fixed randomly created reservoirs, and this concept is considered of its main advantages. However, ESN has been criticized for its randomly created connectivity and weight structure. Finding an optimal reservoir for a given task is an important problem. To address this problem, a cycle reservoir with regular jump network (CRJN) model was proposed. To improve the performance of CRJN and keep the output weights small, we present a simple disjunction algorithm (SDA). First, an appropriately sized reservoir is employed. The weight rates of each internal neuron are then calculated. Finally, internal neurons with weight rates exceeding a threshold are spread into two neurons. A system identification and two time-series benchmark tasks are applied to demonstrate the feasibility and superiority of SDA. Results show that the SDA method outperforms several other improved approaches. Furthermore, this method is able to keep the output weights small, thus increasing the stability of CRJN.

Based on the fractal theory, two different experimental methods were employed to determine and analyze the fractal dimension of the Ca-based sorbents. Fractal dimension of the pore boundary of the ca-based sorbents calcinated at various temperatures were measured firstly by using the scan electron microscope (SEM) and the improved box-counting program, the relationship between the pore boundary fractal dimension and temperatures was derived. Moreover, the factors that affect the fractal dimension, such as additive amount and temperature, were studied by using the mercury porosimetric method. It showed that more additives or higher temperature all decreased fractal dimension of the sorbents and both the two methods presented in this paper could effectively determine and analyze the dimension of the sorbents.