Search

To endow the prey with intelligent movement behavior and improve the performance of Golden Jackal Optimization (GJO), a Q-learning Improved Gold Jackal Optimization (QIGJO) algorithm is proposed. This paper introduces five update mechanisms and proposes double-population Q-learning collaborative mechanism to select appropriate update mechanisms to improve GJO performance. Additionally, a new convergence factor is incorporated to enhance convergence capability of GJO. QIGJO demonstrates excellent performance across 23 benchmark functions, CEC2022, and three classical engineering design problems, indicating high convergence accuracy and significantly enhanced global exploration capability. The reliability optimization model of the hydraulic system for concrete pump trucks was established based on a Continuous-time Multi-dimensional T-S dynamic Fault Tree (CM-TSdFT), considering the two-dimensional factors of operating time and number of impacts. Utilizing QIGJO to optimize this model yielded excellent results, providing valuable methodological support for reliability optimization of hydraulic systems. [ABSTRACT FROM AUTHOR]

For the relativistic navigation system where the position and velocity of the spacecraft are determined through the observation of the relativistic perturbations including stellar aberration and starlight gravitational deflection, a novel parallel Q-learning extended Kalman filter (PQEKF) is presented to implement the measurement bias calibration. The relativistic perturbations are extracted from the inter-star angle measurement achieved with a group of high-accuracy star sensors on the spacecraft. Inter-star angle measurement bias caused by the misalignment of the star sensors is one of the main error sources in the relativistic navigation system. In order to suppress the unfavorable effect of measurement bias on navigation performance, the PQEKF is developed to estimate the position and velocity, together with the calibration parameters, where the Q-learning approach is adopted to fine tune the process noise covariance matrix of the filter automatically. The high performance of the presented method is illustrated via numerical simulations in the scenario of medium Earth orbit (MEO) satellite navigation. The simulation results show that, for the considered MEO satellite and the presented PQEKF algorithm, in the case that the inter-star angle measurement accuracy is about 1 mas, after calibration, the positioning accuracy of the relativistic navigation system is less than 300 m. [ABSTRACT FROM AUTHOR]

Predicting rock tunnel squeezing in underground projects is challenging due to its intricate and unpredictable nature. This study proposes an innovative approach to enhance the accuracy and reliability of tunnel squeezing prediction. The proposed method combines ensemble learning techniques with Q-learning and online Markov chain integration. A deep learning model is trained on a comprehensive database comprising tunnel parameters including diameter (D), burial depth (H), support stiffness (K), and tunneling quality index (Q). Multiple deep learning models are trained concurrently, leveraging ensemble learning to capture diverse patterns and improve prediction performance. Integration of the Q-learning-Online Markov Chain further refines predictions. The online Markov chain analyzes historical sequences of tunnel parameters and squeezing class transitions, establishing transition probabilities between different squeezing classes. The Q-learning algorithm optimizes decision-making by learning the optimal policy for transitioning between tunnel states. The proposed model is evaluated using a dataset from various tunnel construction projects, assessing performance through metrics like accuracy, precision, recall, and F1-score. Results demonstrate the efficiency of the ensemble deep learning model combined with Q-learning-Online Markov Chain in predicting surrounding rock tunnel squeezing. This approach offers insights into parameter interrelationships and dynamic squeezing characteristics, enabling proactive planning and support measures implementation to mitigate tunnel squeezing hazards and ensure underground structure safety. Experimental results show the model achieves a prediction accuracy of 98.11%, surpassing individual CNN and RNN models, with an AUC value of 0.98. [ABSTRACT FROM AUTHOR]

With the large-scale development of solar power generation, highly uncertain photovoltaic (PV) power output has an increasing impact on distribution networks. PV power generation has complex correlations with various weather factors, while the time series embodies multiple temporal characteristics. To more accurately quantify the uncertainty of PV power generation, this paper proposes a short-term PV power probabilistic forecasting method based on the combination of decomposition prediction and multidimensional variable dependency modeling. First, a seasonal and trend decomposition using a Loess (STL)-based PV time series feature decomposition model is constructed to obtain periodic, trend, and residual components representing different characteristics. For different components, this paper develops a periodic component prediction model based on TimeMixer for multi-scale temporal feature mixing, a long short-term memory (LSTM)-based trend component extraction and prediction model, and a multidimensional PV residual probability density prediction model optimized by Vine Copula optimized with Q-Learning. These components' results form a short-term PV probabilistic forecasting method that considers both temporal features and multidimensional variable correlations. Experimentation with data from the Desert Knowledge Australia Solar Center (DKASC) demonstrates that the proposed method reduced root mean square error (RMSE) and mean absolute percentage error (MAPE) by at least 14.8% and 22%, respectively, compared to recent benchmark models. In probability interval prediction, while improving accuracy by 4% at a 95% confidence interval, the interval width decreased by 19%. The results show that the proposed approach has stronger adaptability and higher accuracy, which can provide more valuable references for power grid planning and decision support. [ABSTRACT FROM AUTHOR]

The multi-stage, multi-level assembly job shop scheduling problem (MsMlAJSP) is commonly encountered in the manufacturing of complex customized products. Ensuring production efficiency while effectively improving energy utilization is a key focus in the industry. For the energy-efficient MsMlAJSP (EEMsMlAJSP), an improved imperialist competitive algorithm based on Q-learning (IICA-QL) is proposed to minimize the maximum completion time and total energy consumption. In IICA-QL, a decoding strategy with energy-efficient triggers based on problem characteristics is designed to ensure solution quality while effectively enhancing search efficiency. Additionally, an assimilation operation with operator parameter self-adaptation based on Q-learning is devised to overcome the challenge of balancing exploration and exploitation with fixed parameters; thus, the convergence and diversity of the algorithmic search are enhanced. Finally, the effectiveness of the energy-efficient strategy decoding trigger mechanism and the operator parameter self-adaptation operation based on Q-learning is demonstrated through experimental results, and the effectiveness of IICA-QL for solving the EEMsMlAJSP is verified by comparing it with other algorithms. [ABSTRACT FROM AUTHOR]

The distributed permutation flow shop scheduling problem (DPFSP) is one of the hottest issues in the context of economic globalization. In this paper, a Q-learning enhanced fruit fly optimization algorithm (QFOA) is proposed to solve the DPFSP with the goal of minimizing the makespan. First, a hybrid strategy is used to cooperatively initialize the position of the fruit fly in the solution space and the boundary properties are used to improve the operation efficiency of QFOA. Second, the neighborhood structure based on problem knowledge is designed in the smell stage to generate neighborhood solutions, and the Q-learning method is conducive to the selection of high-quality neighborhood structures. Moreover, a local search algorithm based on key factories is designed to improve the solution accuracy by processing sequences of subjobs from key factories. Finally, the proposed QFOA is compared with the state-of-the-art algorithms for solving 720 well-known large-scale benchmark instances. The experimental results demonstrate the most outstanding performance of QFOA. [ABSTRACT FROM AUTHOR]

With the rapid development of mobile edge computing (MEC) and wireless power transfer (WPT) technologies, the MEC-WPT system makes it possible to provide high-quality data processing services for end users. However, in a real-world WPT-MEC system, the channel gain decreases with the transmission distance, leading to "double near and far effect" in the joint transmission of wireless energy and data, which affects the quality of the data processing service for end users. Consequently, it is essential to design a reasonable system model to overcome the "double near and far effect" and make reasonable scheduling of multi-dimensional resources such as energy, communication and computing to guarantee high-quality data processing services. First, this paper designs a relay collaboration WPT-MEC resource scheduling model to improve wireless energy utilization efficiency. The optimization goal is to minimize the normalization of the total communication delay and total energy consumption while meeting multiple resource constraints. Second, this paper imports a BK-means algorithm to complete the end terminals cluster to guarantee effective energy reception and adapts the whale optimization algorithm with adaptive mechanism (AWOA) for mobile vehicle path-planning to reduce energy waste. Third, this paper proposes an immune differential enhanced deep deterministic policy gradient (IDDPG) algorithm to realize efficient resource scheduling of multiple resources and minimize the optimization goal. Finally, simulation experiments are carried out on different data, and the simulation results prove the validity of the designed scheduling model and proposed IDDPG. [ABSTRACT FROM AUTHOR]

In the past few decades, many multi-objective evolution algorithms (MOEAs) have been proposed, often emphasizing a single crossover operator, which has a significant impact on the algorithm's performance. This paper proposed a novel MOEA, based on the MOEA/D framework and employing Q-learning for adaptive operator selection (QLMOEA/D-AOS). In every Iteration, Q-learning is used to dynamically choose an operator among five crossover operators. To obtain a better distribution of solutions in multi-objective optimization problems with irregular PFs, a new approach for weight vector initializing is proposed. Additionally, to enhance population diversity, a reward calculation method based on two metrics, Spacing and PD, is proposed. Finally, the proposed algorithm is validated for different numbers of objectives, ranging from two to five for multi/many-objective optimization problems. The experimental results demonstrate the significant advantages of the proposed algorithm compared to state-of-the-art MOEAs across multiple test cases. [ABSTRACT FROM AUTHOR]

Advancements in Machine Learning (ML) and Computer Vision have led to notable improvements in the detection of breast cancer. However, the accuracy of the classifier is limited due to imbalanced datasets that cause overfitting. Thus, additional images are needed to improve the classifier's performance. Generative Adversarial Networks (GAN) are used for image augmentation. Still, its limitations, such as the random use of different types of GAN, can lead to a too-good discriminator, causing generator training to fail due to vanishing gradients. Therefore, selecting the appropriate GAN model for the given scenario is crucial for optimal performance. This paper proposes a novel Smart Generative Adversarial Network (Smart GAN) architecture to develop an efficient and computational classification model for a limited imbalanced dataset. Smart GAN uses a three-fold approach. Different types of GAN augment the dataset in the first phase of experimental work, and their evaluation metrics are calculated. In the second phase, the metric scores are used as rewards for the Reinforcement learning model (Q-learning approach). The best augmentation is chosen based on the best Q-values for each metric score. It compares three different Convolutional Neural Networks (CNN) and selects the best-suited network to classify the augmented datasets. The proposed Smart GAN architecture outperforms other existing approaches by giving a better accuracy of 89.62% and 89.91% on Mammographic Image Analysis Society (MIAS) and Digital Database for Screening Mammography (DDSM) augmented datasets, respectively, representing an approximately 10% increment in detection rate compared to non-augmented datasets. [ABSTRACT FROM AUTHOR]

Non-orthogonal Multiple Access (NOMA) techniques offer potential enhancements in spectral efficiency for 5G and 6G wireless networks, facilitating broader network access. Central to realizing optimal system performance are factors like joint power control, user grouping, and decoding order. This study investigates power control and user grouping to optimize spectral efficiency in NOMA uplink systems, aiming to reduce computational difficulty. While previous research on this integrated optimization has identified several near-optimal solutions, they often come with considerable system and computational overheads. To address this, this study employed an improved Grey Wolf Optimizer (GWO), a nature-inspired metaheuristic optimization method. Although GWO is effective, it can sometimes converge prematurely and might lack diversity. To enhance its performance, this study introduces a new version of GWO, integrating Competitive Learning, Q-learning, and Greedy Selection. Competitive learning adopts agent competition, balancing exploration and exploitation and preserving diversity. Q-learning guides the search based on past experiences, enhancing adaptability and preventing redundant exploration of sub-optimal regions. Greedy selection ensures the retention of the best solutions after each iteration. The synergistic integration of these three components substantially enhances the performance of the standard GWO. This algorithm was used to manage power and user-grouping in NOMA systems, aiming to strengthen system performance while restricting computational demands. The effectiveness of the proposed algorithm was validated through numerical evaluations. Simulated outcomes revealed that when applied to the joint challenge in NOMA uplink systems, it surpasses the spectral efficiency of conventional orthogonal multiple access. Moreover, the proposed approach demonstrated superior performance compared to the standard GWO and other state-of-the-art algorithms, achieving reduced system complexity under identical constraints. [ABSTRACT FROM AUTHOR]

With the advancement of radar technology toward multifunctionality and cognitive capabilities, traditional radar countermeasures are no longer sufficient to meet the demands of countering the advanced multifunctional radar (MFR) systems. Rapid and accurate generation of the optimal jamming strategy is one of the key technologies for efficiently completing radar countermeasures. To enhance the efficiency and accuracy of jamming policy generation, an efficient jamming policy generation method based on multi-timescale ensemble Q-learning (MTEQL) is proposed in this paper. First, the task of generating jamming strategies is framed as a Markov decision process (MDP) by constructing a countermeasure scenario between the jammer and radar, while analyzing the principle radar operation mode transitions. Then, multiple structure-dependent Markov environments are created based on the real-world adversarial interactions between jammers and radars. Q-learning algorithms are executed concurrently in these environments, and their results are merged through an adaptive weighting mechanism that utilizes the Jensen–Shannon divergence (JSD). Ultimately, a low-complexity and near-optimal jamming policy is derived. Simulation results indicate that the proposed method has superior jamming policy generation performance compared with the Q-learning algorithm, in terms of the short jamming decision-making time and low average strategy error rate. [ABSTRACT FROM AUTHOR]

The adoption of electric vehicles (EVs) represents a pivotal shift towards sustainable mobility, yet the challenge of efficient charging station recommendations persists, influencing user convenience and EV uptake. This study introduces a novel approach utilizing Q-learning for simulating EV charging station recommendations, aiming to optimize the matching process between EVs and charging infrastructure. By integrating Markov decision processes with Q-learning algorithms, we dynamically adapt recommendations to user behaviours and preferences, significantly enhancing recommendation accuracy and personalization. The methodology involves constructing a simulation environment to model EV charging behaviour, evaluating the performance of the Q-learning based recommendation system under various scenarios. Results demonstrate the effectiveness of this approach in identifying optimal charging strategies, thus contributing to improved user satisfaction and charging station utilization. The findings underscore the importance of innovative technological integration for addressing the complexities of sustainable urban mobility. [ABSTRACT FROM AUTHOR]

Although Q-learning (QL) is widely used in reinforcement learning, it suffers from overestimation bias, which can lead to poor performance in stochastic environments due to its susceptibility to maximization bias. To address this problem, various bias correction mechanisms have been proposed. However, while these mechanisms may reduce overestimation bias, some of them introduce underestimation bias, which is undesirable in some environments. To leverage both overestimation and underestimation biases, we introduce an underestimation mechanism called the min estimator, followed by our proposed Max_Mix_Min Q-learning (M3QL) method, which incorporates a balance parameter β . Our method also considers the number of N Q-functions. Initially, we theoretically analyze why our method benefits from both overestimation and underestimation bias under the assumption of different bias distributions and how sample size N affects the performance of our method. Additionally, we visualize the theoretic analysis results on Meta-chain MDP example. Theoretical analysis demonstrates that M3QL achieves bias reduction compared to QL and the underestimation mechanism. Furthermore, we theoretically prove that M3QL is unbiased on certain values of β . In experimental comparisons with states of arts on Atari benchmark problems, our method consistently outperforms them. We also compare M3QL with the underestimation mechanism and Deep Q-learning (DQN) on these benchmark problems, revealing that M3QL improves the performance of underestimation mechanism and DQN on most of the benchmark problems. [ABSTRACT FROM AUTHOR]

The traditional industrial big data fusion algorithm has low efficiency and difficulty in processing high-dimensional data, this paper proposes a Q-learningbased cascade classifier model for industrial big data fusion. By combining cascade classifier and softmax classifier, feature extraction and data attribute classification of source industrial big data are completed in this cluster. In order to improve the classification rate, an improved Q-learning algorithm is proposed, which makes the improved algorithm randomly select actions in the early stage, and dynamically change in the late stage in the random selection of actions and actions with the highest reward value. It effectively improves the defects of traditional Q-learning algorithm that it is easy to fall into the local optimal and has slow convergence speed. The experimental results show that compared with other advanced fusion algorithms, the proposed method can greatly reduce the network energy consumption and effectively improve the efficiency and accuracy of data fusion under the same data volume. [ABSTRACT FROM AUTHOR]

Penetration Testing (PT), which mimics actual cyber attacks, has become an essential procedure for assessing the security posture of network infrastructures in recent years. Automated PT reduces human labor, increases scalability, and allows for more frequent evaluations. Real-world exploitation still challenges RL-based penetration testing because the agent's many possible actions make it hard for the algorithm to converge. To resolve these shortcomings, a deep learning- model named Adaptive Hunt Tuner algorithm optimized Q-learning based deep Convolutional neural Network (AHT-QCN) is developed for efficient PT. Specifically, the Q-learning employed in this model improves its efficiency by enabling optimal policy learning for decision-making. In addition, the Adaptive Hunt Tuner (AHT) algorithm enhances the model's performance by tuning its parameters with reduced computational time. The experimental outcomes demonstrate that the developed model attains 95.25% accuracy, 97.66% precision, and 93.81% F1 score. [ABSTRACT FROM AUTHOR]

Nowadays, several service providers in urban areas significantly consider vehicular ad hoc networks (VANET). VANETs can enhance road safety, prevent accidents, and grant passengers entertainment. Though in VANET, efficient routing has remained an open problem. VANET is dynamic; the frequent update in the situation originates through several aspects, such as traffic conditions and updates in the road topology, which demand a suitably adaptive routing. The existence of blocking obstacles degrades routing approaches and increases the failure of paths. These issues build an excessive amount of resource utilization and increase network delay. To solve these issues, obstacle detection to minimize delay and Q-learning to improve routing efficiency (ODQI) in VANET is proposed. This mechanism uses the spanning tree algorithm detects the obstacle. Clustering can be used to manage the topology in VANETs. The dingo algorithm selects the best cluster head (CH) based on vehicle bandwidth, speed, and link lifespan. Furthermore, the sender forwards the traffic information from the sender to the receiver by applying a Q-learning algorithm. This learning algorithm computes the award function to choose the forwarder, improving the routing efficiency. Simulation results demonstrate that the ODQI mechanism increases the CH lifetime and minimizes the network delay. [ABSTRACT FROM AUTHOR]

Tracking control of the output probability density function presents significant challenges, particularly when dealing with unknown system models and multiplicative noise disturbances. To address these challenges, this paper introduces a novel tracking control algorithm based on reinforce-ment Q-learning. Initially, a B-spline model is employed to represent the original system, thereby transforming the control problem into a state weight tracking issue within the B-spline stochastic system model. Moreover, to tackle the challenge of unknown stochastic system dynamics and the presence of multiplicative noise, a model-free reinforcement Q-learning algorithm is employed to solve the control problem. Finally, the proposed algorithm's effectiveness is validated through comprehensive simulation examples. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive study on the optimization of electric vehicle (EV) battery management using Q-learning, a powerful reinforcement learning technique. As the demand for electric vehicles continues to grow, there is an increasing need for efficient battery-management strategies to extend battery life, enhance performance, and minimize operating costs. The primary objective of this research is to develop and assess a Q-learning-based approach to address the intricate challenges associated with EV battery management. This paper starts by elucidating the key challenges inherent in EV battery management and discusses the potential advantages of incorporating Q-learning into the optimization process. Leveraging Q-learning's capacity to make dynamic decisions based on past experiences, we introduce a framework that considers state-of-charge, state-of-health, charging infrastructure, and driving patterns as critical state variables. The methodology is detailed, encompassing the selection of state, action, reward, and policy, with the training process informed by real-world data. Our experimental results underscore the efficacy of the Q-learning approach in optimizing battery management. Through the utilization of Q-learning, we achieve substantial enhancements in battery performance, energy efficiency, and overall EV sustainability. A comparative analysis with traditional battery-management strategies is presented to highlight the superior performance of our approach. A comparative analysis with traditional battery-management strategies is presented to highlight the superior performance of our approach, demonstrating compelling results. Our Q-learning-based method achieves a significant 15% improvement in energy efficiency compared to conventional methods, translating into substantial savings in operational costs and reduced environmental impact. Moreover, we observe a remarkable 20% increase in battery lifespan, showcasing the effectiveness of our approach in enhancing long-term sustainability and user satisfaction. This paper significantly enriches the body of knowledge on EV battery management by introducing an innovative, data-driven approach. It provides a comprehensive comparative analysis and applies novel methodologies for practical implementation. The implications of this research extend beyond the academic sphere to practical applications, fostering the broader adoption of electric vehicles and contributing to a reduction in environmental impact while enhancing user satisfaction. [ABSTRACT FROM AUTHOR]

In this paper, a Q-learning-based task scheduling approach for mixed-critical application on heterogeneous multi-cores (QSMix) to optimize their main design challenges is proposed. This approach employs reinforcement learning capabilities to optimize execution time, power consumption, reliability and temperature of the heterogeneous multi-cores during task scheduling process. In QSMix, a reward function is defined to consider all target design parameters simultaneously and is tuned based on applying punishment for unwanted conditions during the learning. The learning process of QSMix is led by utilizing the defined reward function during constructing the Q-table for various execution scenarios. Afterward, the best solution is selected from the constructed Q-table based on the system's policy to achieve a near-optimal solution that meets the existing trade-offs among objectives while considering its constraints properly. To evaluate our proposed QSMix, several experiments are performed to show its effectiveness in finding appropriate solutions and its gradual behavior during learning process. Moreover, the performance of QSMix in terms of optimizing the target design parameters is compared to various related research. The results confirm that QSMix has average improvement about 9% over related studies in joint optimization of execution time, power consumption, reliability and temperature. [ABSTRACT FROM AUTHOR]

The transmission environment of underwater wireless sensor networks is open, and important transmission data can be easily intercepted, interfered with, and tampered with by malicious nodes. Malicious nodes can be mixed in the network and are difficult to distinguish, especially in time-varying underwater environments. To address this issue, this article proposes a GAN-based trusted routing algorithm (GTR). GTR defines the trust feature attributes and trust evaluation matrix of underwater network nodes, constructs the trust evaluation model based on a generative adversarial network (GAN), and achieves malicious node detection by establishing a trust feature profile of a trusted node, which improves the detection performance for malicious nodes in underwater networks under unlabeled and imbalanced training data conditions. GTR combines the trust evaluation algorithm with the adaptive routing algorithm based on Q-Learning to provide an optimal trusted data forwarding route for underwater network applications, improving the security, reliability, and efficiency of data forwarding in underwater networks. GTR relies on the trust feature profile of trusted nodes to distinguish malicious nodes and can adaptively select the forwarding route based on the status of trusted candidate next-hop nodes, which enables GTR to better cope with the changing underwater transmission environment and more accurately detect malicious nodes, especially unknown malicious node intrusions, compared to baseline algorithms. Simulation experiments showed that, compared to baseline algorithms, GTR can provide a better malicious node detection performance and data forwarding performance. Under the condition of 15% malicious nodes and 10% unknown malicious nodes mixed in, the detection rate of malicious nodes by the underwater network configured with GTR increased by 5.4%, the error detection rate decreased by 36.4%, the packet delivery rate increased by 11.0%, the energy tax decreased by 11.4%, and the network throughput increased by 20.4%. [ABSTRACT FROM AUTHOR]

In this work, we explore the application of deep reinforcement learning (DRL) to algorithmic trading. While algorithmic trading is focused on using computer algorithms to automate a predefined trading strategy, in this work, we train a Double Deep Q-Network (DDQN) agent to learn its own optimal trading policy, with the goal of maximising returns whilst managing risk. In this study, we extended our approach by augmenting the Markov Decision Process (MDP) states with sentiment analysis of financial statements, through which the agent achieved up to a 70% increase in the cumulative reward over the testing period and an increase in the Calmar ratio from 0.9 to 1.3. The experimental results also showed that the DDQN agent's trading strategy was able to consistently outperform the benchmark set by the buy-and-hold strategy. Additionally, we further investigated the impact of the length of the window of past market data that the agent considers when deciding on the best trading action to take. The results of this study have validated DRL's ability to find effective solutions and its importance in studying the behaviour of agents in markets. This work serves to provide future researchers with a foundation to develop more advanced and adaptive DRL-based trading systems. [ABSTRACT FROM AUTHOR]

• A novel fault-tolerant controller has been developed for the attitude control of rigid spacecraft based on Q-learning. • This controller obviates the necessity for actuator fault data or extensive fault knowledge. • The controller stability analysis and controller implementation are discussed. This paper presents a novel active fault-tolerant control (FTC) scheme based on reinforcement learning (RL) for rigid spacecraft operating in challenging conditions with simultaneous actuator faults and external disturbances. Initially, the paper outlines the dynamics of a rigid spacecraft afflicted by actuator faults and subject to external disturbances. Subsequently, an observer is designed to swiftly detect actuator faults, ensuring a timely response to fault occurrences. An indirect fault estimator is then employed to estimate the total faults affecting the system. Based on the estimated total faults, the proposed decision mechanism switches the controller from the nominal to the fault-tolerant controller. The proposed fault-tolerant controller is model-free and utilizes the Q-learning algorithm. This Q-learning-based fault-tolerant controller can be implemented online without relying on explicit system models or actuator fault details. Notably, this innovative controller operates independently from fault detection and identification (FDI), utilizing data extracted from system trajectories. The stability of the fault-tolerant controller is established using Lyapunov techniques, providing rigorous validation of its effectiveness in maintaining system stability and achieving satisfactory performance. The performance and adaptability of the proposed approach are assessed through comprehensive simulation studies, emphasizing its capacity to enhance spacecraft fault tolerance in demanding operational scenarios. [ABSTRACT FROM AUTHOR]

Tool condition monitoring (TCM) is crucial in the machining process to confirm product quality as well as process efficiency and minimize downtime. Traditional methods for TCM, while effective to a degree, often fall short in real-time adaptability and predictive accuracy. This research work aims to advance the state-of-the-art methods in predictive maintenance for TCM and improve tool performance and reliability during the milling process. The present work investigates the application of Deep Learning (DL) and Reinforcement Learning (RL) techniques to monitor tool conditions in milling operations. DL models, including Long Short-Term Memory (LSTM) networks, Feed Forward Neural Networks (FFNN), and RL models, including Q-learning and SARSA, are employed to classify tool conditions from the vibration sensor. The performance of the selected DL and RL algorithms is evaluated through performance metrics like confusion matrix, recall, precision, F1 score, and Receiver Operating Characteristics (ROC) curves. The results revealed that RL based on SARSA outperformed other algorithms. The overall classification accuracies for LSTM, FFNN, Q-learning, and SARSA were 94.85%, 98.16%, 98.50%, and 98.66%, respectively. In regard to predicting tool conditions accurately and thereby enhancing overall process efficiency, SARSA showed the best performance, followed by Q-learning, FFNN, and LSTM. This work contributes to the advancement of TCM systems, highlighting the potential of DL and RL techniques to revolutionize manufacturing processes in the era of Industry 5.0. [ABSTRACT FROM AUTHOR]

Abstract To endow the prey with intelligent movement behavior and improve the performance of Golden Jackal Optimization (GJO), a Q-learning Improved Gold Jackal Optimization (QIGJO) algorithm is proposed. This paper introduces five update mechanisms and proposes double-population Q-learning collaborative mechanism to select appropriate update mechanisms to improve GJO performance. Additionally, a new convergence factor is incorporated to enhance convergence capability of GJO. QIGJO demonstrates excellent performance across 23 benchmark functions, CEC2022, and three classical engineering design problems, indicating high convergence accuracy and significantly enhanced global exploration capability. The reliability optimization model of the hydraulic system for concrete pump trucks was established based on a Continuous-time Multi-dimensional T-S dynamic Fault Tree (CM-TSdFT), considering the two-dimensional factors of operating time and number of impacts. Utilizing QIGJO to optimize this model yielded excellent results, providing valuable methodological support for reliability optimization of hydraulic systems.

Clearly determining the magnitude of fracture pressure is a crucial indicator for fracturing design. Traditional methods for predicting fracture pressure suffer from challenges such as difficulties in obtaining required data, low prediction accuracy, and local limitations in application. In light of these issues, the article proposes a fracture pressure prediction model based on reinforcement learning and XGBoost utilizing geophysical well logging data. Based on the relevance analysis, optimal input parameters, including DEPTH, DEN, AC, GR, CRL, and RT, are selected from geophysical well logging data. We have developed a framework for a fracture pressure prediction model based on XGBoost, wherein hyperparameters are fine-tuned using an improved Q-learning algorithm. The optimized XGBoost model for fracture pressure prediction attains outstanding performance metrics, including an R 2 value of 0.992, a root mean square error of 0.006%, and a mean absolute error of 0.539%. In direct comparison with grid search, Bayesian optimization, and ant colony optimization, the improved Q-learning algorithm emerges as the most effective optimization approach. The predictions generated by the proposed method exhibit remarkable consistency with fracture pressure data measured on-site. This approach successfully addresses the shortcomings encountered with traditional fracture pressure prediction methods, such as inadequate accuracy, demanding data prerequisites, and constrained applicability.

Abstract Predicting rock tunnel squeezing in underground projects is challenging due to its intricate and unpredictable nature. This study proposes an innovative approach to enhance the accuracy and reliability of tunnel squeezing prediction. The proposed method combines ensemble learning techniques with Q-learning and online Markov chain integration. A deep learning model is trained on a comprehensive database comprising tunnel parameters including diameter (D), burial depth (H), support stiffness (K), and tunneling quality index (Q). Multiple deep learning models are trained concurrently, leveraging ensemble learning to capture diverse patterns and improve prediction performance. Integration of the Q-learning-Online Markov Chain further refines predictions. The online Markov chain analyzes historical sequences of tunnel parameters and squeezing class transitions, establishing transition probabilities between different squeezing classes. The Q-learning algorithm optimizes decision-making by learning the optimal policy for transitioning between tunnel states. The proposed model is evaluated using a dataset from various tunnel construction projects, assessing performance through metrics like accuracy, precision, recall, and F1-score. Results demonstrate the efficiency of the ensemble deep learning model combined with Q-learning-Online Markov Chain in predicting surrounding rock tunnel squeezing. This approach offers insights into parameter interrelationships and dynamic squeezing characteristics, enabling proactive planning and support measures implementation to mitigate tunnel squeezing hazards and ensure underground structure safety. Experimental results show the model achieves a prediction accuracy of 98.11%, surpassing individual CNN and RNN models, with an AUC value of 0.98.

Abstract Non-orthogonal Multiple Access (NOMA) techniques offer potential enhancements in spectral efficiency for 5G and 6G wireless networks, facilitating broader network access. Central to realizing optimal system performance are factors like joint power control, user grouping, and decoding order. This study investigates power control and user grouping to optimize spectral efficiency in NOMA uplink systems, aiming to reduce computational difficulty. While previous research on this integrated optimization has identified several near-optimal solutions, they often come with considerable system and computational overheads. To address this, this study employed an improved Grey Wolf Optimizer (GWO), a nature-inspired metaheuristic optimization method. Although GWO is effective, it can sometimes converge prematurely and might lack diversity. To enhance its performance, this study introduces a new version of GWO, integrating Competitive Learning, Q-learning, and Greedy Selection. Competitive learning adopts agent competition, balancing exploration and exploitation and preserving diversity. Q-learning guides the search based on past experiences, enhancing adaptability and preventing redundant exploration of sub-optimal regions. Greedy selection ensures the retention of the best solutions after each iteration. The synergistic integration of these three components substantially enhances the performance of the standard GWO. This algorithm was used to manage power and user-grouping in NOMA systems, aiming to strengthen system performance while restricting computational demands. The effectiveness of the proposed algorithm was validated through numerical evaluations. Simulated outcomes revealed that when applied to the joint challenge in NOMA uplink systems, it surpasses the spectral efficiency of conventional orthogonal multiple access. Moreover, the proposed approach demonstrated superior performance compared to the standard GWO and other state-of-the-art algorithms, achieving reduced system complexity under identical constraints.

The peg-in-hole task is important in robotics. Visual inspection is a crucial method for recognition and positioning during this task. Currently, vision-based peg-in-hole techniques suffer from limited applicability and low positioning accuracy. Therefore, this study introduced a general vision-based approach for robotic peg-in-hole tasks. This approach delineates the process into two stages. First, during the object recognition, positioning, and approach phases, a coarse adjustment technique for the assembly pose of the robot's end effector was proposed based on object recognition. This method determines the pose of the hole to be assembled through ellipse fitting, thereby guiding the robot to approach the hole. Second, a Q-learning-based method was introduced to fine adjust the robot end effector's pose and position. Q-learning was applied to the scenario of small-scale adjustment of the robotic peg-in-hole, the reward function based on the pixel area of the gap and the included angle between central axes of peg and hole are designed. Finally, the feasibility and efficacy of this method are substantiated through a series of assembly experiments. [ABSTRACT FROM AUTHOR]

Summary Due to the convenience of virtualization, the live migration of virtual machines is widely used to fulfill optimization objectives in cloud/edge computing. However, live migration may lead to side effects and performance degradation when migration is overused or an unreasonable migration process is carried out. One pressing challenge is how to capture the best opportunity for virtual machine migration. Leveraging rough sets and AI, this paper provides an innovative strategy based on Q-learning that is designed for migration decisions. The highlight of our strategy is the harmonious mechanism for applying rough sets and Q-learning. For the ABDS (adaptive boundary decision system) strategy in this paper, the exploration space of Q learning is confined by the boundary region of rough sets, while the thresholds of the boundary region can be dynamically adjusted by the reaction results from the computing cluster. The structure and mechanism of the ABDS strategy are described in this paper. The corresponding experiments show a firm advantage for the cooperation of rough sets and reinforcement learning algorithms. Considering both the energy consumption and application performance, the ABDS strategy in this paper outperforms the benchmark strategies in comprehensive performance.

Remanufacturing is regarded as a sustainable manufacturing paradigm of energy conservation and environment protection. To improve the efficiency of the remanufacturing process, this work investigates an integrated scheduling problem for disassembly and reprocessing in a remanufacturing process, where product structures and uncertainty are taken into account. First, a stochastic programming model is developed to minimize the maximum completion time (makespan). Second, a Q-learning based hybrid meta-heuristic (Q-HMH) is specially devised. In each iteration, a Q-learning method is employed to adaptively choose a premium algorithm from four candidate ones, including genetic algorithm (GA), artificial bee colony (ABC), shuffled frog-leaping algorithm (SFLA), and simulated annealing (SA) methods. At last, simulation experiments are carried out by using sixteen instances with different scales, and three state-of-the-art algorithms in literature and an exact solver CPLEX are chosen for comparisons. By analyzing the results with the average relative percentage deviation (RPD) metric, we find that Q-HMH outperforms its rivals by 9.79%−26.76%. The results and comparisons verify the excellent competitiveness of Q-HMH for solving the concerned problems.

Abstract The distributed permutation flow shop scheduling problem (DPFSP) is one of the hottest issues in the context of economic globalization. In this paper, a Q-learning enhanced fruit fly optimization algorithm (QFOA) is proposed to solve the DPFSP with the goal of minimizing the makespan. First, a hybrid strategy is used to cooperatively initialize the position of the fruit fly in the solution space and the boundary properties are used to improve the operation efficiency of QFOA. Second, the neighborhood structure based on problem knowledge is designed in the smell stage to generate neighborhood solutions, and the Q-learning method is conducive to the selection of high-quality neighborhood structures. Moreover, a local search algorithm based on key factories is designed to improve the solution accuracy by processing sequences of subjobs from key factories. Finally, the proposed QFOA is compared with the state-of-the-art algorithms for solving 720 well-known large-scale benchmark instances. The experimental results demonstrate the most outstanding performance of QFOA.

Summary: Due to the convenience of virtualization, the live migration of virtual machines is widely used to fulfill optimization objectives in cloud/edge computing. However, live migration may lead to side effects and performance degradation when migration is overused or an unreasonable migration process is carried out. One pressing challenge is how to capture the best opportunity for virtual machine migration. Leveraging rough sets and AI, this paper provides an innovative strategy based on Q-learning that is designed for migration decisions. The highlight of our strategy is the harmonious mechanism for applying rough sets and Q-learning. For the ABDS (adaptive boundary decision system) strategy in this paper, the exploration space of Q learning is confined by the boundary region of rough sets, while the thresholds of the boundary region can be dynamically adjusted by the reaction results from the computing cluster. The structure and mechanism of the ABDS strategy are described in this paper. The corresponding experiments show a firm advantage for the cooperation of rough sets and reinforcement learning algorithms. Considering both the energy consumption and application performance, the ABDS strategy in this paper outperforms the benchmark strategies in comprehensive performance. [ABSTRACT FROM AUTHOR]

Path planning in battlefield environments differs from typical path planning because it may not always involve obstacle avoidance. In fact, obstacles can be utilized to hide from enemies or facilitate troop advancement through destruction and relocation. Therefore, path planning in such environments requires specificity. One widely used method for obstacle avoidance is the artificial potential field (APF) method. This study proposes a strategic artificial potential field (SAPF) algorithm that utilizes obstacles for path planning. The Q-learning algorithm is also being researched for its application in learning methods involving path planning and obstacle avoidance. This study proposes a path planning algorithm utilizing SAPF-based Q-learning that takes into account concealment and cover in various terrains. The proposed Q-SAPF algorithm was compared to Q-learning based on the results of repeated experiments and analysis of variance (ANOVA) and the post hoc Tukey technique to examine the statistical significance and differences between the algorithms, respectively. Q-SAPF outperformed Q-learning by generating shorter path plans with differences in path lengths of 1.96–8.77. It further achieved faster learning times of approximately 14.57–69.29 s and path scores that were higher by approximately 2.73–47.96. Specifically, its success rate was 13.73–54.89% higher than that of Q-learning. The achieved outcome demonstrated the superior learning ability of the Q-SAPF. The study results confirm the feasibility of path planning that incorporates concealment and cover by utilizing obstacles. The findings of this study can contribute to path planning in various situations beyond wartime environments. [ABSTRACT FROM AUTHOR]

The Angry Birds AI Competition engages artificial intelligence agents in a contest based on the game Angry Birds. This tournament has been conducted annually since 2012, with participants competing for high scores. The organizers of this competition provide a basic agent, termed "Naive Agent," as a baseline indicator. This study enhanced the Naive Agent by integrating a profit-sharing approach known as exploitation-oriented learning, which is a type of experience-enhanced learning. The effectiveness of this method was substantiated through numerical experiments. Additionally, this study explored the use of level selection learning within a multi-agent environment and validated the utility of the rationality theorem concerning the indirect rewards in this environment. [ABSTRACT FROM AUTHOR]

In view of the problems existing in the path planning algorithms of unmanned vehicles, such as low search efficiency, slow convergence speed and easy to fall into the local optimal. Based on the characteristics of route planning for unmanned vehicles, this paper introduces Q-Learning into the traditional ant colony algorithm to enhance the learning ability of the algorithm in dynamic environment, so as to improve the overall efficiency of route search. By mapping pheromones into Q values in Q-learning, rapid search in complex environments is realized, and a collection-free path satisfying constraints is quickly found. The results of case analysis show that compared with the traditional ant colony algorithm and the improved ant colony algorithm considering reward and punishment factors, the improved ant colony algorithm based on Q-Learning can effectively reduce the number of iterations, shorten the path optimization time and path length and other performance indicators, and has many advantages in jumping out of the local optimal, improving the global search ability and improving the convergence speed, and has good adaptability and robustness in complex environments. It ensures the safety and stability of unmanned vehicles in complex environments. [ABSTRACT FROM AUTHOR]

Social network analysis is a rapidly evolving research area having several real-life application areas, e.g. digital marketing, epidemiology, spread of misinformation. Influence maximization aims to select a subset of nodes in such manner that the information propagated over the network is maximized. Competitive influence maximization, which describes the phenomena of multiple actors competing for resources within the same infrastructure, can be solved with a greedy approach selecting the seed nodes utilizing the influence strength between nodes. Recently, deep reinforcement learning methods were applied for estimating the influence strength. We train a controller with reinforcement learning for selecting a node list of given length as the initial seed set for the information spread. Our experiments show that deep Q-learning methods are suitable to analyze the competitive influence maximization on trust and distrust based social networks. [ABSTRACT FROM AUTHOR]

In recent years, the increasing production and sales of automobiles have led to a notable rise in congestion on urban road traffic systems, particularly at ramps and intersections with traffic signals. Intelligent traffic signal control represents an effective means of addressing traffic congestion. Reinforcement learning methods have demonstrated considerable potential for addressing complex traffic signal control problems with multidimensional states and actions. In this research, the team propose Q-learning and Deep Q-Network (DQN) based signal control frameworks that use variable phase sequences and cycle times to adjust the order and the duration of signal phases to obtain a stable traffic signal control strategy. Experiments are simulated using the traffic simulator Simulation of Urban Mobility (SUMO) to test the average speed and the lane occupancy rate of vehicles entering the ramp to evaluate its safety performance and test the vehicle's traveling time to assess its stability. The simulation results show that both reinforcement learning algorithms are able to control cars in dynamic traffic environments with higher average speed and lower lane occupancy rate than the no-control method and that the DQN control model improves the average speed by about 10% and reduces the lane occupancy rate by about 30% compared to the Q-learning control model, providing a higher safety performance. [ABSTRACT FROM AUTHOR]

This research explores the use of Q-Learning for real-time swarm (Q-RTS) multi-agent reinforcement learning (MARL) algorithm for robotic applications. This study investigates the efficacy of Q-RTS in the reducing convergence time to a satisfactory movement policy through the successful implementation of four and eight trained agents. Q-RTS has been shown to significantly reduce search time in terms of training iterations, from almost a million iterations with one agent to 650,000 iterations with four agents and 500,000 iterations with eight agents. The scalability of the algorithm was addressed by testing it on several agents' configurations. A central focus was placed on the design of a sophisticated reward function, considering various postures of the agents and their critical role in optimizing the Q-learning algorithm. Additionally, this study delved into the robustness of trained agents, revealing their ability to adapt to dynamic environmental changes. The findings have broad implications for improving the efficiency and adaptability of robotic systems in various applications such as IoT and embedded systems. The algorithm was tested and implemented using the Georgia Tech Robotarium platform, showing its feasibility for the above-mentioned applications. [ABSTRACT FROM AUTHOR]

The transition to Industry 4.0 and 5.0 underscores the need for integrating humans into manufacturing processes, shifting the focus towards customization and personalization rather than traditional mass production. However, human performance during task execution may vary. To ensure high human–robot teaming (HRT) performance, it is crucial to predict performance without negatively affecting task execution. Therefore, to predict performance indirectly, significant factors affecting human performance, such as engagement and task load (i.e., amount of cognitive, physical, and/or sensory resources required to perform a particular task), must be considered. Hence, we propose a framework to predict and maximize the HRT performance. For the prediction of task performance during the development phase, our methodology employs features extracted from physiological data as inputs. The labels for these predictions—categorized as accurate performance or inaccurate performance due to high/low task load—are meticulously crafted using a combination of the NASA TLX questionnaire, records of human performance in quality control tasks, and the application of Q-Learning to derive task-specific weights for the task load indices. This structured approach enables the deployment of our model to exclusively rely on physiological data for predicting performance, thereby achieving an accuracy rate of 95.45% in forecasting HRT performance. To maintain optimized HRT performance, this study further introduces a method of dynamically adjusting the robot's speed in the case of low performance. This strategic adjustment is designed to effectively balance the task load, thereby enhancing the efficiency of human–robot collaboration. [ABSTRACT FROM AUTHOR]

Vehicular ad hoc networks (VANETs) use multiple channels to communicate using wireless access in vehicular environment (WAVE) standards to provide a variety of vehicle-related applications. The current IEEE 802.11p WAVE communication channel structure is composed of one control channel (CCH) and several service channels (SCHs). SCHs are used for non-safety data transmission, while the CCH is used for broadcasting beacons, control, and safety. WAVE devices transmit data that alternate between CCHs and SCHs, and each channel is active for a duration called the CCH interval (CCHI) and SCH interval (SCHI), respectively. Currently, both intervals are fixed at 50 ms. However, fixed-length intervals cannot effectively respond to dynamically changing traffic loads. Additionally, when many vehicles are simultaneously using the limited channel resources for data transmission, the network performance significantly degrades due to numerous packet collisions. Herein, we propose an adaptive resource allocation technique for efficient data transmission. The technique dynamically adjusts the SCHI and CCHI to improve network performance. Moreover, to reduce data collisions and optimize the network's backoff distribution, the proposed scheme applies reinforcement learning (RL) to provide an intelligent channel access algorithm. The simulation results demonstrate that the proposed scheme can ensure high throughputs and low transmission delays. [ABSTRACT FROM AUTHOR]

Cyclic air braking is a key factor affecting the safe operation of trains on long downhill sections. However, a train's cycle braking strategy is constrained by multiple factors such as driving environment, speed, and air-refilling time. A Q-learning algorithm-based cyclic braking strategy for a heavy haul train on long downhill sections is proposed to address this challenge. First, the operating environment of a heavy haul train on long downhill sections is designed, considering various constraint parameters, such as the characteristics of special operating routes, allowable operating speeds, and train tube air-refilling time. Second, the operating status and braking operation of a heavy haul train on long downhill sections are discretized in order to establish a Q-table based on state–action pairs. The training of algorithm performance is achieved by continuously updating Q-tables. Finally, taking the heavy haul train formation as the study object, actual line data from the Shuozhou–Huanghua Railway are used for experimental simulation, and different hyperparameters and entry speed conditions are considered. The results show that the safe and stable cyclic braking of a heavy haul train on long downhill sections is achieved. The effectiveness of the Q-learning control strategy is verified. [ABSTRACT FROM AUTHOR]

This paper studies the fully cooperative game tracking control problem (FCGTCP) for a class of discrete-time multi-player linear systems with unknown dynamics. The reference trajectory is generated by a command generator system. An augmented multi-player systems composed of the origin multi-player systems and the command generator system is constructed, and an exponential discounted cost function is introduced to derive an augmented fully cooperative game tracking algebraic Riccati equation (FCGTARE). When the system dynamics are known, a model-based policy iteration (PI) algorithm is proposed to solve the augmented FCGTARE. Furthermore, to relax the system dynamics, an online reinforcement Q-learning algorithm is designed to obtain the solution to the augmented FCGTARE. The convergence of designed online reinforcement Q-learning algorithm is proved. Finally, two simulation examples are given to verify the validity of the model-based PI algorithm and online reinforcement Q-learning algorithm. [ABSTRACT FROM AUTHOR]

In the context of wireless systems going forward, particularly in the Beyond 5G (B5G) era, where high data rates and low latency are critical, D2D communication is a pivotal technology that has many advantages. In D2D communication, the allocation of resources plays a critical role in achieving higher throughput while ensuring interference management, improved spectrum and energy efficiency, and system fairness. Conventional resource allocation methodologies encounter challenges in dynamically changing and diverse communication environments. To deal with the dynamic and unpredictable character of channel characteristics, we utilize a distributed iterative resource allocation technique based on the reinforcement learning (RL) approach that empowers the system to learn and adapt to the wireless environment autonomously. In this paper, we formulate a distributed Q learning-based RL method as its real-time learning capabilities, reduced communication overhead, and exploration efficiency make it well-suited for adapting to dynamic D2D environments compared to other RL methods. By applying the Q-learning method, the D2D devices act as learning agents striving to maximize the cumulative rewards. Through interactions with the environment and continuous learning from feedback, these agents adapt to real-time resource allocation decisions over time. Comparing our proposed Q-learning method to the state-of-the-art RL techniques, simulation results show improvements in energy and spectrum efficiency, latency, an increase in Jain's fairness index, and improvements in overall system throughput of about 6 %–8 %. The scalability is found to be 1.69 interpreting that Q-learning exhibits a good scalability as the throughput does not abruptly decrease for an increasing number of devices.

In order to solve problem of low-energy efficiency caused by device-to-device (D2D) interference in hybrid D2D cellular network, energy efficiency improvement based on joint power control and channel allocation (EEPC) algorithm was proposed to improve the energy efficiency.The optimization problem of maximizing energy efficiency was established with the constraint of ensuring the minimum rate of D2D users and cellular users.By using block coordinate descent method, the optimization problem was transformed into two sub-problems of channel allocation and power control, which were solved by Q-learning algorithm, Dinkelbach algorithm and majorization-minimization (MM) respectively.The greedy search factor in Q-learning algorithm was improved, and a dynamic search factor was used to balance the relationship between exploration and exploitation.Performance analysis shows that the proposed EEPC algorithm improves the energy efficiency of the system.

The lack of a central controller, severe resource constraints, and multi-path data routing have turned data exchanges into one of the fundamental challenges of the Internet of Things. Despite numerous research efforts on various aspects of routing and data exchanges, some fundamental challenges such as the instant negative impacts of selecting the best possible path and the absence of measures to observe the dynamic conditions of nodes still exist. This study introduces a method called RI-RPL, based on the development of the RPL routing protocol, along with the use of reinforcement learning to address these challenges effectively. To achieve this, RI-RPL is designed in three general stages. In the first stage, routers are aligned with optimizing the RPL protocol with a focus on the Q-learning algorithm. In the second stage, based on learning and convergence, changes in the parents' learning in different network conditions are supported. In the third stage, control and management changes are coordinated. The reason for choosing this algorithm is its ability to address the desired challenges effectively without wasting network resources for calculations. Simulation results using the Cooja software show that the proposed RI-RPL method, compared to similar recent methods such as ELBRP, RLQRPL, and RPL, has improved successful delivery rates by 4.03%, 13.26%, and 28.87%, respectively, for end-to-end delay by 3.04%, 9.82%, and 13.12%, respectively, for energy consumption optimization by 10.43%, 28.91%, and 36.35%, respectively, for throughput by 10.23%, 28.45%, and 46.88%, respectively, and for network data loss rate by 15.06%, 34.95%, and 49.66%, respectively. [ABSTRACT FROM AUTHOR]

This research paper presents the Buckley-James Q-learning (BJ-Q) algorithm, a cutting-edge method designed to optimize personalized treatment strategies, especially in the presence of right censoring. We critically assess the algorithm's effectiveness in improving patient outcomes and its resilience across various scenarios. Central to our approach is the innovative use of the survival time to impute the reward in Q-learning, employing the Buckley-James method for enhanced accuracy and reliability. Our findings highlight the significant potential of personalized treatment regimens and introduce the BJ-Q learning algorithm as a viable and promising approach. This work marks a substantial advancement in our comprehension of treatment dynamics and offers valuable insights for augmenting patient care in the ever-evolving clinical landscape. [ABSTRACT FROM AUTHOR]

With the continuous advancement of deep space exploration missions, the solar system boundary exploration mission is established as one of the China's most important deep space scientific exploration missions. The mission of the solar system boundary exploration has many challenges such as ultra-remote detection distance, ultra-long operation time, and ultra-long communication delay. Therefore, the problem of high-precision autonomous navigation needs to be solved urgently. This paper designs an autonomous intelligent navigation method based on X-ray pulsars in the cruise phase, which estimate the motion state of the probe in real time. The proposed navigation method employs the Q-learning Extended Kalman filter (QLEKF) to improve navigation accuracy during long periods of self-determining running. The QLEKF selects automatically the error covariance matrix parameter of the process noise and the measurement noise by the reward mechanism of reinforcement learning. Compared to the traditional EKF and AEKF, the QLEKF improves the estimation accuracy of position and velocity. Finally, the simulation result demonstrates the effectiveness and the superiority of the intelligent navigation algorithm based on QLEKF, which can satisfy the high-precision navigation requirements in the cruise phase of the solar system boundary exploration. [ABSTRACT FROM AUTHOR]

This research article delves into the development of a reinforcement learning (RL)-based continuous authentication system utilizing behavioral biometrics for user identification on computing devices. Keystroke dynamics are employed to capture unique behavioral biometric signatures, while a reward-driven RL model is deployed to authenticate users throughout their sessions. The proposed system augments conventional authentication mechanisms, fortifying them with an additional layer of security to create a robust continuous authentication framework compatible with static authentication systems. The methodology entails training an RL model to discern atypical user typing patterns and identify potentially suspicious activities. Each user's historical data are utilized to train an agent, which undergoes preprocessing to generate episodes for learning purposes. The environment involves the retrieval of observations, which are intentionally perturbed to facilitate learning of nonlinear behaviors. The observation vector encompasses both ongoing and summarized features. A binary and minimalist reward function is employed, with principal component analysis (PCA) utilized for encoding ongoing features, and the double deep Q-network (DDQN) algorithm implemented through a fully connected neural network serving as the policy net. Evaluation results showcase training accuracy and equal error rate (EER) ranging from 94.7% to 100% and 0 to 0.0126, respectively, while test accuracy and EER fall within the range of approximately 81.06% to 93.5% and 0.0323 to 0.11, respectively, for all users as encoder features increase in number. These outcomes are achieved through RL's iterative refinement of rewards via trial and error, leading to enhanced accuracy over time as more data are processed and incorporated into the system. [ABSTRACT FROM AUTHOR]