Search

ABSTRACT In this article, we propose an adaptive Kalman filtering with adaptive window length based on Q‐learning for dynamic systems with unknown model information. The iteration step length of the Q‐function is quantitatively adjusted through the influence function. The adaptive Kalman filtering algorithm is used to set an appropriate weight matrix for the Q‐function to estimate unknown model parameters. One numerical example and a practice‐oriented case are given to illustrate the effectiveness of the proposed method. It is shown that this filtering can provide state estimates of best accuracy among all the compared methods when the model mismatch and noise statistical characteristics change. [ABSTRACT FROM AUTHOR]

To endow the prey with intelligent movement behavior and improve the performance of Golden Jackal Optimization (GJO), a Q-learning Improved Gold Jackal Optimization (QIGJO) algorithm is proposed. This paper introduces five update mechanisms and proposes double-population Q-learning collaborative mechanism to select appropriate update mechanisms to improve GJO performance. Additionally, a new convergence factor is incorporated to enhance convergence capability of GJO. QIGJO demonstrates excellent performance across 23 benchmark functions, CEC2022, and three classical engineering design problems, indicating high convergence accuracy and significantly enhanced global exploration capability. The reliability optimization model of the hydraulic system for concrete pump trucks was established based on a Continuous-time Multi-dimensional T-S dynamic Fault Tree (CM-TSdFT), considering the two-dimensional factors of operating time and number of impacts. Utilizing QIGJO to optimize this model yielded excellent results, providing valuable methodological support for reliability optimization of hydraulic systems. [ABSTRACT FROM AUTHOR]

For the relativistic navigation system where the position and velocity of the spacecraft are determined through the observation of the relativistic perturbations including stellar aberration and starlight gravitational deflection, a novel parallel Q-learning extended Kalman filter (PQEKF) is presented to implement the measurement bias calibration. The relativistic perturbations are extracted from the inter-star angle measurement achieved with a group of high-accuracy star sensors on the spacecraft. Inter-star angle measurement bias caused by the misalignment of the star sensors is one of the main error sources in the relativistic navigation system. In order to suppress the unfavorable effect of measurement bias on navigation performance, the PQEKF is developed to estimate the position and velocity, together with the calibration parameters, where the Q-learning approach is adopted to fine tune the process noise covariance matrix of the filter automatically. The high performance of the presented method is illustrated via numerical simulations in the scenario of medium Earth orbit (MEO) satellite navigation. The simulation results show that, for the considered MEO satellite and the presented PQEKF algorithm, in the case that the inter-star angle measurement accuracy is about 1 mas, after calibration, the positioning accuracy of the relativistic navigation system is less than 300 m. [ABSTRACT FROM AUTHOR]

Clinicians and patients must make treatment decisions at a series of key decision points throughout disease progression. A dynamic treatment regime is a set of sequential decision rules that return treatment decisions based on accumulating patient information, like that commonly found in electronic medical record (EMR) data. When applied to a patient population, an optimal treatment regime leads to the most favorable outcome on average. Identifying optimal treatment regimes that maximize residual life is especially desirable for patients with life-threatening diseases such as sepsis, a complex medical condition that involves severe infections with organ dysfunction. We introduce the residual life value estimator (ReLiVE), an estimator for the expected value of cumulative restricted residual life under a fixed treatment regime. Building on ReLiVE, we present a method for estimating an optimal treatment regime that maximizes expected cumulative restricted residual life. Our proposed method, ReLiVE-Q, conducts estimation via the backward induction algorithm Q-learning. We illustrate the utility of ReLiVE-Q in simulation studies, and we apply ReLiVE-Q to estimate an optimal treatment regime for septic patients in the intensive care unit using EMR data from the Multiparameter Intelligent Monitoring Intensive Care database. Ultimately, we demonstrate that ReLiVE-Q leverages accumulating patient information to estimate personalized treatment regimes that optimize a clinically meaningful function of residual life. [ABSTRACT FROM AUTHOR]

Predicting rock tunnel squeezing in underground projects is challenging due to its intricate and unpredictable nature. This study proposes an innovative approach to enhance the accuracy and reliability of tunnel squeezing prediction. The proposed method combines ensemble learning techniques with Q-learning and online Markov chain integration. A deep learning model is trained on a comprehensive database comprising tunnel parameters including diameter (D), burial depth (H), support stiffness (K), and tunneling quality index (Q). Multiple deep learning models are trained concurrently, leveraging ensemble learning to capture diverse patterns and improve prediction performance. Integration of the Q-learning-Online Markov Chain further refines predictions. The online Markov chain analyzes historical sequences of tunnel parameters and squeezing class transitions, establishing transition probabilities between different squeezing classes. The Q-learning algorithm optimizes decision-making by learning the optimal policy for transitioning between tunnel states. The proposed model is evaluated using a dataset from various tunnel construction projects, assessing performance through metrics like accuracy, precision, recall, and F1-score. Results demonstrate the efficiency of the ensemble deep learning model combined with Q-learning-Online Markov Chain in predicting surrounding rock tunnel squeezing. This approach offers insights into parameter interrelationships and dynamic squeezing characteristics, enabling proactive planning and support measures implementation to mitigate tunnel squeezing hazards and ensure underground structure safety. Experimental results show the model achieves a prediction accuracy of 98.11%, surpassing individual CNN and RNN models, with an AUC value of 0.98. [ABSTRACT FROM AUTHOR]

Given the weak convergence, limited balance capacity, and optimization limitations observed in the Marine Predators Algorithm (MPA), which draws inspiration from the predatory behavior of marine organisms during evolutionary processes, this study introduces a Reinforcement Learning Marine Predators Algorithm (RLMPA). Firstly, based on the predatory characteristics at different stages, we have designed three location update strategies for search agents aimed at creating high-quality candidate solutions from three perspectives. In particular, ranking paired mutually beneficial learning is specifically designed to expand the scope of exploration to generate as many high-quality candidate solutions as possible for future generations. The Gaussian random walk learning is specifically designed to achieve better optimization in the transitional phase by adjusting the step-size control parameters, successfully completing the transition from exploration to local exploitation phase. Additionally, modified somersault foraging strategy is introduced to accelerate local convergence and perform more extensive local exploitation. Secondly, we integrate reinforcement learning into MPA and use Q-learning mechanism to adaptively select location update strategies. Agents fully utilize the collected information to evaluate the next action of the agents, coordinate the exploration phase and exploitation phase, and enhance the global optimization ability. Finally, compared with 10 competitive algorithms, RLMPA achieves better comprehensive performance in global optimization ability, search efficiency and convergence speed on 41 test functions and 5 practical engineering problems. In the Friedman rank sum tests, RLMPA achieves a preferable overall ranking, and has certain ascendant preponderances in solving practical problems with stability, effectiveness and robustness. [ABSTRACT FROM AUTHOR]

We present a trial design for sequential multiple assignment randomized trials (SMARTs) that use a tailoring function instead of a binary tailoring variable allowing for simultaneous development of the tailoring variable and estimation of dynamic treatment regimens (DTRs). We apply methods for developing DTRs from observational data: tree‐based regression learning and Q‐learning. We compare this to a balanced randomized SMART with equal re‐randomization probabilities and a typical SMART design where re‐randomization depends on a binary tailoring variable and DTRs are analyzed with weighted and replicated regression. This project addresses a gap in clinical trial methodology by presenting SMARTs where second stage treatment is based on a continuous outcome removing the need for a binary tailoring variable. We demonstrate that data from a SMART using a tailoring function can be used to efficiently estimate DTRs and is more flexible under varying scenarios than a SMART using a tailoring variable. [ABSTRACT FROM AUTHOR]

With the rapid development of mobile edge computing (MEC) and wireless power transfer (WPT) technologies, the MEC-WPT system makes it possible to provide high-quality data processing services for end users. However, in a real-world WPT-MEC system, the channel gain decreases with the transmission distance, leading to "double near and far effect" in the joint transmission of wireless energy and data, which affects the quality of the data processing service for end users. Consequently, it is essential to design a reasonable system model to overcome the "double near and far effect" and make reasonable scheduling of multi-dimensional resources such as energy, communication and computing to guarantee high-quality data processing services. First, this paper designs a relay collaboration WPT-MEC resource scheduling model to improve wireless energy utilization efficiency. The optimization goal is to minimize the normalization of the total communication delay and total energy consumption while meeting multiple resource constraints. Second, this paper imports a BK-means algorithm to complete the end terminals cluster to guarantee effective energy reception and adapts the whale optimization algorithm with adaptive mechanism (AWOA) for mobile vehicle path-planning to reduce energy waste. Third, this paper proposes an immune differential enhanced deep deterministic policy gradient (IDDPG) algorithm to realize efficient resource scheduling of multiple resources and minimize the optimization goal. Finally, simulation experiments are carried out on different data, and the simulation results prove the validity of the designed scheduling model and proposed IDDPG. [ABSTRACT FROM AUTHOR]

In the past few decades, many multi-objective evolution algorithms (MOEAs) have been proposed, often emphasizing a single crossover operator, which has a significant impact on the algorithm's performance. This paper proposed a novel MOEA, based on the MOEA/D framework and employing Q-learning for adaptive operator selection (QLMOEA/D-AOS). In every Iteration, Q-learning is used to dynamically choose an operator among five crossover operators. To obtain a better distribution of solutions in multi-objective optimization problems with irregular PFs, a new approach for weight vector initializing is proposed. Additionally, to enhance population diversity, a reward calculation method based on two metrics, Spacing and PD, is proposed. Finally, the proposed algorithm is validated for different numbers of objectives, ranging from two to five for multi/many-objective optimization problems. The experimental results demonstrate the significant advantages of the proposed algorithm compared to state-of-the-art MOEAs across multiple test cases. [ABSTRACT FROM AUTHOR]

Advancements in Machine Learning (ML) and Computer Vision have led to notable improvements in the detection of breast cancer. However, the accuracy of the classifier is limited due to imbalanced datasets that cause overfitting. Thus, additional images are needed to improve the classifier's performance. Generative Adversarial Networks (GAN) are used for image augmentation. Still, its limitations, such as the random use of different types of GAN, can lead to a too-good discriminator, causing generator training to fail due to vanishing gradients. Therefore, selecting the appropriate GAN model for the given scenario is crucial for optimal performance. This paper proposes a novel Smart Generative Adversarial Network (Smart GAN) architecture to develop an efficient and computational classification model for a limited imbalanced dataset. Smart GAN uses a three-fold approach. Different types of GAN augment the dataset in the first phase of experimental work, and their evaluation metrics are calculated. In the second phase, the metric scores are used as rewards for the Reinforcement learning model (Q-learning approach). The best augmentation is chosen based on the best Q-values for each metric score. It compares three different Convolutional Neural Networks (CNN) and selects the best-suited network to classify the augmented datasets. The proposed Smart GAN architecture outperforms other existing approaches by giving a better accuracy of 89.62% and 89.91% on Mammographic Image Analysis Society (MIAS) and Digital Database for Screening Mammography (DDSM) augmented datasets, respectively, representing an approximately 10% increment in detection rate compared to non-augmented datasets. [ABSTRACT FROM AUTHOR]

Non-orthogonal Multiple Access (NOMA) techniques offer potential enhancements in spectral efficiency for 5G and 6G wireless networks, facilitating broader network access. Central to realizing optimal system performance are factors like joint power control, user grouping, and decoding order. This study investigates power control and user grouping to optimize spectral efficiency in NOMA uplink systems, aiming to reduce computational difficulty. While previous research on this integrated optimization has identified several near-optimal solutions, they often come with considerable system and computational overheads. To address this, this study employed an improved Grey Wolf Optimizer (GWO), a nature-inspired metaheuristic optimization method. Although GWO is effective, it can sometimes converge prematurely and might lack diversity. To enhance its performance, this study introduces a new version of GWO, integrating Competitive Learning, Q-learning, and Greedy Selection. Competitive learning adopts agent competition, balancing exploration and exploitation and preserving diversity. Q-learning guides the search based on past experiences, enhancing adaptability and preventing redundant exploration of sub-optimal regions. Greedy selection ensures the retention of the best solutions after each iteration. The synergistic integration of these three components substantially enhances the performance of the standard GWO. This algorithm was used to manage power and user-grouping in NOMA systems, aiming to strengthen system performance while restricting computational demands. The effectiveness of the proposed algorithm was validated through numerical evaluations. Simulated outcomes revealed that when applied to the joint challenge in NOMA uplink systems, it surpasses the spectral efficiency of conventional orthogonal multiple access. Moreover, the proposed approach demonstrated superior performance compared to the standard GWO and other state-of-the-art algorithms, achieving reduced system complexity under identical constraints. [ABSTRACT FROM AUTHOR]

With the advancement of radar technology toward multifunctionality and cognitive capabilities, traditional radar countermeasures are no longer sufficient to meet the demands of countering the advanced multifunctional radar (MFR) systems. Rapid and accurate generation of the optimal jamming strategy is one of the key technologies for efficiently completing radar countermeasures. To enhance the efficiency and accuracy of jamming policy generation, an efficient jamming policy generation method based on multi-timescale ensemble Q-learning (MTEQL) is proposed in this paper. First, the task of generating jamming strategies is framed as a Markov decision process (MDP) by constructing a countermeasure scenario between the jammer and radar, while analyzing the principle radar operation mode transitions. Then, multiple structure-dependent Markov environments are created based on the real-world adversarial interactions between jammers and radars. Q-learning algorithms are executed concurrently in these environments, and their results are merged through an adaptive weighting mechanism that utilizes the Jensen–Shannon divergence (JSD). Ultimately, a low-complexity and near-optimal jamming policy is derived. Simulation results indicate that the proposed method has superior jamming policy generation performance compared with the Q-learning algorithm, in terms of the short jamming decision-making time and low average strategy error rate. [ABSTRACT FROM AUTHOR]

The adoption of electric vehicles (EVs) represents a pivotal shift towards sustainable mobility, yet the challenge of efficient charging station recommendations persists, influencing user convenience and EV uptake. This study introduces a novel approach utilizing Q-learning for simulating EV charging station recommendations, aiming to optimize the matching process between EVs and charging infrastructure. By integrating Markov decision processes with Q-learning algorithms, we dynamically adapt recommendations to user behaviours and preferences, significantly enhancing recommendation accuracy and personalization. The methodology involves constructing a simulation environment to model EV charging behaviour, evaluating the performance of the Q-learning based recommendation system under various scenarios. Results demonstrate the effectiveness of this approach in identifying optimal charging strategies, thus contributing to improved user satisfaction and charging station utilization. The findings underscore the importance of innovative technological integration for addressing the complexities of sustainable urban mobility. [ABSTRACT FROM AUTHOR]

Although Q-learning (QL) is widely used in reinforcement learning, it suffers from overestimation bias, which can lead to poor performance in stochastic environments due to its susceptibility to maximization bias. To address this problem, various bias correction mechanisms have been proposed. However, while these mechanisms may reduce overestimation bias, some of them introduce underestimation bias, which is undesirable in some environments. To leverage both overestimation and underestimation biases, we introduce an underestimation mechanism called the min estimator, followed by our proposed Max_Mix_Min Q-learning (M3QL) method, which incorporates a balance parameter β . Our method also considers the number of N Q-functions. Initially, we theoretically analyze why our method benefits from both overestimation and underestimation bias under the assumption of different bias distributions and how sample size N affects the performance of our method. Additionally, we visualize the theoretic analysis results on Meta-chain MDP example. Theoretical analysis demonstrates that M3QL achieves bias reduction compared to QL and the underestimation mechanism. Furthermore, we theoretically prove that M3QL is unbiased on certain values of β . In experimental comparisons with states of arts on Atari benchmark problems, our method consistently outperforms them. We also compare M3QL with the underestimation mechanism and Deep Q-learning (DQN) on these benchmark problems, revealing that M3QL improves the performance of underestimation mechanism and DQN on most of the benchmark problems. [ABSTRACT FROM AUTHOR]

Evolutionary game theory provides a platform to investigate the emergence of cooperation in population consisting of selfish agents. In this work, we study evolutionary games on networks in which agents cooperate or defect according to Q-learning algorithms with extended state space. Extended state space provides agents two types of information, local environment information based on the cooperation level in agents’ neighborhood and personal information based on the last action of agents. Through numerical simulations, we find that rich information on local environment tends to improve cooperation in the population no matter whether personal information is present or not. Moreover, we show that, for the same local environment information, the introduction of personal information may improve cooperation except for the situations with low amount of local environment information where personal information deteriorates cooperation in bad-condition environment. For the same total information, the absence of personal information promotes cooperation in bad-condition environment while the presence of personal information promotes cooperation in good-condition environment. By investigating the distributions and temporal behaviors of Q-values, we present explanations for the above statements. This work suggests an effective way of extending the state space in evolutionary games incorporated with Q-learning algorithm to enhance cooperation. [ABSTRACT FROM AUTHOR]

Tracking control of the output probability density function presents significant challenges, particularly when dealing with unknown system models and multiplicative noise disturbances. To address these challenges, this paper introduces a novel tracking control algorithm based on reinforce-ment Q-learning. Initially, a B-spline model is employed to represent the original system, thereby transforming the control problem into a state weight tracking issue within the B-spline stochastic system model. Moreover, to tackle the challenge of unknown stochastic system dynamics and the presence of multiplicative noise, a model-free reinforcement Q-learning algorithm is employed to solve the control problem. Finally, the proposed algorithm's effectiveness is validated through comprehensive simulation examples. [ABSTRACT FROM AUTHOR]

In this paper, a Q-learning-based task scheduling approach for mixed-critical application on heterogeneous multi-cores (QSMix) to optimize their main design challenges is proposed. This approach employs reinforcement learning capabilities to optimize execution time, power consumption, reliability and temperature of the heterogeneous multi-cores during task scheduling process. In QSMix, a reward function is defined to consider all target design parameters simultaneously and is tuned based on applying punishment for unwanted conditions during the learning. The learning process of QSMix is led by utilizing the defined reward function during constructing the Q-table for various execution scenarios. Afterward, the best solution is selected from the constructed Q-table based on the system's policy to achieve a near-optimal solution that meets the existing trade-offs among objectives while considering its constraints properly. To evaluate our proposed QSMix, several experiments are performed to show its effectiveness in finding appropriate solutions and its gradual behavior during learning process. Moreover, the performance of QSMix in terms of optimizing the target design parameters is compared to various related research. The results confirm that QSMix has average improvement about 9% over related studies in joint optimization of execution time, power consumption, reliability and temperature. [ABSTRACT FROM AUTHOR]

The transmission environment of underwater wireless sensor networks is open, and important transmission data can be easily intercepted, interfered with, and tampered with by malicious nodes. Malicious nodes can be mixed in the network and are difficult to distinguish, especially in time-varying underwater environments. To address this issue, this article proposes a GAN-based trusted routing algorithm (GTR). GTR defines the trust feature attributes and trust evaluation matrix of underwater network nodes, constructs the trust evaluation model based on a generative adversarial network (GAN), and achieves malicious node detection by establishing a trust feature profile of a trusted node, which improves the detection performance for malicious nodes in underwater networks under unlabeled and imbalanced training data conditions. GTR combines the trust evaluation algorithm with the adaptive routing algorithm based on Q-Learning to provide an optimal trusted data forwarding route for underwater network applications, improving the security, reliability, and efficiency of data forwarding in underwater networks. GTR relies on the trust feature profile of trusted nodes to distinguish malicious nodes and can adaptively select the forwarding route based on the status of trusted candidate next-hop nodes, which enables GTR to better cope with the changing underwater transmission environment and more accurately detect malicious nodes, especially unknown malicious node intrusions, compared to baseline algorithms. Simulation experiments showed that, compared to baseline algorithms, GTR can provide a better malicious node detection performance and data forwarding performance. Under the condition of 15% malicious nodes and 10% unknown malicious nodes mixed in, the detection rate of malicious nodes by the underwater network configured with GTR increased by 5.4%, the error detection rate decreased by 36.4%, the packet delivery rate increased by 11.0%, the energy tax decreased by 11.4%, and the network throughput increased by 20.4%. [ABSTRACT FROM AUTHOR]

The primary problem for a resource-limited Wireless Sensor Network is how to extend the system reliability without sacrificing system performance like reception rate and network connectivity. This method effectively deploys sensor nodes and ensures connectivity between the nodes and the transceiver station. Reinforcement Learning (RL) can effectively schedule the sensor nodes' unsupervised activities. A Nash Q-Learning inspired node task scheduling (QL-TS) for service and connection management has been described in this work. An energy-saving MAC Protocol (ESMACP) has been developed to extend the system lifetime. The primary aim of this model is to use the suggested QL-TS-ESMACP that allows sensor devices to learn their best action with minimal energy. The correctness and dependability of the QL-TS-ESMACP can be demonstrated by comparing it with other existing methods. QL-TS-ESMACP outperforms other models in terms of energy efficiency, coverage, node lifespan, and packet delivery ratio in the simulation. [ABSTRACT FROM AUTHOR]

In this work, we explore the application of deep reinforcement learning (DRL) to algorithmic trading. While algorithmic trading is focused on using computer algorithms to automate a predefined trading strategy, in this work, we train a Double Deep Q-Network (DDQN) agent to learn its own optimal trading policy, with the goal of maximising returns whilst managing risk. In this study, we extended our approach by augmenting the Markov Decision Process (MDP) states with sentiment analysis of financial statements, through which the agent achieved up to a 70% increase in the cumulative reward over the testing period and an increase in the Calmar ratio from 0.9 to 1.3. The experimental results also showed that the DDQN agent's trading strategy was able to consistently outperform the benchmark set by the buy-and-hold strategy. Additionally, we further investigated the impact of the length of the window of past market data that the agent considers when deciding on the best trading action to take. The results of this study have validated DRL's ability to find effective solutions and its importance in studying the behaviour of agents in markets. This work serves to provide future researchers with a foundation to develop more advanced and adaptive DRL-based trading systems. [ABSTRACT FROM AUTHOR]

In logistics transportation, the electric vehicle routing problem (EVRP) is researched widely in order to save vehicle power expenditure, reduce transportation costs, and improve service quality. The power expenditure model and routing algorithm are essential for resolving EVRP. To align the routing schedule more reasonable and closer to reality, this paper employs a three-dimensional power expenditure model to calculate the power expenditure of EVs. In this model, the power expenditure of the EVs during the process of going up and downhill is considered to solve the routing schedule of logistics transportation in mountainous areas. This study combines Q-learning and the Re-insertion Genetic Algorithm (Q-RIGA) to design EV routes with low electricity expenditure and reduced transportation costs. The Q-learning algorithm is used to improve route initialization and obtain high-quality initial routes, which are further optimized by RIGA. Tested in a collection of randomly dispersed customer groups, the advantages of the proposed method in terms of convergence speed and power expenditure are confirmed. The three-dimensional power expenditure model with consideration of elevation is used to conduct simulation experiments on the distribution example of Sanlian Dairy in Guizhou to verify that the improved model features broader application and higher practical value. [ABSTRACT FROM AUTHOR]

• A novel fault-tolerant controller has been developed for the attitude control of rigid spacecraft based on Q-learning. • This controller obviates the necessity for actuator fault data or extensive fault knowledge. • The controller stability analysis and controller implementation are discussed. This paper presents a novel active fault-tolerant control (FTC) scheme based on reinforcement learning (RL) for rigid spacecraft operating in challenging conditions with simultaneous actuator faults and external disturbances. Initially, the paper outlines the dynamics of a rigid spacecraft afflicted by actuator faults and subject to external disturbances. Subsequently, an observer is designed to swiftly detect actuator faults, ensuring a timely response to fault occurrences. An indirect fault estimator is then employed to estimate the total faults affecting the system. Based on the estimated total faults, the proposed decision mechanism switches the controller from the nominal to the fault-tolerant controller. The proposed fault-tolerant controller is model-free and utilizes the Q-learning algorithm. This Q-learning-based fault-tolerant controller can be implemented online without relying on explicit system models or actuator fault details. Notably, this innovative controller operates independently from fault detection and identification (FDI), utilizing data extracted from system trajectories. The stability of the fault-tolerant controller is established using Lyapunov techniques, providing rigorous validation of its effectiveness in maintaining system stability and achieving satisfactory performance. The performance and adaptability of the proposed approach are assessed through comprehensive simulation studies, emphasizing its capacity to enhance spacecraft fault tolerance in demanding operational scenarios. [ABSTRACT FROM AUTHOR]

The peg-in-hole task is important in robotics. Visual inspection is a crucial method for recognition and positioning during this task. Currently, vision-based peg-in-hole techniques suffer from limited applicability and low positioning accuracy. Therefore, this study introduced a general vision-based approach for robotic peg-in-hole tasks. This approach delineates the process into two stages. First, during the object recognition, positioning, and approach phases, a coarse adjustment technique for the assembly pose of the robot's end effector was proposed based on object recognition. This method determines the pose of the hole to be assembled through ellipse fitting, thereby guiding the robot to approach the hole. Second, a Q-learning-based method was introduced to fine adjust the robot end effector's pose and position. Q-learning was applied to the scenario of small-scale adjustment of the robotic peg-in-hole, the reward function based on the pixel area of the gap and the included angle between central axes of peg and hole are designed. Finally, the feasibility and efficacy of this method are substantiated through a series of assembly experiments. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) and imitation learning (IL) are quite two useful machine learning techniques that were shown to be potential in enhancing navigation performance. Basically, both of these methods try to find a policy decision function in a reinforcement learning fashion or through imitation. In this paper, we propose a novel algorithm named Reinforcement Imitation Learning (RIL) that naturally combines RL and IL together in accelerating more reliable and efficient navigation in dynamic environments. RIL is a hybrid approach that utilizes RL for policy optimization and IL as some kind of learning from expert demonstrations with the inclusion of guidance. We present the comparison of the convergence of RIL with conventional RL and IL to provide the support for our algorithm's performance in a dynamic environment with moving obstacles. The results of the testing indicate that the RIL algorithm has better collision avoidance and navigation efficiency than traditional methods. The proposed RIL algorithm has broad application prospects in many specific areas such as an autonomous driving, unmanned aerial vehicles, and robots. [ABSTRACT FROM AUTHOR]

Disturbance occurs inevitably on a metro line, resulting in train delays and low service quality. To improve the efficiency of disturbance management, this article investigates the integrated rescheduling problem of train timetable and rolling stock circulation. To balance service quality and operational cost, the considered problem is formulated as a multi-objective optimization model by capturing train operation processes as running arcs, dwell arcs and coming-out/back arcs based on the classical space–time network. For the purpose of improving computational efficiency, a Q-learning-based solution approach is proposed to derive a hierarchical policy for integrated rescheduling. The decisions on train timetable follow a lower-level policy that can be learned in the pre-training process, and the decisions on rolling stock circulation follow a higher-level policy that can be generated in the real-time planning process. Finally, two sets of numerical examples are carried out to evaluate the proposed method. [ABSTRACT FROM AUTHOR]

SaaS is a cloud service model where users obtain software access rights by paying subscription fees. Due to the diversity of business operations, users exhibit significant variations in the online access rates for different software. Consequently, there are variations in the cloud computing resources consumed by different software applications. To avoid the risk of violating SLAs and incurring penalty payments, SaaS operators optimize the computational resource allocation for various software applications and impose subscription limits on each category of software. Considering SLA constraints, this paper formulated a resourceconstrained nonlinear integer programming model with the objective of maximizing revenue. Due to the computational complexity of the model, it cannot be solved in polynomial time, and this paper proposed a Q-learning-PSO hybrid algorithm for this NP-hand problem. This algorithm embedded Q-learning into PSO to dynamically adjust PSO parameters, thereby avoiding the issues of local optima and low computational efficiency associated with direct PSO application. Simulation experiments validate the effectiveness of the model and algorithm in different scenarios. The results indicate that the algorithm can achieve higher revenue for subscription limits and resource allocation with superior solving efficiency under the condition of limited cloud computing resources. Specifically, in scenarios with significant demand fluctuations, operators should aim to reduce the resource contention ratio of software. This can be achieved by provisioning an ample amount of virtual machine resources and enforcing strict subscription limits to ensure the quality of service, consequently reducing penalty payments. Conversely, in scenarios with minimal demand fluctuations, operators have the flexibility to increase the resource contention ratio of software. By relaxing subscription limits, they can seize a larger market share, thus realizing revenue maximization. [ABSTRACT FROM AUTHOR]

This article proposes a novel data‐driven framework of distributed optimal consensus for discrete‐time linear multi‐agent systems under general digraphs. A fully distributed control protocol is proposed by using linear quadratic regulator approach, which is proved to be a sufficient and necessary condition for optimal control of multi‐agent systems through dynamic programming and minimum principle. Moreover, the control protocol can be constructed by using local information with the aid of the solution of the algebraic Riccati equation (ARE). Based on the Q‐learning method, a reinforcement learning framework is presented to find the solution of the ARE in a data‐driven way, in which we only need to collect information from an arbitrary follower to learn the feedback gain matrix. Thus, the multi‐agent system can achieve distributed optimal consensus when system dynamics and global information are completely unavailable. For output feedback cases, accurate state information estimation is established such that optimal consensus control is realized. Moreover, the data‐driven optimal consensus method designed in this article is applicable to general digraph that contains a directed spanning tree. Finally, numerical simulations verify the validity of the proposed optimal control protocols and data‐driven framework. [ABSTRACT FROM AUTHOR]

The Angry Birds AI Competition engages artificial intelligence agents in a contest based on the game Angry Birds. This tournament has been conducted annually since 2012, with participants competing for high scores. The organizers of this competition provide a basic agent, termed "Naive Agent," as a baseline indicator. This study enhanced the Naive Agent by integrating a profit-sharing approach known as exploitation-oriented learning, which is a type of experience-enhanced learning. The effectiveness of this method was substantiated through numerical experiments. Additionally, this study explored the use of level selection learning within a multi-agent environment and validated the utility of the rationality theorem concerning the indirect rewards in this environment. [ABSTRACT FROM AUTHOR]

In Multi-Label Classification (MLC), each data sample is characterized by multiple labels. In ML, there is no restriction on the number of classes a data sample could belong to. Various methods are proposed to construct a model to map data samples to the class labels. Transforming the MLC problem into several binary classification problems is the baseline approach, also called the binary relevance technique. As an alternative solution, the label powerset approach transforms the MLC problem into a multi-class classification problem by building a binary classifier for every label composition. However, in Ensemble Methods (EMs), multiple classifiers are consolidated to form a multi-label ensemble classifier. In this case, each classifier predicts a class label. Thereupon, the predicted class labels are joined using an ensemble method. This paper presents a neoteric EM classifier using the Ridge Regression-based Principal Label Space Transformation (RR-PLST). In the proposed model, the PLST utilizes SVD to discover correlations between the labels and data samples. Dimensionality reduction is a well-known technique to enhance the model's precision and diminish execution time. Consequently, the OPLS technique is recruited to reduce data dimensions in the proposed model. Moreover, in the proposed model, a Q-Learning method is used to amend the initially predicted labels. To demonstrate the effectiveness of the proposed model, vast experiments are conducted on the various widely used multi-label datasets. The proposed algorithm results are compared with similar algorithms by the AP, EM, HS, Macro, and Micro F1 metrics. The experimental results demonstrate the superiority of the proposed model. [ABSTRACT FROM AUTHOR]

The transition to Industry 4.0 and 5.0 underscores the need for integrating humans into manufacturing processes, shifting the focus towards customization and personalization rather than traditional mass production. However, human performance during task execution may vary. To ensure high human–robot teaming (HRT) performance, it is crucial to predict performance without negatively affecting task execution. Therefore, to predict performance indirectly, significant factors affecting human performance, such as engagement and task load (i.e., amount of cognitive, physical, and/or sensory resources required to perform a particular task), must be considered. Hence, we propose a framework to predict and maximize the HRT performance. For the prediction of task performance during the development phase, our methodology employs features extracted from physiological data as inputs. The labels for these predictions—categorized as accurate performance or inaccurate performance due to high/low task load—are meticulously crafted using a combination of the NASA TLX questionnaire, records of human performance in quality control tasks, and the application of Q-Learning to derive task-specific weights for the task load indices. This structured approach enables the deployment of our model to exclusively rely on physiological data for predicting performance, thereby achieving an accuracy rate of 95.45% in forecasting HRT performance. To maintain optimized HRT performance, this study further introduces a method of dynamically adjusting the robot's speed in the case of low performance. This strategic adjustment is designed to effectively balance the task load, thereby enhancing the efficiency of human–robot collaboration. [ABSTRACT FROM AUTHOR]

Vehicular ad hoc networks (VANETs) use multiple channels to communicate using wireless access in vehicular environment (WAVE) standards to provide a variety of vehicle-related applications. The current IEEE 802.11p WAVE communication channel structure is composed of one control channel (CCH) and several service channels (SCHs). SCHs are used for non-safety data transmission, while the CCH is used for broadcasting beacons, control, and safety. WAVE devices transmit data that alternate between CCHs and SCHs, and each channel is active for a duration called the CCH interval (CCHI) and SCH interval (SCHI), respectively. Currently, both intervals are fixed at 50 ms. However, fixed-length intervals cannot effectively respond to dynamically changing traffic loads. Additionally, when many vehicles are simultaneously using the limited channel resources for data transmission, the network performance significantly degrades due to numerous packet collisions. Herein, we propose an adaptive resource allocation technique for efficient data transmission. The technique dynamically adjusts the SCHI and CCHI to improve network performance. Moreover, to reduce data collisions and optimize the network's backoff distribution, the proposed scheme applies reinforcement learning (RL) to provide an intelligent channel access algorithm. The simulation results demonstrate that the proposed scheme can ensure high throughputs and low transmission delays. [ABSTRACT FROM AUTHOR]

Cyclic air braking is a key factor affecting the safe operation of trains on long downhill sections. However, a train's cycle braking strategy is constrained by multiple factors such as driving environment, speed, and air-refilling time. A Q-learning algorithm-based cyclic braking strategy for a heavy haul train on long downhill sections is proposed to address this challenge. First, the operating environment of a heavy haul train on long downhill sections is designed, considering various constraint parameters, such as the characteristics of special operating routes, allowable operating speeds, and train tube air-refilling time. Second, the operating status and braking operation of a heavy haul train on long downhill sections are discretized in order to establish a Q-table based on state–action pairs. The training of algorithm performance is achieved by continuously updating Q-tables. Finally, taking the heavy haul train formation as the study object, actual line data from the Shuozhou–Huanghua Railway are used for experimental simulation, and different hyperparameters and entry speed conditions are considered. The results show that the safe and stable cyclic braking of a heavy haul train on long downhill sections is achieved. The effectiveness of the Q-learning control strategy is verified. [ABSTRACT FROM AUTHOR]

The lack of a central controller, severe resource constraints, and multi-path data routing have turned data exchanges into one of the fundamental challenges of the Internet of Things. Despite numerous research efforts on various aspects of routing and data exchanges, some fundamental challenges such as the instant negative impacts of selecting the best possible path and the absence of measures to observe the dynamic conditions of nodes still exist. This study introduces a method called RI-RPL, based on the development of the RPL routing protocol, along with the use of reinforcement learning to address these challenges effectively. To achieve this, RI-RPL is designed in three general stages. In the first stage, routers are aligned with optimizing the RPL protocol with a focus on the Q-learning algorithm. In the second stage, based on learning and convergence, changes in the parents' learning in different network conditions are supported. In the third stage, control and management changes are coordinated. The reason for choosing this algorithm is its ability to address the desired challenges effectively without wasting network resources for calculations. Simulation results using the Cooja software show that the proposed RI-RPL method, compared to similar recent methods such as ELBRP, RLQRPL, and RPL, has improved successful delivery rates by 4.03%, 13.26%, and 28.87%, respectively, for end-to-end delay by 3.04%, 9.82%, and 13.12%, respectively, for energy consumption optimization by 10.43%, 28.91%, and 36.35%, respectively, for throughput by 10.23%, 28.45%, and 46.88%, respectively, and for network data loss rate by 15.06%, 34.95%, and 49.66%, respectively. [ABSTRACT FROM AUTHOR]

This research paper presents the Buckley-James Q-learning (BJ-Q) algorithm, a cutting-edge method designed to optimize personalized treatment strategies, especially in the presence of right censoring. We critically assess the algorithm's effectiveness in improving patient outcomes and its resilience across various scenarios. Central to our approach is the innovative use of the survival time to impute the reward in Q-learning, employing the Buckley-James method for enhanced accuracy and reliability. Our findings highlight the significant potential of personalized treatment regimens and introduce the BJ-Q learning algorithm as a viable and promising approach. This work marks a substantial advancement in our comprehension of treatment dynamics and offers valuable insights for augmenting patient care in the ever-evolving clinical landscape. [ABSTRACT FROM AUTHOR]

When man⁃made or natural disasters occur suddenly, the rapid deployment of search and rescue (SAR) robots is crucial for saving lives. To accomplish rescue tasks, SAR robots need to autonomously plan paths in continuously dynamic and unknown environments to reach the rescue target locations. This paper proposes a sensor configuration scheme for SAR robots, applying a Q-learning algorithm based on Q-table and neural networks to achieve autonomous control of SAR robots. It addresses the challenge of path planning in unknown environments, specifically how to avoid static and dynamic obstacles. Balancing the exploration and exploitation during the training process is one of the challenges in reinforcement learning. This paper introduces a mixed optimization method for dynamically selecting search strategies, building upon greedy search and Boltzmann search. Simulations are conducted using MATLAB, and the results indicate that the proposed method is feasible and effective. SAR robots equipped with the sensor configuration can effectively respond to environmental changes, reaching target locations while successfully avoiding both static and dynamic obstacles. [ABSTRACT FROM AUTHOR]

Personalized clinical decision support systems are increasingly being adopted due to the emergence of data-driven technologies, with this approach now gaining recognition in critical care. The task of incorporating diverse patient conditions and treatment procedures into critical care decision-making can be challenging due to the heterogeneous nature of medical data. Advances in Artificial Intelligence (AI), particularly Reinforcement Learning (RL) techniques, enables the development of personalized treatment strategies for severe illnesses by using a learning agent to recommend optimal policies. In this study, we propose a Deep Reinforcement Learning (DRL) model with a tailored reward function and an LSTM-GRU-derived state representation to formulate optimal treatment policies for vasopressor administration in stabilizing patient physiological states in critical care settings. Using an ICU dataset and the Medical Information Mart for Intensive Care (MIMIC-III) dataset, we focus on patients with Acute Respiratory Distress Syndrome (ARDS) that has led to Sepsis, to derive optimal policies that can prioritize patient recovery over patient survival. Both the DDQN (RepDRL-DDQN) and Dueling DDQN (RepDRL-DDDQN) versions of the DRL model surpass the baseline performance, with the proposed model's learning agent achieving an optimal learning process across our performance measuring schemes. The robust state representation served as the foundation for enhancing the model's performance, ultimately providing an optimal treatment policy focused on rapid patient recovery. [ABSTRACT FROM AUTHOR]

This research study examines the impact of shifts and lunch breaks on mining operations, particularly focusing on delays in hauling equipment used to supply extracted material to crushers. These delays significantly reduce productivity, averaging below 80% during regular working hours and adversely affecting mine profitability. To address this issue, a Q-learning-based deep reinforcement learning model was developed, utilizing real-world data from mining operations. The model aimed to achieve a 90% coverage in material supply to the crushers. A simulation environment, closely resembling the physical mining setting, was created to test the trucks as agents. Various scenarios, including equipment selection, cycle time, queue times, and material types, were considered. Based on the results, a deep learning model was trained to maximize coverage by determining the optimal combination of trucks and crushers. The solution successfully achieved a 90% supply coverage during shift changes and lunch breaks, with average execution times of less than 1 ms, making it suitable for real-time applications. This research demonstrates the effectiveness of the proposed Q-learning deep reinforcement learning model in optimizing material supply and enhancing mining productivity. By addressing delays and improving operational efficiency, this model holds significant potential for improving profitability in mining operations. [ABSTRACT FROM AUTHOR]

In many real-life scenarios, multiple agents necessitate cooperation to accomplish tasks. Benefiting from the significant success of deep learning, many single-agent deep reinforcement learning algorithms have been extended to multi-agent scenarios. Overestimation in value estimation of Q-learning is a significant issue that has been studied comprehensively in the single-agent domains, but rarely in multi-agent reinforcement learning. In this paper, we first demonstrate that Q-learning-based multi-agent reinforcement learning (MARL) methods generally have notably serious overestimation issues, which cannot be alleviated by current methods. To tackle this problem, we introduce the double critic networks structure and the delayed policy update to Q-learning-based multi-agent MARL methods, which reduce the overestimation and enhance the quality of policy updating. To demonstrate the versatility of our proposed method, we select several Q-learning based MARL methods and evaluate them on several multi-agent tasks on the multi-agent particle environment and SMAC. Experimental results demonstrate that the proposed method can avoid the overestimation problem and significantly improve performance. Besides, application in the Traffic Signal Control verifies the feasibility of applying the proposed method in real-world scenarios. [ABSTRACT FROM AUTHOR]

In this paper, we study the path-planning problem of emergency fire control robots in the nuclear environment. Given the high risk of the atomic environment, the irregularity of spatial shape, and the complex distribution of obstacles, a robot path planning method is proposed based on the combination of Q -learning and BCD raster map decomposition method. It realizes the automatic elimination control of the nuclear-contaminated environment and reduces the exposure risk of manual intervention operation. First, Q -learning, a reinforcement learning model, is used to establish the optimal path between the start and end points of the operation area. Second, the BCD raster map decomposition method is used to realize the global division of the operation area. Then, an improved partition merging method based on the Q -learning optimal path is proposed to complete the job sub-region merging and cover path planning. Finally, the simulation experiment proves that the technique can quickly and stably achieve the global path coverage of the unique operating environment of the nuclear domain. [ABSTRACT FROM AUTHOR]

This paper presents a least-mean-square-based (LMS-based) background calibration algorithm with reinforcement learning optimization to calibrate the capacitor mismatch in successive approximation-register (SAR) analog-to-digital converters (ADCs). When calibrating capacitor mismatch, the convergence speed and stability of the conventional LMS-based algorithm cannot be balanced due to the fixed iteration step size. To improve the calibration effect, nine different iteration step sizes are proposed for calibration. An iteration step size selection strategy is obtained by training the Q-learning algorithm with the conventional LMS-based calibration iteration process. Leading by this strategy, the proposed calibration algorithm performs better convergence speed and stability than the conventional LMS-based calibration algorithm. A 12-bit sub-radix-2 redundant SAR ADC is used to test our proposed calibration algorithm for better evaluation. Behavioral simulations show that the proposed calibration algorithm converges at 0.065 M sampling points, and the standard deviation of the effective number of bits (ENOB) in the convergence state is only 0.034-bit. [ABSTRACT FROM AUTHOR]

Summary: In this article, we propose an adaptive Kalman filtering based on Q‐learning for partial model‐free dynamic systems. First, a cost function is defined to iteratively update the prior state value when the model parameters are unknown. Then, the observations in a period of time are utilized to improve the accuracy and updating speed of the prior state estimation by means of the multi‐innovation least squares. Next, considering that the weight matrix in the cost function will change due to external noise noise and model mismatch, the innovation‐based adaptive estimation algorithm is presented to adjust the weight matrix by using the covariance of the information sequence. Finally, the proposed algorithms are applied to estimate the water level of a quadruple water tank system. [ABSTRACT FROM AUTHOR]

I characterize the outcomes of a class of model-free reinforcement learning algorithms, such as stateless Q-learning, in a prisoner's dilemma. The behavior is studied in the limit as players stop experimenting after sufficiently exploring their options. A closed form relationship between the learning rate and game payoffs reveals whether the players will learn to cooperate or defect. The findings have implications for algorithmic collusion and also apply to asymmetric learners with different experimentation rules. [ABSTRACT FROM AUTHOR]

This paper investigates the output feedback (OPFB) tracking control problem for discrete-time linear (DTL) systems with unknown dynamics. To solve this problem, we use an augmented system approach, which first transforms the tracking control problem into a regulation problem with a discounted performance function. The solution to this problem is derived using a Bellman equation, based on the Q-function. In order to overcome the challenges of unmeasurable system state variables, we employ a multistep Q-learning algorithm that surpasses the advantages of the policy iteration (PI) and value iteration (VI) techniques and state reconstruction methods for output feedback control. As such, the requirement for an initial stabilizing control policy for the PI method is removed and the convergence speed of the learning algorithm is improved. Finally, we demonstrate the effectiveness of the proposed scheme using a simulation example. [ABSTRACT FROM AUTHOR]

Research on dynamic treatment regimes has enticed extensive interest. Many methods have been proposed in the literature, which, however, are vulnerable to the presence of misclassification in covariates. In particular, although Q‐learning has received considerable attention, its applicability to data with misclassified covariates is unclear. In this article, we investigate how ignoring misclassification in binary covariates can impact the determination of optimal decision rules in randomized treatment settings, and demonstrate its deleterious effects on Q‐learning through empirical studies. We present two correction methods to address misclassification effects on Q‐learning. Numerical studies reveal that misclassification in covariates induces non‐negligible estimation bias and that the correction methods successfully ameliorate bias in parameter estimation. [ABSTRACT FROM AUTHOR]

StarCraft is a popular real-time strategy game that has been widely used as a research platform for artificial intelligence. Micromanagement refers to the process of making each unit perform appropriate actions separately, depending on the current state in the the multi-agent system comprising all of the units, i.e., the fine-grained control of individual units for common benefit. Therefore, cooperation between different units is crucially important to improve the joint strategy. We have selected multi-agent deep reinforcement learning to tackle the problem of micromanagement. In this paper, we propose a method for learning cooperative strategies in StarCraft based on role-based montonic value function factorization (RoMIX). RoMIX learns roles based on the potential impact of each agent on the multi-agent task; it then represents the action value of a role in a mixed way based on monotonic value function factorization. The final value is calculated by accumulating the action value of all roles. The role-based learning improves the cooperation between agents on the team, allowing them to learn the joint strategy more quickly and efficiently. In addition, RoMIX can also reduce storage resources to a certain extent. Experiments show that RoMIX can not only solve easy tasks, but it can also learn better cooperation strategies for more complex and difficult tasks. [ABSTRACT FROM AUTHOR]

Vulnerabilities such as SQL injection represent a serious challenge to security. While tools with a pre-defined logic are commonly used in the field of penetration testing, the continually evolving nature of the security challenge calls for models able to learn autonomously from experience. In this paper we build on previous results on the development of reinforcement learning models devised to exploit specific forms of SQL injection, and we design agents that are able to tackle a varied range of SQL injection vulnerabilities, virtually comprising all the archetypes normally considered by experts. We show that our agents, trained on a synthetic environment, perform a transfer of learning among the different SQL injections challenges; in particular, they learn to use their queries to efficiently gain knowledge about multiple vulnerabilities at once. We also introduce a novel and more versatile way to interpret server messages that reduces reliance on expert inputs. Our simulations show the feasibility of our approach which easily deals with a number of homogeneous challenges, as well as some of its limitations when presented with problems having higher degrees of uncertainty. [ABSTRACT FROM AUTHOR]

In accordance with the actual production circumstances of enterprises, a scheduling problem model is designed for open-shop environments, considering AGV transport time. A Q-learning-based method is proposed for the resolution of such problems. Based on the characteristics of the problem, a hybrid encoding approach combining process encoding and AGV encoding is applied. Three pairs of actions are constituted to form the action space. Decay factors and a greedy strategy are utilized to perturb the decision-making of the intelligent agent, preventing it from falling into local optima while simultaneously facilitating extensive exploration of the solution space. Finally, the proposed method proved to be effective in solving the open-shop scheduling problem considering AGV transport time through multiple comparative experiments. [ABSTRACT FROM AUTHOR]

Some protocols operated in the MAC layer and the open-source interconnections model to share the packet delivery and the network channel to deliver the packet is done. Transmitting two or more messages through one channel causes problems in packing and information delivery. This causes the lagging of the data to reach the destinations. So, the method of multiple hubs and the host must be created for the packet and the information delivery. This makes the fast transmission of data and messages. Method: The Tunnel monitoring system and the FANET are used for analysis in this study for the research, enabling the mobile radio networks for the examination. The FANET-based flying network system allows the paradigm for accessing human analysis. Then the drone-related data can be helped. The tunnel monitoring system navigates the information purpose and routing of the drone flying and gathering the data. Then based on human analysis, 40% of the data can be analyzed using the system's formation. Result: These results support the coverage and the tunnel monitoring process for detecting the navigation system. Conclusion: Also, some tunnel-based detection using drones is found in this study based on the wireless muometric navigation system that can be enabled using Q-learning. [ABSTRACT FROM AUTHOR]

Intelligent load scheduling for the smart city is essential due to the growth of industrialization, which directly consumes energy from the grid. The energy management system in industrialization is enhanced using the Internet of things in the same domain. The general need in a smart city is to maintain the energy utilization and cost that the industrial Internet of things could achieve for industry application. Researchers have also found a solution to this problem, but every time the generator cannot sell energy, the industry purchases the same. There is a chance for the industry to supply its energy to other industries or the grid. By keeping this constraint, an intelligent approach to load scheduling based on optimized Q-Net is proposed in this paper, comprising the supplies as renewable energy, battery, and industrial energy. Also, the reactive power flow of smart cities during scheduling is mitigated by classifying the energy available in the bus network. The proposed method scheduled demand power with 99% accuracy with the lowest cost of 13.7041$/KW for peak hr. scheduling with a demand of 6000KW/hr. [ABSTRACT FROM AUTHOR]

This paper demonstrates the potential of reinforcement learning techniques for the secure joint relay and jammer selection in the dual-hop wireless cooperative network. The existing techniques solve the optimization problem to find near-optimal secrecy. However, due to the computational complexity, most of these algorithms are not scalable to large networks. In this work, two model-free reinforcement learning-based secure joint relay and jammer selection techniques are developed. The objective is to maximize the physical layer security (PLS) of the dual-hop wireless network. The secure joint relay and jammer selection is transformed into a prediction-based problem and solved using reinforcement learning. We introduced Q-learning-based joint relay and jammer selection (QL-JRJS) and deep Q-learning-based joint relay and jammer selection (DQL-JRJS) algorithms. The secrecy performance of the introduced algorithms is compared with the existing techniques. The simulation results demonstrated that the proposed DQL-JRJS algorithm on average achieves 38.7%, 32.1%, and 21.6% less secrecy outage probability (SOP) as compared to the greedy method (GM), vector alignment method (VAM), and QL-JRJS algorithm. However, QL-JRJS only mimics the performance of DQL-JRJS for small values of state action pairs. The proposed algorithms are best suited for situations where the exact global channel state information (CSI) for all the links is unavailable. [ABSTRACT FROM AUTHOR]