Search

Artificial intelligence (AI) has the potential to simplify and optimize various steps of the brachytherapy workflow, and this literature review aims to provide an overview of the work done in this field. We conducted a literature search in June 2022 on PubMed, Embase, and Cochrane for papers that proposed AI applications in brachytherapy. A total of 80 papers satisfied inclusion/exclusion criteria. These papers were categorized as follows: segmentation (24), registration and image processing (6), preplanning (13), dose prediction and treatment planning (11), applicator/catheter/needle reconstruction (16), and quality assurance (10). AI techniques ranged from classical models such as support vector machines and decision tree-based learning to newer techniques such as U-Net and deep reinforcement learning, and were applied to facilitate small steps of a process (e.g., optimizing applicator selection) or even automate the entire step of the workflow (e.g., end-to-end preplanning). Many of these algorithms demonstrated human-level performance and offer significant improvements in speed. AI has potential to augment, automate, and/or accelerate many steps of the brachytherapy workflow. We recommend that future studies adhere to standard reporting guidelines. We also stress the importance of using larger sample sizes and reporting results using clinically interpretable measures. [ABSTRACT FROM AUTHOR]

The increasing integration of distributed energy resources has led to heightened complexity in distribution network models, posing challenges of uncertainty and volatility to the operation and control of distribution networks. Simultaneously, the widespread deployment of metering and sensing devices has furnished data-driven approaches with copious data sources. As a potent tool for data mining, artificial intelligence (AI) algorithms have the capacity to harness extensive datasets, thus furnishing comprehensive energy services for distribution grids. This paper provides a comprehensive exposition of AI techniques applied in the realm of distribution network voltage control (DNVC). Initially, a brief overview is provided for the four voltage control frameworks and the data calibration of DNVC, serving as the foundation for various voltage control algorithms. Subsequently, in ascending order of algorithmic advancement, the application of improved heuristic algorithms, deep learning (DL) algorithms structured around artificial neural networks (ANNs), and deep reinforcement learning (DRL) algorithms in the DNVC domain are sequentially introduced, and the relationships among the three AI technologies in framework, control principles, and performance are systematically analyzed. Additionally, attention is directed toward the algorithmic progress made in addressing specific challenges within the DNVC process. Lastly, building upon extant research, the paper summarizes four prominent developmental directions for AI in DNVC: safety exploration, information privacy, multi-feeder collaboration, and heightened uncertainty. Corresponding solutions to each of these directions are individually discussed. • The research status and applications of mainly used AI methods in distribution network voltage control is reviewed. • The relationships among the three artificial intelligence methods are systematically analyzed. • The existing algorithms, along with their strengths, limitations, and relationshipsare highlighted. • The research directions of artificial intelligence methods in distribution network voltage control are discussed in detail. [ABSTRACT FROM AUTHOR]

The past decade has witnessed the rapid development and adoption of machine and deep learning (ML & DL) methodologies in agricultural systems, showcased by great successes in applications such as smart crop management, smart plant breeding, smart livestock farming, precision aquaculture farming, and agricultural robotics. However, these conventional ML/DL models have certain limitations: they heavily rely on large, costly-to-acquire labeled datasets for training, require specialized expertise for development and maintenance, and are mostly tailored for specific tasks, thus lacking generalizability. Recently, large pre-trained models, also known as foundation models (FMs), have demonstrated remarkable successes in language, vision, and decision-making tasks across various domains. These models are trained on a vast amount of data from multiple domains and modalities. Once trained, they can accomplish versatile tasks with just minor fine-tuning and minimal task-specific labeled data. Despite their proven effectiveness and huge potential, there has been little exploration of applying FMs to agriculture artificial intelligence (AI). Therefore, this study aims to explore the potential of FMs in the field of smart agriculture. In particular, conceptual tools and technical background are presented to facilitate the understanding of the problem space and uncover new research directions in this field. To this end, recent FMs in the general computer science (CS) domain are reviewed, and the models are categorized into four categories: language FMs, vision FMs, multimodal FMs, and reinforcement learning FMs. Subsequently, the process of developing agriculture FMs (AFMs) is outlined and their potential applications in smart agriculture are discussed. In addition, the unique challenges and risks associated with developing AFMs are discussed, including model training, validation, and deployment. Through this study, the advancement of AI in agriculture is explored by introducing AFMs as a promising paradigm that can significantly mitigate the reliance on extensive labeled datasets and enhance the efficiency, effectiveness, and generalization of agricultural AI systems. To facilitate further research, a well-classified and actively updated list of papers on AFMs is organized and accessible at https://github.com/JiajiaLi04/Agriculture-Foundation-Models. • Basics of large language and foundation models. • Review of potential applications of large language and foundation models in agriculture. • Outline challenges and opportunities. [ABSTRACT FROM AUTHOR]

• Proposing dynamic-scale conservation voltage reduction (DCVR) to solve technical issues of autonomous microgrids (AMGs). • AI-based deep reinforcement learning (DRL) technique is proposed for implementing DCVR. • Multi-agent DRL structure has been designed, and the agents are trained to realize decentralized DCVR. • Accurate Q-sharing is preserved while implementing the DCVR. • DCVR addresses the dynamic instabilities of AMGs by improving the primary frequency response. Conservation voltage reduction (CVR) is implemented in power systems to mitigate power consumption in steady-state time scales. We propose dynamic-scale CVR (DCVR) as a potent solution to provide cost-effective frequency support in inverter-interfaced (micro) grids. The proposed DCVR reduces the voltage profile in dynamics and consequently power reduction helps to maintain instant production-consumption balance for dynamic frequency support. However, DCVR implementation in autonomous MGs (AMGs) faces critical control and stability problems. DCVR is implemented by controlling the voltage, which is a local variable, and makes it difficult to be realized through a decentralized structure. Besides, preserving accurate reactive power sharing (Q-sharing) through conventional droop controllers while employing the DCVR is another critical concern. In this light, this paper proposes a novel artificial intelligence (AI)-based decentralized control structure to implement the DCVR in AMGs for tackling the existing issues. The multi-agent deep reinforcement learning (DRL) model, with a deep Q-network (DQN) algorithm, is adopted to address the grid instabilities and inaccurate Q-sharing issues that arise due to incorporating the DCVR. The proposed method is able to handle the nonlinearity and complexity of the system while maintaining proper dynamic performance and AMG stability. Simulation results in MATLAB/Simulink prove the effectiveness of the proposed control method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel graph-based semi-supervised learning approach for traffic light management in multiple intersections. Specifically, the basic premise behind our paper is that if we know some of the occupied roads and predict which roads will be congested, we can dynamically change traffic lights at the intersections that are connected to the roads anticipated to be congested. Comparative performance evaluations show that the proposed approach can produce comparable average vehicle waiting time and reduce the training/learning time of learning adequate traffic light configurations for all intersections within a few seconds, while a deep learning-based approach can be trained in a few days for learning similar light configurations. • With this work we aimed to optimize traffic signal systems by dynamically setting the system to behave as per predicted traffic flow. • Vehicle flow is predicted using Graph-based semi-supervised and active learning on edge flows. • The systems were created and tested using Simulation Of Urban Mobility (SUMO). • Vehicle data and routes were also generated using this application. • The proposed approach was compared against other methods, such as an occupancy based method, a scoring-based method and reinforcement learning. [ABSTRACT FROM AUTHOR]

Artificial intelligence and all its supporting tools, e.g. machine and deep learning in computational intelligence-based systems, are rebuilding our society (economy, education, life-style, etc.) and promising a new era for the social welfare state. In this paper we summarize recent advances in data science and artificial intelligence within the interplay between natural and artificial computation. A review of recent works published in the latter field and the state the art are summarized in a comprehensive and self-contained way to provide a baseline framework for the international community in artificial intelligence. Moreover, this paper aims to provide a complete analysis and some relevant discussions of the current trends and insights within several theoretical and application fields covered in the essay, from theoretical models in artificial intelligence and machine learning to the most prospective applications in robotics, neuroscience, brain computer interfaces, medicine and society, in general. [ABSTRACT FROM AUTHOR]

The increasing complexity of power systems makes existing deep reinforcement learning-based emergency voltage control methods face challenges in learning speed and data utilization efficiency. Meanwhile, the accumulated data containing expert experience and domain knowledge has not been fully utilized to improve the performance of the deep reinforcement learning methods. To address the above issues, a novel hybrid emergency voltage control method that combines expert experience and machine intelligence is proposed in this paper. Specifically, the expert experience in the off-line demonstration is extracted through a behavioral cloning model and the deep reinforcement learning method is applied to discover and learn new knowledge autonomously. A special supervised expert loss is designed to utilize the pre-trained behavioral cloning model to assist the self-learning process. The demonstration is dynamically updated during the training process such that the behavioral cloning model and the deep reinforcement learning model can facilitate each other continuously. Experiments are conducted on the open-source RLGC platform to validate the performance and the experimental results show that the proposed method can effectively improve the learning speed and the applicability of the model to different test situations. [ABSTRACT FROM AUTHOR]

This work develops a deep reinforcement learning based approach for Six Degree-of-Freedom (DOF) planetary powered descent and landing. Future Mars missions will require advanced guidance, navigation, and control algorithms for the powered descent phase to target specific surface locations and achieve pinpoint accuracy (landing error ellipse <5 m radius). This requires both a navigation system capable of estimating the lander's state in real-time and a guidance and control system that can map the estimated lander state to a commanded thrust for each lander engine. In this paper, we present a novel integrated guidance and control algorithm designed by applying the principles of reinforcement learning theory. The latter is used to learn a policy mapping the lander's estimated state directly to a commanded thrust for each engine, resulting in accurate and almost fuel-optimal trajectories over a realistic deployment ellipse. Specifically, we use proximal policy optimization, a policy gradient method, to learn the policy. Another contribution of this paper is the use of different discount rates for terminal and shaping rewards, which significantly enhances optimization performance. We present simulation results demonstrating the guidance and control system's performance in a 6-DOF simulation environment and demonstrate robustness to noise and system parameter uncertainty. [ABSTRACT FROM AUTHOR]

Graphical abstract Abstract In this paper, artificial Intelligence (AI) algorithm is used in the Rietveld refinement process instead of the human decision. The program, PowderBot, is developed based on Fullprof engine, which proves the effectiveness of AI in Rietveld refinement. In this program, the decision making in refinement process is modelled as a Markov decision process (MDP), and solved by a reinforcement learning algorithm. PowderBot is designed to be a self-learning system capable of conducting structure refinement without human intervention. The program has already been successfully applied to Rietveld refinements. We hope this paper will encourage more Rietveld programs become more intelligent by the help of AI algorithm. [ABSTRACT FROM AUTHOR]

• Cover all FSL papers from the 2000s to now, elaborate its evolution history. • Give an understandable hierarchical taxonomy for FSL approaches. • Categorize meta learning FSL methods and reveal their underlying relationship. • Analyze several emerging extensional research hotspots of FSL. • Conclude existing FSL applications and its future directions. The capability of learning and generalizing from very few samples successfully is a noticeable demarcation separating artificial intelligence and human intelligence. Despite the long history dated back to the early 2000s and the widespread attention in recent years with booming deep learning, few surveys for few sample learning (FSL) are available. We extensively study almost all papers of FSL spanning from the 2000s to now and provide a timely and comprehensive survey for FSL. In this survey, we review the evolution history and current progress on FSL, categorize FSL approaches into the generative model based and discriminative model based kinds in principle, and emphasize particularly on the meta learning based FSL approaches. We also summarize several recently emerging extensional topics of FSL and review their latest advances. Furthermore, we highlight the important FSL applications covering many research hotspots in computer vision, natural language processing, audio and speech, reinforcement learning and robotic, data analysis, etc. Finally, we conclude the survey with a discussion on promising trends in the hope of providing guidance and insights to follow-up researches. [ABSTRACT FROM AUTHOR]

Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%. • A novel lifelong learning architecture for resource allocation in heterogeneous IIoT. • Dual-attention deep reinforcement learning provides local- and global- feature fusion. • Federated meta learning based knowledge fusion learns the commonness of models. • Reinforcement learning based knowledge transfer learns the properties of model. • Knowledge fusion and knowledge transfer provide lifelong learning ability for system. [ABSTRACT FROM AUTHOR]

Home automation is seen as a potential pillar of the smart city revolution that combines smart mobility, lifestyle and ecosystem governed by intelligent sensors connected to the internet. Households can save money and be more comfortable with automated appliances. The cost of electricity and user comfort are fundamentally contradictory, so they can be presented as a dynamic multi-objective optimization problem with fluctuating priorities for the customer to use various devices at different times. For this reason, this paper proposes an advanced Intelligent Home Energy Management (IHEM) approach based on reinforcement learning to achieve home demand response (DR) efficiency. The optimal formulation of the one-hour-ahead energy consumption scheduling problem is considered a Markov Decision Process (MDP) with discrete time steps. An efficient Neural Network (NN)-based approach with a Q-learning algorithm is developed to address this problem, enabling the IHEM system to achieve better cost-effective scheduling performance. The accurate data of electricity price and energy supplied by the Photovoltaic (PV) system are analyzed in sliding periods by machine learning for uncertainty prediction. Using the newly developed approach, which has the dual objective of minimizing the electricity bill, it is possible to obtain scheduling decisions for appliances and energy storage. The results show that the proposed optimization method reduces the monthly electricity costs by 20% compared to the Integer Linear Programming (ILP)-based HEMS method. • Artificial intelligence (AI) approach to monitor household energy consumption. • An efficient deep reinforcement learning algorithm to control activity recognition in smart homes. • Design and implementation of Intelligent home energy management system. • Converge promptly and considerably reduce operating expenses adopting AI-based approach. [ABSTRACT FROM AUTHOR]

The past decade has witnessed the rapid development of Artificial Intelligence (AI), especially the explosion of deep learning-related connectionist approaches. This study combines traditional literature review, bibliometric methods, and the Science of Science (SciSci) theory to scrutinize the development context of AI in the last decade on AMiner. 4 4 www.aminer.cn , AMiner is an academic mining system, which will be introduced in the following section. With the assistance of AMiner tools and datasets, this paper aims to describe a further explicit context and evolution of AI in the past decade from the development of connectionist approaches. Five aspects of the past decade are highlighted: self-learning and self-coding algorithms, Recurrent Neural Networks (RNN) algorithms, reinforcement learning, pre-trained models, and other typical deep learning algorithms, which represent the significant progress of this field. By combining these critical parts, we then summarize the current limitations and corresponding future of AI trends in the next decade and discuss some topics about the next generation of AI. Discoveries in this paper will benefit AI research in promoting understanding of the current critical stage and future trends of AI development and the AI industry in the dramatic ascendant for the academic research results transformation and its industrial layout. [ABSTRACT FROM AUTHOR]

Rotating machinery fault can cause accidents like loss of flow or turbine trip that seriously threaten the operation safety of nuclear power plants (NPPs). Artificial intelligence algorithms, like machine learning or deep learning methods, can implement fault diagnosis by sample learning with no reliance on fault mechanism or physics model of the equipment. However, the accumulated fault samples are small due to high operation safety requirements of the plant. Small sample learning is challenging and usually leads to degradation of model performance. The emerging deep reinforcement learning (DRL) algorithm can incorporate the advantages of automatic feature extraction from deep learning algorithm and interactive learning from reinforcement learning algorithm, is expected to have better learning ability and robustness. In this paper, two DRL fault diagnosis models are proposed and compared. Experiment results show that the proposed models can achieve very high diagnosis accuracy of over 99% and outperform all the baseline models (support vector machine, convolutional neural network and gated recurrent unit neural network) in all test cases in this paper. • Fault diagnosis of rotating machinery is important for nuclear power plants. • Two deep reinforcement learning-based fault diagnosis models are proposed. • Experiment analysis verify the effectiveness of the proposed models. • Excellent diagnostic accuracy of over 99% is achieved. • The proposed models outperform all baseline models in all test cases. [ABSTRACT FROM AUTHOR]

In this paper, a deep reinforcement learning (DRL)-based controller for path following of an unmanned surface vehicle (USV) is proposed. The proposed controller can self-develop a vehicle's path following capability by interacting with the nearby environment. A deep deterministic policy gradient (DDPG) algorithm, which is an actor-critic-based reinforcement learning algorithm, was adapted to capture the USV's experience during the path-following trials. A Markov decision process model, which includes the state, action, and reward formulation, specially designed for the USV path-following problem is suggested. The control policy was trained with repeated trials of path-following simulation. The proposed method's path-following and self-learning capabilities were validated through USV simulation and a free-running test of the full-scale USV. • A deep reinforcement learning-based controller for path following of an unmanned surface vehicle is proposed. • Using the proposed method, control policy for USV path following is trained with USV simulation. • The trained control policy is validated through USV free-running test of the full-scale USV. [ABSTRACT FROM AUTHOR]

Predicting electricity demand data is considered an essential task in decisions taking, and establishing new infrastructure in the power generation network. To deliver a high-quality electricity demand prediction, this paper proposes a hybrid combination technique, based on a deep learning model of Convolutional Neural Networks and Echo State Networks, named as CESN. Daily electricity demand data from four sites (Roderick, Rocklea, Hemmant and Carpendale), located in Southeast Queensland, Australia, have been used to develop the proposed hybrid prediction model. The study also analyzes five other machine learning-based models (support vector regression, multilayer perceptron, extreme gradient boosting, deep neural network, and Light Gradient Boosting) to compare and evaluate the outcomes of the proposed deep learning approach. The results obtained in the experimental study showed that the proposed hybrid deep learning model is able to obtain the highest performance compared to other existing models developed for daily electricity demand data forecasting. Based on the statistical approaches utilized in this study, the proposed hybrid approach presents the highest prediction accuracy among the compared models. The obtained results showed that the proposed hybrid deep learning algorithm is an excellent and accurate electricity demand forecasting method, which outperformed the state of the art algorithms that are currently used in this problem. [Display omitted] • A Hybrid deep learning model is proposed for daily electricity demand prediction. • Convolutional Neural Networks and Echo State Networks are integrated into a single forecasting algorithm. • The proposed deep learning-based approach is excellent in electricity demand prediction. • The proposed algorithm has practical use in inter-daily electricity demand prediction. [ABSTRACT FROM AUTHOR]

How to design System of Systems has been widely concerned in recent years, especially in military applications. This problem is also known as SoS architecting, which can be boiled down to two subproblems: selecting a number of systems from a set of candidates and specifying the tasks to be completed for each selected system. Essentially, such a problem can be reduced to a combinatorial optimization problem. Traditional exact solvers such as branch-bound algorithm are not efficient enough to deal with large scale cases. Heuristic algorithms are more scalable, but if input changes, these algorithms have to restart the searching process. Re-searching process may take a long time and interfere with the mission achievement of SoS in highly dynamic scenarios, e.g., in the Mosaic Warfare. In this paper, we combine artificial intelligence with SoS architecting and propose a deep reinforcement learning approach DRL-SoSDP for SoS design. Deep neural networks and actor–critic algorithms are used to find the optimal solution with constraints. Evaluation results show that the proposed approach is superior to heuristic algorithms in both solution quality and computation time, especially in large scale cases. DRL-SoSDP can find great solutions in a near real-time manner, showing great potential for cases that require an instant reply. DRL-SoSDP also shows good generalization ability and can find better results than heuristic algorithms even when the scale of SoS is much larger than that in training data. [ABSTRACT FROM AUTHOR]

Schank (1980) wrote an editorial for Intelligence on "How much intelligence is there in artificial intelligence?". In this paper, we revisit this question. We start with a short overview of modern AI and showcase some of the AI breakthroughs in the four decades since Schank's paper. We follow with a description of the main techniques these AI breakthroughs were based upon, such as deep learning and reinforcement learning; two techniques that have deep roots in psychology. Next, we discuss how psychologically plausible AI is and could become given the modern breakthroughs in AI's ability to learn. We then access the main question of how intelligent AI systems actually are. For example, are there AI systems that can solve human intelligence tests? We conclude that Shank's observation, that intelligence is all about generalization and that AI is not particularly good at this, has, so far, withstood the test of time. Finally, we consider what AI insights could mean for the study of individual differences in intelligence. We close with how AI can further Intelligence research and vice versa, and look forward to fruitful interactions in the future. • Recent AI breakthroughs, such as deep learning and reinforcement learning, have deep roots in psychology. • Modern AI models are much more human and brain like at the implementational level. • There is nothing wrong with AI's crystallized intelligence, but generalization is still a weakness of AI systems. • The psychological relevance of AI extends to areas such as the study of individual differences and cognitive development. • We expect fruitful interactions with regard to the measurement of natural and artificial intelligence. [ABSTRACT FROM AUTHOR]

• Reinforcement learning (RL) formalization of the algorithmic trading problem. • Novel trading strategy based on deep reinforcement learning (DRL), denominated TDQN. • Rigorous performance assessment methodology for algorithmic trading. • TDQN algorithm delivers promising results surpassing benchmark strategies. This scientific research paper presents an innovative approach based on deep reinforcement learning (DRL) to solve the algorithmic trading problem of determining the optimal trading position at any point in time during a trading activity in the stock market. It proposes a novel DRL trading policy so as to maximise the resulting Sharpe ratio performance indicator on a broad range of stock markets. Denominated the Trading Deep Q-Network algorithm (TDQN), this new DRL approach is inspired from the popular DQN algorithm and significantly adapted to the specific algorithmic trading problem at hand. The training of the resulting reinforcement learning (RL) agent is entirely based on the generation of artificial trajectories from a limited set of stock market historical data. In order to objectively assess the performance of trading strategies, the research paper also proposes a novel, more rigorous performance assessment methodology. Following this new performance assessment approach, promising results are reported for the TDQN algorithm. [ABSTRACT FROM AUTHOR]

Improvement of traffic signal control (TSC) efficiency has been found to lead to improved urban transportation and enhanced quality of life. Recently, the use of reinforcement learning (RL) in various areas of TSC has gained significant traction; thus, we conducted a systematic literature review as a systematic, comprehensive, and reproducible review to dissect all the existing research that applied RL in the network-level TSC domain, called as RL in NTSC or RL-NTSC for brevity. The review only targeted the network-level articles that tested the proposed methods in networks with two or more intersections. This review covers 160 peer-reviewed articles from 30 countries published from 1994 to March 2020. The goal of this study is to provide the research community with statistical and conceptual knowledge, summarize existence evidence, characterize RL applications in NTSC domains, explore all applied methods and major first events in the defined scope, and identify areas for further research based on the explored research problems in current research. We analyzed the extracted data from the included articles in the following seven categories: (i) publication and authors' data, (ii) method identification and analysis, (iii) environment attributes and traffic simulation, (iv) application domains of RL-NTSC, (v) major first events of RL-NTSC and authors' key statements, (vi) code availability, and (vii) evaluation. This paper provides a comprehensive view of the past 26 years of research on applying RL to NTSC. It also reveals the role of advancing deep learning methods in the revival of the research area, the rise of using non-commercial microscopic traffic simulators, a lack of interaction between traffic and transportation engineering practitioners and researchers, and a lack of proposal and creation of testbeds which can likely bring different communities together around common goals. • A review on Reinforcement Learning in the network-scale Traffic Signal Control area. • Presents a comprehensive systematic literature review of 160 included articles. • Consolidates and characterizes the existing research on the defined area. • Explores the methods, applications, domains, and first events in the defined scope. • Identifies past and present trends and directions for further research in the area. [ABSTRACT FROM AUTHOR]

Precise prescription of medication dosing is crucial to patients, especially among Intensive Care Unit (ICU) patients. However, improper administration of some sensitive therapeutic medications (e.g., heparin) might place patients at unneeded risk, even life-threatening. Numerous factors such as a patient's clinical phenotype, genotype, and environmental factors will affect the heparin dose response. As a result, it is challenging to prescribe the optimal initial dose of heparin. In this paper, an individualized dosing policy is proposed to determine the optimal initial dose and minimize the risk of mis-dosing, as well as preventing the patients from late complications associated with medications dosing. A latent batch-constrained deep reinforcement learning (RL) algorithm is proposed to guarantee the safety of the medication recommendation system. The agent can observe a latent representation of patents and generate medication dosing solutions in successive and limited action spaces. The individualized dosing policy aims to reduce the extrapolation errors in the off-policy algorithms, by evaluating over-dosing and under-dosing of heparin in patients. Our results evaluated on Medical Information Mart for Intensive Care III (MIMIC-III) database demonstrate that the latent batch-constrained RL algorithm can work effectively from the retrospective data, showing promise to be used in future medication dosing policies. [ABSTRACT FROM AUTHOR]

Water scarcity is a permanent problem that faces all over the world. Artificial Intelligence (AI) has many machine learning methods used to solve many problems in all fields. This paper suggests a novel and efficient approach to finding a trans-membrane pressure using Deep Reinforcement Learning (DRL). Our system uses Deep Deterministic Policy Gradient (DDPG) agent to adjust the pressure across the membrane. This adjustment considers the Salt Rejection (SR) to be 99% to investigate the desired water flux. The system takes the maximum height of the water in the tank (h max), the salt concentration of feed flow (C), the temperature of feed flow (T), the recovery ratio (R), and the salt rejection ratio (SR) as input, and returns the water flux Q p. The results show the effectiveness and the power of the DDPG agent in finding that pressure. The agent is trained in a small number of episodes (150), and the average reward value is high. [ABSTRACT FROM AUTHOR]

: Wearable technologies have added completely new and fast emerging tools to the popular field of personal gadgets. Aside from being fashionable and equipped with advanced hardware technologies such as communication modules and networking, wearable devices have the potential to fuel artificial intelligence (AI) methods with a wide range of valuable data. : Various AI techniques such as supervised, unsupervised, semi-supervised and reinforcement learning (RL) have already been used to carry out various tasks. This paper reviews the recent applications of wearables that have leveraged AI to achieve their objectives. : Particular example applications of supervised and unsupervised learning for medical diagnosis are reviewed. Moreover, examples combining the internet of things, wearables, and RL are reviewed. Application examples of wearables will be also presented for specific domains such as medical, industrial, and sport. Medical applications include fitness, movement disorder, mental health, etc. Industrial applications include employee performance improvement with the aid of wearables. Sport applications are all about providing better user experience during workout sessions or professional gameplays. : The most important challenges regarding design and development of wearable devices and the computation burden of using AI methods are presented. Finally, future challenges and opportunities for wearable devices are presented. • Applications of artificial intelligence methods such as supervised, unsupervised, etc. in wearable devices are reviewed. • Different types of wearable devices are introduced. • The body parts on which the wearable devises are installed are investigated. • Distribution of existing wearable devices based on target body parts and their applications is presented. • Comprehensive treatment of domain-specific applications of wearable devices such as sports, healthcare, industrial and manufacturing is provided. • Wearable technologies challenges such as data collection, data transmission, etc. are pointed out. • Future directions regarding application of artificial intelligence methods in wearable devices are discussed. [ABSTRACT FROM AUTHOR]

The combinatorial optimization (CO) problems on the graph are the core and classic problems in artificial intelligence (AI) and operations research (OR). For example, the Vehicle Routing Problem (VRP) and Traveling Salesman Problem (TSP) are fascinating NP-hard problems and have important significance for the existing transportation system. Traditional methods such as heuristics methods, exact algorithms, and solution solvers can already find approximate solutions on small-scale graphs. However, they are helpless for large-scale graphs and other problems with similar structures. Moreover, traditional methods often require artificially designed heuristic functions to aid decision-making. In recent years, more and more work has focused on applying deep learning and reinforcement learning (RL) to learn heuristics, which allows us to learn the internal structure of the graph end-to-end and find the optimal path under the guidance of heuristic rules. However, most of these still need manual assistance, and the RL method used has the problems of low sampling efficiency and small searchable space. This paper proposes a novel framework (called OmegaZero) based on Alphago Zero, which does not prescribe expert experience or label data but is trained through self-play. We divide the learning into two stages: in the first stage, we employ graph attention network (GAT) and GRU to learn node representations and memory history trajectories. In the second stage, we employ Monte Carlo tree search (MCTS) and deep RL to search for the solution space and train the model. [ABSTRACT FROM AUTHOR]

Artificial intelligence is steadily increasing its impact on everyday life. Therefore, the societal issues of artificial intelligence have become an important concern in the AI research. The presence of data that reflects human biases towards historically discriminated groups defined by sensitive features such as race and gender, results in machine learning models which discriminate against these groups. In order to tackle the impact of bias in data, researchers developed a variety of specialized machine learning algorithms which are able to satisfy different fairness constraints imposed on the model. Group fairness constraints do not fit standard machine learning formulations easily due to their non-differentiable nature. In this paper we developed a technique for learning a fair classifier by Monte Carlo policy gradient method which naturally deals with such non-differentiable constraints. Our methodology focuses on direct optimization of both group fairness metric and predictive performance of the model. In addition, we propose two different variance reduction techniques of gradient estimation. We compare our models to seven other related and state-of-the-art models and demonstrate that they are able to achieve better trade-off between accuracy and unfairness. To the best of our knowledge, this is the first fair classification algorithm which solves the issue of non-differentiable constraints by reinforcement learning techniques. [ABSTRACT FROM AUTHOR]

The Deep Reinforcement Learning (DRL) algorithm, Proximal Policy Optimization (PPO2), is deployed on a custom spacecraft (S/C) build and loss model to determine if an Artificial Intelligence (AI) can learn to monitor satellite constellation health and determine an optimal replacement strategy. A custom environment is created to simulate how S/C are built, launched, generate revenue, and finally decay. The reinforcement learning agent successfully learned an optimal policy for two models: a Simplified Model where the financial cost of actions is ignored; and an Advanced Model where the financial cost of actions is a major element. In both models the AI monitors the constellations and takes multiple strategic and tactical actions to replace satellites to maintain constellation performance. The Simplified Model showed that the PPO2 algorithm was able to converge on an optimal solution after ∼ 200,000 simulations. The Advanced Model was much more difficult for the AI to learn, and thus, the performance drops during the early episodes, but eventually converges to an optimal policy at ∼ 25,000,000 simulations. With the Advanced Model, the AI is taking actions that are successfully providing strategies for constellation management and satellite replacements which include these actions' financial implications. Thus, the methods in this paper provide initial research developments towards a real-world tool and an AI application that can aid various Aerospace businesses in managing Low Earth Orbit (LEO) constellations. This type of AI application may become imperative for deploying and maintaining small satellite mega-constellations. [ABSTRACT FROM AUTHOR]

• The multi-agent defense and attack environment is reconstructed. • Several algorithms are applied to solve the considered problem. • We redefine the state space, the action space and the reward functions accordingly. • Comparison experiments are conducted to show the performance of the employed models. Learning to cooperate among agents has always been an important research topic in artificial intelligence. Multi-agent defense and attack, one of the important issues in multi-agent cooperation, requires multiple agents in the environment to learn effective strategies to achieve their goals. Deep reinforcement learning (DRL) algorithms have natural advantages dealing with continuous control problems especially under situations with dynamic interactions, and have provided new solutions for those long-studied multi-agent cooperation problems. In this paper, we start from deep deterministic policy gradient (DDPG) algorithm and then introduce multi-agent DDPG (MADDPG) to solve the multi-agent defense and attack problem under different situations. We reconstruct the considered environment, redefine the continuous state space, continuous action space, reward functions accordingly, and then apply deep reinforcement learning algorithms to obtain effective decision strategies. Several experiments considering different confrontation scenarios are conducted to validate the feasibility and effectiveness of the DRL-based methods. Experimental results show that through learning the agents can make better decisions, and learning with MADDPG achieves superior performance than learning with other DRL-based models, which also explains the importance and necessity of mastering other agents' information. [ABSTRACT FROM AUTHOR]

Future ocean exploration will be dominated by a large-scale deployment of marine robots such as unmanned surface vehicles (USVs). Without the involvement of human operators, USVs exploit oceans, especially the complex marine environments, in an unprecedented way with an increased mission efficiency. However, current autonomy level of USVs is still limited, and the majority of vessels are being remotely controlled. To address such an issue, artificial intelligence (AI) such as reinforcement learning can effectively equip USVs with high-level intelligence and consequently achieve full autonomous operation. Also, by adopting the concept of multi-agent intelligence, future trend of USV operations is to use them as a formation fleet. Current researches in USV formation control are largely based upon classical control theories such as PID, backstepping and model predictive control methods with the impact by using advanced AI technologies unclear. This paper, therefore, paves the way in this area by proposing a distributed deep reinforcement learning algorithm for USV formations. More importantly, using the proposed algorithm USV formations can learn two critical abilities, i.e. adaptability and extendibility that enable formations to arbitrarily increase the number of USVs or change formation shapes. The effectiveness of algorithms has been verified and validated through a number of computer-based simulations. [ABSTRACT FROM AUTHOR]

• Online network reconfiguration is introduced to manage distribution networks. • A Deep Q-learning algorithm is used to find the optimal network topology. • The proposed algorithm is more scalable than other algorithms. • The computation time of the algorithm is low enough for practical applications. • The algorithm can increase DRES utilization without heavy capital investment. With increasing number of distributed renewable energy sources integrated in power distribution networks, network security issues such as line overloading or bus voltage violations are becoming increasingly common. Traditional capital-intensive system reinforcements could lead to overinvestment. Moreover, active network management solutions, which have emerged as important alternatives, may become a financial burden for distribution system operators or reduce profits for owners of distributed renewable energy sources, or both. To address these limitations, this paper proposes an online network reconfiguration scheme based on a deep reinforcement learning approach. In this scheme, the distribution network operator modifies the network topology to change the power flow when the reliability of network is threatened. Because the variability of distributed renewable energy is large in self-sufficient distribution networks, the reconfiguration process needs to be performed online within short time intervals, which involves the use of conventional algorithms. To solve this problem efficiently, a deep q-learning model is utilized to determine the optimal network topology. Performances of proposed and other algorithms were compared in modified CIGRE 14-bus and IEEE 123-bus test network, as well as varying penalties for frequent switching operation in consideration of physical characteristic of the network. Simulation results demonstrated that the proposed algorithm showed almost identical performances with brute-force search algorithm in both test networks, satisfying network constraints over almost all timespans. Further, the proposed method required very small computation times - under a second per each state and its scalability was verified by comparing the computation time between two test networks. [ABSTRACT FROM AUTHOR]

• DRL-based control methods are proposed for ORC for engine waste heat recovery. • The DRL control for ORC superheat performs considerably better than PID control. • The DRL-based PID control is more robust than the DRL control. • The switching DRL control performs well and exhibits sufficient robustness. • Useful reference and motivation for application of DRL to thermodynamic systems. The organic Rankine cycle (ORC) is a promising technology for engine waste heat recovery. During real-world operation, the engine working condition varies frequently to satisfy the power demand; thus, the transient nature of engine waste heat presents significant control challenges for the ORC. To control the superheat of the ORC precisely under a transient heat source, several optimal control methods have been used such as model predictive control and dynamic programing. However, most of them depend strongly on the accurate prediction of future disturbances. Deep reinforcement learning (DRL) is an artificial-intelligence algorithm that can overcome the aforementioned disadvantage, but the potential of DRL in control of thermodynamic systems has not yet been investigated. Thus, this paper proposes two DRL-based control methods for controlling the superheat of ORC under a transient heat source. One directly uses the DRL agent to learn the control strategy (DRL control), and the other uses the DRL agent to optimize the parameters of the proportional–integral–derivative (PID) controller (DRL-based PID control). Additionally, a switching mechanism between different DRL controllers is proposed for improving the training efficiency and enlarging the operation range of the controller. The results of this study indicate that the DRL agent can satisfactorily perform the control task and optimize the traditional controller under the trained and untrained transient heat source. Specifically, the DRL control can track the reference superheat with an average error of only 0.19 K, whereas that of the traditional PID control is 2.16 K. Furthermore, the proposed switching DRL control exhibits excellent tracking performance with an average error of only 0.21 K and robustness over a wide range of operation conditions. The successful application of DRL demonstrates its considerable potential for the control of thermodynamic systems, providing a useful reference and motivation for the application to other thermodynamic systems. [ABSTRACT FROM AUTHOR]

With advances in smart grid technologies, demand response has played a major role in improving the reliability of grids and reduce the cost for customers. Implementing the demand response scheme for industry is more necessary than for other sectors, because its energy consumption is often considered the largest. This paper proposes a multi-agent deep reinforcement learning based demand response scheme for energy management of discrete manufacturing systems. In this regard, the industrial manufacturing system is initially formulated as a partially-observable Markov game; then, a multi-agent deep deterministic policy gradient algorithm is adopted to obtain the optimal schedule for different machines. A typical lithium-ion battery assembly manufacturing system is used to demonstrate the effectiveness of the proposed scheme. Simulation results show that the presented demand response algorithm can minimize electricity costs and maintain production tasks, as compared to a benchmark without demand response. Moreover, the performance of the multi-agent deep reinforcement learning approach against a mathematical model method is investigated. [ABSTRACT FROM AUTHOR]

In this paper, a deep reinforcement learning (DRL)-based collision avoidance method is proposed for an unmanned surface vehicle (USV). This approach is applicable to the decision-making stage of collision avoidance, which determines whether the avoidance is necessary, and if so, determines the direction of the avoidance maneuver. To utilize the visual recognition capability of deep neural networks as a tool for analyzing the complex and ambiguous situations that are typically encountered, a grid map representation of the ship encounter situation was suggested. For the composition of the DRL network, we proposed a neural network architecture and semi-Markov decision process model that was specially designed for the USV collision avoidance problem. The proposed DRL network was trained through repeated simulations of collision avoidance. After the training process, the DRL network was implemented in collision avoidance experiments and simulations to evaluate its situation recognition and collision avoidance capability. • A deep reinforcement learning-based USV collision avoidance method is proposed. • To analyze the complex and ambiguous ship encounter situations, a grid map representation was suggested. • The proposed DRL network was implemented in collision avoidance experiments and simulations to evaluate its situation recognition and collision avoidance capability. [ABSTRACT FROM AUTHOR]