Search

Although deep reinforcement learning (DRL) has recently emerged as a promising technique for optimal trade execution, two problems still remain unsolved: (1) the lack of a generalized model for a large collection of stocks and execution time horizons; and (2) the inability to accurately train algorithms due to the discrepancy between the simulation environment and real market. In this article, we address the two issues by utilizing a widely used reinforcement learning (RL) algorithm called proximal policy optimization (PPO) with a long short-term memory (LSTM) network and by building our proprietary order execution simulation environment based on historical level 3 market data of the Korea Stock Exchange (KRX). This paper, to the best of our knowledge, is the first to achieve generalization across 50 stocks and across an execution time horizon ranging from 165 to 380 min along with dynamic target volume. The experimental results demonstrate that the proposed algorithm outperforms the popular benchmark, the volume-weighted average price (VWAP), highlighting the potential use of DRL for optimal trade execution in real-world financial markets. Furthermore, our algorithm is the first commercialized DRL-based optimal trade execution algorithm in the South Korea stock market. [ABSTRACT FROM AUTHOR]

In South Korea, there have been many studies on efficient building-energy management using renewable energy facilities in single zero-energy houses or buildings. However, such management was limited due to spatial and economic problems. To realize a smart zero-energy city, studying efficient energy integration for the entire city, not just for a single house or building, is necessary. Therefore, this study was conducted in the eco-friendly energy town of Chungbuk Innovation City. Chungbuk successfully realized energy independence by converging new and renewable energy facilities for the first time in South Korea. This study analyzes energy data collected from public buildings in that town every minute for a year. We propose a smart city building-energy management model based on the results that combine various renewable energy sources with grid power. Supervised learning can determine when it is best to sell surplus electricity, or unsupervised learning can be used if there is a particular pattern or rule for energy use. However, it is more appropriate to use reinforcement learning to maximize rewards in an environment with numerous variables that change every moment. Therefore, we propose a power distribution algorithm based on reinforcement learning that considers the sales of Energy Storage System power from surplus renewable energy. Finally, we confirm through economic analysis that a 10% saving is possible from this efficiency. [ABSTRACT FROM AUTHOR]

South Korea has the highest incidence of mental diseases such as depression among OECD countries. It is important to identify the presence of mental diseases in the early stages, and to treat them so that the symptoms do not get worse such as physical diseases. However, on the contrary the perception of mental illness in Korea is still somewhat negative. Therefore, it is important to establish a system that allows people to diagnose and treat initial symptoms of mental illness with electronic devices anytime, anywhere, even if they do not visit any facility. Therefore, in this study, the current criteria for diagnosis of depression and trends in chatbots related to depression were analyzed in relation to mental illness especially depression. Additionally, a chatbot structure for diagnosis and treatment of depression for Koreans suffering from early depression of locating near the baseline in the diagnostic criteria for depression is proposed based on multi-turn conversation dataset and reinforcement learning. [ABSTRACT FROM AUTHOR]

The conventional volt-VAR control (VVC) in distribution systems has limitations in solving the overvoltage problem caused by massive solar photovoltaic (PV) deployment. As an alternative method, VVC using solar PV smart inverters (PVSIs) has come into the limelight, which can respond quickly and effectively to solve the overvoltage problem by absorbing reactive power. However, the network power loss, that is, the sum of line losses in the distribution network, increases with reactive power. Dynamic distribution network reconfiguration (DNR), which hourly controls the network topology by controlling sectionalizing and tie switches, can also solve the overvoltage problem and reduce network loss by changing the power flow in the network. In this study, to improve the voltage profile and minimize the network power loss, we propose a control scheme that integrates the dynamic DNR with volt-VAR control of PVSIs. The proposed control scheme is practically usable for three reasons: Primarily, the proposed scheme is based on a deep reinforcement learning (DRL) algorithm, which does not require accurate distribution system parameters. Furthermore, we propose the use of a heterogeneous multiagent DRL algorithm to control the switches centrally and PVSIs locally. Finally, a practical communication network in the distribution system is assumed. PVSIs only send their status to the central control center, and there is no communication between the PVSIs. A modified 33-bus distribution test feeder reflecting the system conditions of South Korea is used for the case study. The results of this case study demonstrates that the proposed control scheme effectively improves the voltage profile of the distribution system. In addition, the proposed scheme reduces the total power loss in the distribution system, which is the sum of the network power loss and curtailed energy, owing to the voltage violation of the solar PV output. [ABSTRACT FROM AUTHOR]

To reduce the waste of fresh foods, one of the e-commerce companies in South Korea utilizes lateral transshipment in the network of online platforms and offline shops, which is called the online–offline channel system (OOCS). Even though the OOCS has achieved success in real practice, there is room for further study on this system with regard to deriving a transshipment policy. For this reason, this study aims to develop a solution approach that could derive a promising policy and analyze the impacts of transshipment in the OOCS. The main contributions are summarized as follows. First, we propose a model to deal with the proactive transshipment of perishable products in the OOCS. In particular, this is the first study that introduces the concept of the heterogeneous shelf life considering different properties of online and offline channels. Second, we develop the hybrid deep reinforcement learning (DRL) approach by combining the soft actor–critic algorithm with two novel acceleration methods. The developed method could obtain a promising policy without assumptions about demand distribution and mitigate computational burdens by reducing action spaces. On a set of experiments carried out on real-world demand data, the transshipment policy derived from the hybrid DRL approach could obtain the best profit compared to existing algorithms. Third, we examine the impacts of transshipment by differing types of demand and varying the unit transshipment cost parameter. We find that transshipment substantially reduces the outdating cost by allowing the offline channel to make good use of the old products that will be discarded in the online channel, which is new to the literature. • We studied a proactive transshipment of fresh food in the online–offline channel system. • We developed the hybrid deep reinforcement learning approach based on the soft actor–critic algorithm. • We observed the outperformance of the proposed approach compared to benchmark algorithms. • We analyzed the impact of transshipment by varying demand types and cost parameters. • We found that transshipment reduces the outdating cost by utilizing a connection of online and offline channels. [ABSTRACT FROM AUTHOR]

The Korea–Japan Joint Development Zone (JDZ) Agreement of 30 January 1974 was one of a number of innovative arrangements for cooperation in an overlapping maritime area. Nevertheless, four decades after its entry into force in 1978, the Agreement has not produced a single drop of oil. In contrast with Korea's positive view on an oil discovery, Japan has held a negative stance since its concessionaires' release of rights under the Agreement in 1992. Without Japan's cooperation, the Agreement is a dead instrument. This article looks, from a Korean perspective, for the solutions whereby the Korea–Japan JDZ Agreement can operate. [ABSTRACT FROM AUTHOR]

In this work, an advantage actor critic (A2C) based intelligent automatic frequency control (AFC) system was developed for X-band linear accelerator (LINAC). A2C is one type of reinforcement learning, which indicates how software agents should perform actions in an environment. In this paper, the A2C based AFC algorithm and its environment design, simulation result, and controller hardware and software processes are described. The objective of our design is to match LINAC and magnetron frequency, and it is implemented via reward shaping using comparison with the reflected power in the adjacent time. The simulation with the A2C algorithm was implemented in two modes, namely, periodic and White Gaussian Noise (WGN) waves, for analysis with temperature and random disturbance, respectively. In order to create artificial disturbance in the experiment, the magnetron shaft was shifted randomly every 0.5 s by using WGN with 18° angle of the step motor. The standard deviation of the reflected power was 5.63 kW, and the average power was 130.9 kW. To obtain maximum reward at the beginning of the A2C training, the adjacent frequency is needed. The measured average reflected power and standard deviation were 122.8 kW and 1.75 kW, respectively, after 2000 iterations. The results show that the reflected power and standard deviation of the AFC with A2C were lower compared to open-loop with artificial disturbance. The RF station of a medical X-band LINAC was used as the test bench, and the performance was confirmed by the results of an experiment conducted at Sungkyunkwan University in Korea. [ABSTRACT FROM AUTHOR]

Nowadays, Automated Vehicle (AV) technology is gaining attention as a candidate to improve the efficiency of Bus Rapid Transit (BRT) systems. However, there are still some challenges in AV technology including limited perception range and lack of cooperation capability in mixed traffic situations with drivers. The emerging Connected and Automated Vehicles (CAVs) and Cooperative Intelligent Transportation System (C-ITS) offer an unprecedented opportunity to solve such challenges. As a result, this study presents a framework for Connected and Automated BRT (CA-BRT), including a cloud-based architecture and a deep reinforcement learning system for Sectionalized Speed Guidance (SSG) system designed for CAVs. The proposed framework is field-tested in Sejong City in South Korea, where there are various road environments such as bus stops, overpasses, underground tunnels, intersections, and crosswalks. The driving performance of the proposed system is compared with different types of control scenarios, and the results from the field tests show that the proposed system improves the driving performance of the AVs in various aspects including driving safety, ride comfort, and energy efficiency with downstream information obtained from road infrastructures. • Proposes Architecture for Traffic Management Center in Cloud Platform. • Proposes Speed Guidance System for Connected and Automated Bus Rapid Transit. • Evaluates Deep Reinforcement Learning for Vehicle Control in Real-world Experiments. [ABSTRACT FROM AUTHOR]

• A reinforcement learning framework for energy storage systems is presented. • The Markov Decision Process is designed for reinforcement learning. • Five reinforcement learning algorithms were used to assess the multi-use framework. • Deep recurrent Q-network and Proximal Policy Optimization outperformed the others. • Experimental results indicate the near-optimal performance of the proposed method. This study proposes a multi-use energy storage system (ESS) framework to participate in both price-based and incentive-based demand response programs with reinforcement learning (RL) on the demand side. We focused on industrial customers, to provide them the opportunity to obtain additional profits through market participation in addition to managing their load. Since industrial customers pay their electricity bills according to the time of use tariff structure, they can benefit if they can shift their electricity usage from high-price hours to low-price hours. Furthermore, they are able obtain additional incentives by fulfilling a dispatch signal from the system operator by using the ESS. To model the multi-use ESS by industrial users, we used the RL framework to make customer decisions. The RL approach uses a control action policy by interacting with an environment with no prior knowledge. For this, we formulated the ESS operation as a Markov decision process so that the environmental information obtained by the customers provides RL, which takes optimal actions for the current environment considering customer benefits. We developed several RL agents to identify an acceptable control agent. We utilized the actual industrial load profile in South Korea to train the RL agents. The experimental results demonstrated that the proposed framework can make near-optimal decisions for using ESS in multiple ways. [ABSTRACT FROM AUTHOR]

An airline scheduler plans flight schedules with efficient resource utilization. However, unpredictable airline disruptions, such as temporary closures of an airports, cause schedule perturbations. Therefore, recovering disrupted flight schedules is essential for airlines. Many previous studies have relied on copies of flight arcs, which could affect the quality of solutions, and have not addressed the key measure of airlines' on-time performance as their objective. To fill these research gaps, we propose Q-learning and Double Q-learning algorithms using the reinforcement learning approach for aircraft recovery to support airline operations. We present an artificial environment of daily flight schedules and the Markov decision process for aircraft recovery. The proposed approach is first compared with existing algorithms on the benchmark instance. In comparison with other algorithms, the developed Q-learning and Double Q-learning algorithms obtain high-quality solutions within the proper computation time. To verify that the proposed approach can be applicable to a real-world case and can adapt to realistic conditions, we employ a domestic flight schedule from one of the airlines in South Korea. We evaluate the reinforcement learning approach on a set of experiments carried out on real-world data. Computational experiments show that reinforcement learning algorithms recover disrupted flight schedules effectively, and that our approaches flexibly adapt to various objectives and realistic conditions. • We cover the multi-fleet aircraft recovery in cases of airline disruption. • Q-learning and Double Q-learning algorithms are developed for aircraft recovery. • We propose an environment of daily flight schedules and the Markov decision process. • We observe the outperformance of the proposed algorithms compared to existing methods. • We analyze the impact of disruption through experiments with a real case of Korea. [ABSTRACT FROM AUTHOR]

Purpose In the past 20 years, the research in the field of robotics and intelligent systems has gradually increased. With the advancement of related fields, service robots have received extensive attention from industry and academia (R. Reddy, 2006). The increasingly serious aging trend has brought great challenges to the current society. Due to illness or physical decline, it is difficult for the elderly to carry out their daily life smoothly. Service robots can actively interact with the elderly and provide corresponding help. For example, reminding meals, reminding taking medicines, assisting in doing sports and so on are possible (Mast M et al., 2015). For the safety assurance of the elderly, the robot should be actuated properly for speed regulation and strong recognition ability, combined with an audiovisual system to prevent accidental collision with elderly people. In elderly care, robots need to respond to the voice commands of the elderly, and current speech recognition technology makes this possible. (Park C et al., 2012). This paper proposes a well-established software engineering approach for properly coordinating service robotic systems. Method The core of the service robot behavior system is the interaction and cooperation between the elderly and the service robot, which is important to design a simple and stable system. Therefore, when building a service robot behavior system, it is necessary to clearly define the system architecture to complete the corresponding behaviors. Based on the three artificial intelligence algorithms of BERT, CNN, and DQN, this paper proposes a service robot platform to perform corresponding behaviors according to categories by analyzing the spoken instructions of speakers. (Figure 1). Through an audiovisual system to obtain the activity information of the elderly, the safety status and cognitive status of the elderly are monitored. Service robots collect data through cameras and microphones and then pass it on to other systems. The main purpose of a behavior system is to ensure safety and reduce the burden on caregivers by tracking the behavior of the elderly and providing the caregivers with the latest status reports. In this system, the BERT algorithm takes the processed sound data as input, outputs relevant features, and classifies them through CNN. Pre-defined event categories closely related to the daily life of the elderly are obtained. The actuator system provides the locomotion capabilities of service robots. DQN performs fixed behaviors through camera data and event categories to help the elderly go about their daily lives. (Figure 2). Results and Discussion A service robot should be able to perform several behaviors autonomously to provide various services for elderly beings in a dynamic and partially unknown environment by applying both technology and knowledge. The service robot proposed in this paper accurately is divided into the audiovisual, behavior, and actuator systems, which enables expandable scalability and module-based maintenance. Specially, by utilizing BERT and CNN, the performance of recognizing voices is improved more than traditional approaches. In the future, it is required to enhance data transfer structures among the systems. [ABSTRACT FROM AUTHOR]

[Display omitted] • The economic optimization model for planning the rooftop PV system was developed. • Geographic information system (GIS)-based reinforcement learning (RL) was applied. • The developed model achieved the economic profit of 539,197 USD for 123 buildings. • The developed model outperformed the existing planning methods by 4.4% and 4.3%. • The 91.8% lower global warming potential than the Korean mix grid was achieved. This study aimed to develop a geographic information system (GIS)-based reinforcement learning (RL) model for optimal planning of a rooftop PV system, considering the uncertainty of future scenarios throughout the life cycle of buildings. To that end, GIS was used to establish the spatial data for the rooftop PV installation, and an RL model was developed to maximize the economic profit of the rooftop PV installation in various locations and future scenarios. The developed model was applied to residential buildings in Nonhyeon district, South Korea to evaluate their economic profitability and to compare the model with the existing planning methods. With the use of the developed GIS-based RL model, the rooftop PV system became economically feasible, achieving average economic profit of 539,197 USD over all scenarios for all target buildings which was higher than that of the existing models by 4.4% and 4.3%. Furthermore, the developed model outperformed the existing models especially in volatile scenarios with lower solar radiation. Therefore, the use of the proposed GIS-based RL model can optimize the economic feasibility of rooftop PV systems for buildings, which will benefit building owners and community-level energy business owners. In conclusion, the developed model can promote the adoption of rooftop PV systems, which have 91.8% lower global warming potential than the Korean mixed grid, without additional subsidies to achieve Korea's national CO2 emission reduction plan. [ABSTRACT FROM AUTHOR]

Background: Detecting bacteremia among surgical in-patients is more obscure than other patients due to the inflammatory condition caused by the surgery. The previous criteria such as systemic inflammatory response syndrome or Sepsis-3 are not available for use in general wards, and thus, many clinicians usually rely on practical senses to diagnose postoperative infection.Objective: This study aims to evaluate the performance of continuous monitoring with a deep learning model for early detection of bacteremia for surgical in-patients in the general ward and the intensive care unit (ICU).Methods: In this retrospective cohort study, we included 36,023 consecutive patients who underwent general surgery between October and December 2017 at a tertiary referral hospital in South Korea. The primary outcome was the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) for detecting bacteremia by the deep learning model, and the secondary outcome was the feature explainability of the model by occlusion analysis.Results: Out of the 36,023 patients in the data set, 720 cases of bacteremia were included. Our deep learning-based model showed an AUROC of 0.97 (95% CI 0.974-0.981) and an AUPRC of 0.17 (95% CI 0.147-0.203) for detecting bacteremia in surgical in-patients. For predicting bacteremia within the previous 24-hour period, the AUROC and AUPRC values were 0.93 and 0.15, respectively. Occlusion analysis showed that vital signs and laboratory measurements (eg, kidney function test and white blood cell group) were the most important variables for detecting bacteremia.Conclusions: A deep learning model based on time series electronic health records data had a high detective ability for bacteremia for surgical in-patients in the general ward and the ICU. The model may be able to assist clinicians in evaluating infection among in-patients, ordering blood cultures, and prescribing antibiotics with real-time monitoring. [ABSTRACT FROM AUTHOR]

Short-term load forecasting (STLF) is very important for planning and operating power systems and markets. Various algorithms have been developed for STLF. However, numerous utilities still apply additional correction processes, which depend on experienced professionals. In this study, an STLF algorithm that uses a similar day selection method based on reinforcement learning is proposed to substitute the dependence on an expert's experience. The proposed algorithm consists of the selection of similar days, which is based on the reinforcement algorithm, and the STLF, which is based on an artificial neural network. The proposed similar day selection model based on the reinforcement learning algorithm is developed based on the Deep Q-Network technique, which is a value-based reinforcement learning algorithm. The proposed similar day selection model and load forecasting model are tested using the measured load and meteorological data for Korea. The proposed algorithm shows an improvement accuracy of load forecasting over previous algorithms. The proposed STLF algorithm is expected to improve the predictive accuracy of STLF because it can be applied in a complementary manner along with other load forecasting algorithms. [ABSTRACT FROM AUTHOR]