Search

To operate safely in a dynamic environment, autonomous vehicles must possess the same level of predictive driving abilities as human drivers and must be capable of anticipating the future actions of other dynamic objects in the environment, especially those of neighboring vehicles. The development of safe autonomous vehicles (AVs) poses a challenging task as it requires algorithms that can make real-time decisions in unpredictable circumstances. Reinforcement learning (RL) presents a promising approach for AV control, as it utilizes trial and error to enable optimal decision-making. However, traditional RL algorithms are unsuitable for safety-critical applications, as they may explore unsafe actions, potentially resulting in accidents. Safe reinforcement learning (SRL) algorithms have been developed to address this issue, prioritizing safe and reliable decisions. These algorithms incorporate constraints to prevent unsafe actions or utilize techniques to estimate action risk and avoid actions deemed excessively risky. Despite computational challenges, SRL holds significant promise for AV control, and is likely to play a crucial role in developing safe and reliable systems. SRL methods are critical for the general adoption of autonomous vehicles by guaranteeing their safety and reliability. These algorithms utilize methods like uncertainty and risk estimation along with penalty functions, to avoid excessively risky actions and have the potential to significantly reduce accidents and build public trust in autonomous driving. However, there are challenges that need to be addressed, such as the dynamic nature of real-world traffic, high computational costs, and the diversity of road design; and these varying conditions make the designing, testing, and validating of SRL algorithms difficult. Despite these challenges, SRL presents a promising solution, through integrating new sensing technologies and machine learning techniques, to develop safe, efficient, and environmentally friendly transportation systems.

River water quality management and monitoring are essential responsibilities for communities near rivers. Government decision-makers should monitor important quality factors like temperature, dissolved oxygen (DO), pH, and biochemical oxygen demand (BOD). Among water quality parameters, the BOD throughout 5 days is an important index that must be detected by devoting a significant amount of time and effort, which is a source of significant concern in both academic and commercial settings. The traditional experimental and statistical methods cannot give enough accuracy or solve the problem for a long time to detect something. This study used a unique hybrid model called MVMD-LWLR, which introduced an innovative method for forecasting BOD in the Klang River, Malaysia. The hybrid model combines a locally weighted linear regression (LWLR) model with a wavelet-based kernel function, along with multivariate variational mode decomposition (MVMD) for the decomposition of input variables. In addition, categorical boosting (Catboost) feature selection was used to discover and extract significant input variables. This combination of MVMD-LWLR and Catboost is the first use of such a complete model for predicting BOD levels in the given river environment. In addition, an optimization process was used to improve the performance of the model. This process utilized the gradient-based optimization (GBO) approach to fine-tune the parameters and better the overall accuracy of predicting BOD levels. To assess the robustness of the proposed method, we compared it to other popular models such as kernel ridge (KRidge) regression, LASSO, elastic net, and gaussian process regression (GPR). Several metrics, comprising root-mean-square error (RMSE), R (correlation coefficient), U95% (uncertainty coefficient at 95% level), and NSE (Nash–Sutcliffe efficiency), as well as visual interpretation, were used to evaluate the predictive efficacy of hybrid models. Extensive testing revealed that, in forecasting the BOD parameter, the MVMD-LWLR model outperformed its competitors. Consequently, for BOD forecasting, the suggested MVMD-LWLR optimized with the GBO algorithm yields encouraging and reliable results, with increased forecasting accuracy and minimal error. [ABSTRACT FROM AUTHOR]

Heat and mass transport through evaporation or drying processes occur in many applications such as food processing, pharmaceutical products, solar-driven vapor generation, textile design, and electronic cigarettes. In this paper, the transport of water from a fresh potato considered as a wet porous media with laminar convective dry air fluid flow governed by Darcy's law in two-dimensional is highlighted. Governing equations of mass conservation, momentum conservation, multiphase fluid flow in porous media, heat transfer, and transport of participating fluids and gases through evaporation from liquid to gaseous phase are solved simultaneously. In this model, the variable is block locations, the object function is changing water saturation inside the porous medium and the constraint is the constant mass of porous material. It shows that there is an optimal configuration for the purpose of water removal from the specimen. The results are compared with experimental and analytical methods Benchmark. Then for the purpose of configuration optimization, multi-agent reinforcement learning (MARL) is used while multiple porous blocks are considered as agents that transfer their moisture content with the environment in a real-world scenario. MARL has been tested and validated with previous conventional effective optimization simulations and its superiority proved. Our study examines and proposes an effective method for validating and testing multiagent reinforcement learning models and methods using a multiagent simulation. [ABSTRACT FROM AUTHOR]

In comparison to commonly employed iterative learning controls and reinforced learning techniques in model predictive controls for buildings, a supervised learning based iterative learning control platform that is more suitable and computationally efficient for real-world applications is proposed. The proposed control system relies on a data-driven model and utilizes the Random Forest algorithm to develop an HVAC start-stop model; this model considers only a limited system history period that can influence the current state, thus avoiding prolonged learning periods and time-consuming exploration. Specifically, within the current timeframe, the HVAC start-stop model learns from daily errors, and start and stop times “labeled as adjusted” accordingly.The proposed platform was validated against the TRNSYS baseline of a research facility, which was meticulously calibrated with actual measurements. In comparison with the convention, the proposed approach yielded significant energy savings of 6.5–7.6 % in HVAC annual energy consumption, while maintaining temperature comfort for approximately 97–98 % of the annual operating days. Notably, by implementing supply air volume ramp-up in conjunction with HVAC optimal start control, temperature comfort for up to 99 % of the annual operating days was achieved, along with a notable 9.7 % reduction in HVAC annual energy consumption.

This paper proposes an Enhanced Edge-Linked Caching (EELC) scheme for Internet of Things (IoT) environments under Information-Centric Networking (ICN), employing an advanced use of Proximal Policy Optimization (PPO), a form of deep reinforcement learning, to inform the caching decisions of edge nodes. The rapid proliferation of IoT devices has led to significant challenges in managing content efficiently within networks, particularly in terms of scalability, latency, and energy consumption. Traditional IP-based architectures are inadequate in addressing these challenges, necessitating a shift towards a content-centric approach provided by ICN. By leveraging PPO, EELC dynamically adapts to changing IoT network conditions, optimizing caching strategies to enhance energy efficiency and improve network responsiveness. In our simulation, we verify the performance of EELC in comparison to the Edge-Linked Caching (ELC) and Leave Copy Everywhere (LCE) approaches under different cache sizes and Zipf-distributed content requests. EELC performs far better than ELC under energy efficiency, cache hit ratios, and server hit reduction in all tested scenarios. This indicates that EELC could be a potential approach for significantly improving network efficiency and responsiveness in IoT networks.

Wireless Sensor Networks (WSNs) play an instrumental role in monitoring and data collection across a plethora of domains. As these networks expand and get more intricate, the need for energy-efficient and smart sensing models becomes paramount. This paper introduces the energy-efficient collaborative sensing model using game theory for WSNs. This model is unique in its approach, amalgamating game theory and reinforced learning to ensure optimal energy consumption without compromising the quality of service. A cornerstone of this model is the introduction of the Selection Propensity Index (SPI), which helps in the decision-making process of choosing the right sensors for specific tasks. Moreover, the model employs a Distributed Anticipatory Time-slot selection Algorithm (DATA), an RL-based algorithm that facilitates collaborative communication. This aspect ensures that the sensors while communicating, select the time slots that would result in minimal energy expenditure and optimal data transmission. Simulations are performed over the proposed model and the results obtained showcase that the proposed model outperforms the existing similar methods in terms of energy efficiency, packet drop ratio, and throughout. The proposed model exhibits an enhancement in terms of network lifetime by 202%, throughput by 30%, and is faster by more than 60% compared to the existing methods running on full load, thereby providing an innovative approach to energy efficiency in sensor networks.

As vocational education is transforming constantly, there is an urgent demand in the field of education for smart teaching systems to be able to respond to students' personal learning needs in a more dynamic way, but a review of currently available algorithms reveals that the common application of existing methods lacks a deep enough understanding of students' individual differences. Out of these concerns, this study aims to propose a novel and hybrid framework for the design of smart teaching systems based on reinforced learning and deep learning, so as to overcome the shortcomings of existing research and more accurately predict students' personal needs. Besides, an end-to-end model with a retrieval attention mechanism has been designed for generating responses with precise information about students' learning needs. This study provides a smart teaching scheme for vocational education that is new, efficient, and humane, while also providing a solid theoretical foundation for the reform and innovation of the education system in the future. [ABSTRACT FROM AUTHOR]

In this study, we propose a novel structural condition assessment method based on adaptive vision feature extractions and reinforced learning-assisted evolution. First, the ‘features from accelerated segment test’ (FAST) algorithm cooperating with the Kanade-Lucas-Tomasi algorithm synergistically captures the displacements from the video clips. However, the fixed threshold values in the FAST algorithm may not satisfy the pixel requirements for different images. Second, for any evolutionary algorithms (EAs), their search modes significantly affect the optimization performance but are relatively single and monotonous. Therefore, they may perform poorly for some high-dimensional and complicated multi-objective functions. To resolve these two critical problems, firstly, we propose an adaptive feature points extraction strategy during the displacements acquisition stage. Secondly, a novel local search framework subjected to the reinforced learning framework is designed for EAs as an improvement. The proposed structural condition assessment method is used to evaluate a space frame structure by optimizing the vibration data-based multi-sample objective function. The damaged locations and severities of the frame can be well identified. [ABSTRACT FROM AUTHOR]

Artificial Intelligence has been playing a profound role in the global economy, social progress, and people's daily life. With the increasing capabilities and accuracy of AI, the application of AI will have more impacts on manufacturing and service areas in the era of industry 4.0. This study conducts a systematic literature review to study the state-of-the-art on AI in industry 4.0. This paper describes the development of industries and the evolution of AI. This paper also identifies that the development and application of AI will bring not only opportunities but also challenges to industry 4.0. The findings provide a valuable reference for researchers and practitioners through a multi-angle systematic analysis of AI. In the era of industry 4.0, AI system will become an innovative and revolutionary assistance to the whole industry. [ABSTRACT FROM AUTHOR]

Innovation in UAV design technologies over the last decade and a half has resulted in capabilities that flourished the development of unique and complex multi-mission capable UAVs. These emerging new distinctive designs of UAVs necessitate development of intelligent and robust Control Laws which are independent of inherent plant variations besides being adaptive to environmental changes for achieving desired design objectives. Current research focuses on development of a control framework which aims to maximize the glide range for an experimental UAV employing reinforcement learning (RL)-based intelligent control architecture. A distinct model-free RL technique, abbreviated as 'MRL', is suggested which is capable of handling UAV control complications while keeping the computation cost low. At core, the basic RL DP algorithm has been sensibly modified to cater for the continuous state and control space domains associated with the current problem. Review of the performance characteristics through analysis of the results indicates the prowess of the presented algorithm to dynamically adapt to the changing environment, thereby making it suitable for complex designed UAV applications. Nonlinear simulations carried out under varying environmental conditions illustrated the effectiveness of the proposed methodology and its success over the conventional classical approaches. [ABSTRACT FROM AUTHOR]

Internet of Vehicles (IoV) has been viewed as a pivotal innovation for laying out Intelligent Transportation Systems in shrewd urban communities and is perhaps of the most encouraging application later on Internet of Things. the 6th era has started to arise. With the appearance of 6G interchanges, tremendous organization frameworks, be generally utilized, and the amount of organization hubs will rise remarkable development, which brings about extremely high energy utilization. As perhaps of the most encouraging application in future Internet of Things, Internet of Vehicles (IoV) has been recognized as an essential innovation for fostering the Clever Transportation Frameworks in brilliant urban areas. With the rise of the sixth generation (6G) correspondences innovations, monstrous network frameworks will be thickly sent and the quantity of network hubs will increment dramatically, leading to very high energy utilization. There has been an upsurge important to foster the green IoV towards maintainable vehicular correspondence and networking in the 6G time. Be that as it may, as an extraordinary versatile ad-hoc network, the energy cost in an IoV framework includes the correspondence and calculation energy in addition to the fuel utilization and the power cost of moving vehicles. Besides, the energy collecting innovation, which is probably going to be adopted generally in 6G frameworks, will entangle the improvement of energy proficiency in the whole framework.A change in the 6G period, there has been an expansion in interest in fostering the green IoV for supportable vehicle correspondence and systems administration. Nonetheless, as a remarkable versatile impromptu organization, an IoV framework's energy cost incorporates energy for correspondence and figuring. Reinforcement Learning (RL) can successfully address issues with decision-making to accomplish this goal. Large-scale wireless networks have enormous and complex state and action spaces, though. As a result, RL might not be able to identify the ideal plan of action in a timely manner. To solve this problem, Deep Reinforcement Learning (DRL), a infusion of RL and DL, has been developed. [ABSTRACT FROM AUTHOR]

While the design practice in the architecture, engineering, and construction (AEC) industry continues to be a creative activity, approaching the design problem from a perspective of the decision-making science has remarkable potentials that manifest in the delivery of high-performing sustainable structures. These possible gains can be attributed to the myriad of decision-making tools and technologies that can be implemented to assist design efforts, such as artificial intelligence (AI) that combines computational power and data wisdom. Such combination comes to extreme importance amid the mounting pressure on the AEC industry players to deliver economic, environmentally friendly, and socially considerate structures. Despite the promising potentials, the utilization of AI, particularly reinforced learning (RL), to support multidisciplinary design endeavours in the AEC industry is still in its infancy. Thus, the present research discusses developing and applying a Markov Decision Process (MDP) model, an RL application, to assist the preliminary multidisciplinary design efforts in the AEC industry. The experimental work shows that MDP models can expedite identifying viable design alternatives within the solutions space in multidisciplinary design while maximizing the likelihood of finding the optimal design. [ABSTRACT FROM AUTHOR]

Recent industrial robotics covers a broad part of the manufacturing spectrum and other human everyday life applications; the performance of these devices has become increasingly important. Positioning accuracy and repeatability, as well as operating speed, are essential in any industrial robotics application. Robot positioning errors are complex due to the extensive combination of their sources and cannot be compensated for using conventional methods. Some robot positioning errors can be compensated for only using machine learning (ML) procedures. Reinforced machine learning increases the robot's positioning accuracy and expands its implementation capabilities. The provided methodology presents an easy and focused approach for industrial in situ robot position adjustment in real-time during production setup or readjustment cases. The scientific value of this approach is a methodology using an ML procedure without huge external datasets for the procedure and extensive computing facilities. This paper presents a deep q-learning algorithm applied to improve the positioning accuracy of an articulated KUKA youBot robot during operation. A significant improvement of the positioning accuracy was achieved approximately after 260 iterations in the online mode and initial simulation of the ML procedure. [ABSTRACT FROM AUTHOR]

This paper presents a model of hierarchical associative memory, which can be used as a basis for building artificial cognitive agents of general purpose. With the help of this model, one of the most important problems of modern machine learning and artificial intelligence in general can be solved — the ability for a cognitive agent to use "life experience" to process the context of the situation in which he was, is and, possibly, will be. This model is applicable for artificial cognitive agents functioning both in specially designed virtual worlds and in objective reality. The use of hierarchical associative memory as a long-term memory of artificial cognitive agents will allow the latter to effectively navigate both in the general knowledge accumulated by mankind and in their life experience. The novelty of the presented work is based on the author's approach to the construction of context-dependent artificial cognitive agents using an interdisciplinary approach, in particular, based on the achievements of artificial intelligence, cognitology, neurophysiology, psychology and sociology. The relevance of this work is based on the keen interest of the scientific community and the high social demand for the creation of general-level artificial intelligence systems. Associative hierarchical memory, based on the use of an approach similar to the hypercolumns of the human cerebral cortex, is becoming one of the important components of an artificial intelligent agent of the general level. The article will be of interest to all researchers working in the field of building artificial cognitive agents and related fields. [ABSTRACT FROM AUTHOR]

Automated Vehicles (AVs) are poised to disrupt travel patterns and the sustainability of transportation networks. Conventional methods for studying these changes, such as stated preference surveys and agent-based simulations, have limitations. Serious games offer a promising alternative, providing a controlled and engaging environment for investigating travel behavior. In our study, 200 participants, grouped into sessions of 10, engaged in a competitive serious game simulating 50 daily choices of travel mode and departure time across three automated options. Two scenarios were examined: one with recurring congestion and another with nonrecurring congestion. Automated transit had fixed schedules, while private and shared rides could adapt to a congested bottleneck. Results revealed that ridesharing dominated, reaching 60% mode share under recurring congestion, displacing transit, and a comparative equilibrium emerged between shared and private rides. In the nonrecurring congestion scenario, ridesharing dropped to 37%, and a comparable multimodal equilibrium developed. Participants rarely achieved the optimal score, attaining a maximum of 88% of its potential. This study highlights a policy paradox: unregulated AV traffic can reduce transit use, exacerbate recurring congestion, yet necessitate increased transit investment to address nonrecurring congestion, confirming the Downs-Thomson paradox. Creating appealing mass transit alternatives is imperative to ensure efficiency and sustainability in the era of automated mobility. [ABSTRACT FROM AUTHOR]

This paper presents a novel methodology to develop itinerary choice models (ICM) for air travelers that addresses the limitations of the traditional utility-maximization approach. The methodology integrates a reinforcement learning algorithm and an airline network competition analysis model. The reinforcement learning algorithm searches for the values of parameters of the itinerary choice model while considering maximizing a reward function. The reward function is measured as the negative of the difference between the estimated and observed system metrics. The airline network competition analysis model is used to calculate the estimated system metrics. It is a simulation model that represents passenger-itinerary assignment. It captures the demand–supply interactions at the network level while considering the competition among all airlines. An ICM system is calibrated using the developed framework considering the global airline network, which includes more than 500,000 airport pairs. Validating the model against ground truth data shows that the developed model adequately captures the travelers' itinerary choice behavior and replicates the competition pattern among airlines. [ABSTRACT FROM AUTHOR]

One of the main challenges in robotics is the development of systems that can adapt to their environment and achieve autonomous behavior. Current approaches typically aim to achieve this by increasing the complexity of the centralized controller by, e.g., direct modeling of their behavior, or implementing machine learning. In contrast, we simplify the controller using a decentralized and modular approach, with the aim of finding specific requirements needed for a robust and scalable learning strategy in robots. To achieve this, we conducted experiments and simulations on a specific robotic platform assembled from identical autonomous units that continuously sense their environment and react to it. By letting each unit adapt its behavior independently using a basic Monte Carlo scheme, the assembled system is able to learn and maintain optimal behavior in a dynamic environment as long as its memory is representative of the current environment, even when incurring damage. We show that the physical connection between the units is enough to achieve learning, and no additional communication or centralized information is required. As a result, such a distributed learning approach can be easily scaled to larger assemblies, blurring the boundaries between materials and robots, paving the way for a new class of modular "robotic matter" that can autonomously learn to thrive in dynamic or unfamiliar situations, for example, encountered by soft robots or self-assembled (micro)robots in various environments spanning from the medical realm to space explorations. [ABSTRACT FROM AUTHOR]

Recent industrial robotics covers a broad part of the manufacturing spectrum and other human everyday life applications; the performance of these devices has become increasingly important. Positioning accuracy and repeatability, as well as operating speed, are essential in any industrial robotics application. Robot positioning errors are complex due to the extensive combination of their sources and cannot be compensated for using conventional methods. Some robot positioning errors can be compensated for only using machine learning (ML) procedures. Reinforced machine learning increases the robot’s positioning accuracy and expands its implementation capabilities. The provided methodology presents an easy and focused approach for industrial in situ robot position adjustment in real-time during production setup or readjustment cases. The scientific value of this approach is a methodology using an ML procedure without huge external datasets for the procedure and extensive computing facilities. This paper presents a deep q-learning algorithm applied to improve the positioning accuracy of an articulated KUKA youBot robot during operation. A significant improvement of the positioning accuracy was achieved approximately after 260 iterations in the online mode and initial simulation of the ML procedure.

Set on the north side of Dublin city, Killester College of Further Education (KCFE) offers full and parttime courses to adults of all ages. Each course stipulates that each student must secure work placement in a workplace and attend one day each week for the academic year. For certain courses, work placement can prove very difficult to secure. To counter this ongoing issue, the teaching staff and management of the college have been developing links with organisations and employers in the Dublin area to establish formal work placement arrangements. Since early 2018 the college has now entered into formal agreements with these organisations to create the Killester College Internship Programme. These companies are currently providing students with work placement one day per week. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation (UDA) is a transfer learning task where the annotations of the source domain are available, but only have access to the unlabelled target data during training. Previous methods minimise the domain gap by performing distribution alignment between the source and target domains, which has a notable limitation, i.e., at the domain level, but neglecting the sample-level differences, thus preventing the model from achieving superior performance. To solve this, we improve the UDA task with an inter-domain sample-level matching scheme. We apply the widely-used Triplet loss to match the inter-domain samples. To reduce the catastrophic effect of the inaccurate pseudo-labels generated during training, we propose a novel uncertainty measurement method and use this uncertainty to select reliable pseudo-labels automatically. As the selection is uncertainty-aware, the pseudo labels are progressively refined as the training is performed. We apply the advanced Gumbel Softmax technique to realise an adaptive Top-k scheme to achieve adaptive selection. To enable the global ranking optimisation within one batch for the domain matching, the whole model is optimised via a reinforced attention mechanism, using the Average Precision (AP) of the domain matching as the reward. Our model AdaTriplet-RA achieves State-of-the-art results on several public benchmark datasets, and its effectiveness is validated via comprehensive ablation study. Our method improves the accuracy of the baseline by 9.7% (using ResNet-101 as the backbone network) and 6.2% (ResNet-50) on the VisDa dataset and 4.22% (ResNet-50) on the DomainNet dataset. The source code is publicly available at: https://github.com/shuxy0120/AdaTriplet-RA. • An uncertainty-aware Triplet loss to refine the pseudo-labels progressively in domain matching, facilitating the domain adaptation. • A novel Top-k scheme for uncertainty-aware sample selection in domain matching. • A novel reinforced attention mechanism algorithm to enhance the feature representation. • The proposed method significantly improves the performance on several widely applied benchmarks. [ABSTRACT FROM AUTHOR]

Background: As the Internet of Things (IoT) has become more prevalent in recent years, digital twins have attracted a lot of attention. A digital twin is a virtual representation that replicates a physical object or process over a period of time. These tools directly assist in reducing the manufacturing and supply chain lead time to produce a lean, flexible, and smart production and supply chain setting. Recently, reinforced machine learning has been introduced in production and logistics systems to build prescriptive decision support platforms to create a combination of lean, smart, and agile production setup. Therefore, there is a need to cumulatively arrange and systematize the past research done in this area to get a better understanding of the current trend and future research directions from the perspective of Industry 4.0. Methods: Strict keyword selection, search strategy, and exclusion criteria were applied in the Scopus database (2010 to 2021) to systematize the literature. Results: The findings are snowballed as a systematic review and later the final data set has been conducted to understand the intensity and relevance of research work done in different subsections related to the context of the research agenda proposed. Conclusion: A framework for data-driven digital twin generation and reinforced learning has been proposed at the end of the paper along with a research paradigm.

Internet of Things (IoT) has revolutionized the automation of a network of devices utilizing the functions of the individual nodes of sensors and devices connected to the network. Without other external factors to consider, the number of nodes of devices in the network is directly proportional to the energy requirement and consumption. This research presents the development of an abstract model to predict and reduce energy consumption in an IoT ecosystem, based on a reinforcement learning approach. To measure the potential improvement, two devices were deployed in an IoT ecosystem - one was set up with the default configuration, and the second one with a designed heuristic and an integrated reinforcement learning model. Using the second setup, the average daily consumption was reduced by 19.92W, from 2,841.35W to 2,821.43W. This difference is statistically significant (p < 0.001). The node-based savings in energy consumption can collectively bring larger energy savings when applied to all nodes across an IoT ecosystem. [ABSTRACT FROM AUTHOR]

In this study, Big Data Analytics has been applied to implement smart manufacturing services performed by local commercial laundry Small and Medium Sized Enterprises (SMEs), which, to be specific, is called Smart Laundry Services (SLSs). This laundry service first uses traffic big data inputs from the City Traffic monitoring Server and abstract patterns to develop dynamic optimal scheduling routes for logistic terminals after taking pick-up orders from hotel supply managers. On the other hand, the laundry floor, which isfor fully automatic operation through integrated smart manufacturing with reinforced learning algorithms, will automatically classify the arriving orders according to the Radio Frequency Identification (RFID) tags on the orders, which contain data for each order such as material type, forbidden policies, detergent suggestion, and other relevant specifications. The orders are immediately processed and assigned to a laundry terminal with least waiting time and with optimal laundry parameters learned by the algorithm. The processed orders are then shipped back to the hotels by static scheduling routes with the optimal patterns estimated from real-time traffic big data. The entire operation, from initial orders to dropping back orders, are monitored by hotel supply managers and laundry SMEs' personnel on their computers or mobile devices such as cell phones via an APP. As a result, this proposed integrated Smart Manufacturing architecture characterized with cutting-edge technologies such as big data analytics and intelligent production will make the commercial laundry SMEs more competitive in local markets. [ABSTRACT FROM AUTHOR]

During the control of reaching movements, a key contribution of the visual system is the localization of relevant environmental targets. In motor adaptation processes, the visual evaluation of effector motor behavior enables learning from errors, which demands continuous visual attentional focus. However, most current adaptation paradigms include static targets; therefore, when a learning situation develops in a highly variable environment and there is a double demand for visual resources (environment and motor performance), the evolution of learning processes is unknown. In order to understand how learning processes evolve in a variable environment, a video game task was designed in which subjects were asked to manage a 60° counterclockwise‐rotated cursor to capture descending targets with initially unpredictable trajectories. During the task, the cursor and eye movements were recorded to dissect visuomotor coordination. We observed that the pursuit of the targets conditioned a predominant and continuous visual inspection of the environment instead of the rotated cursor. As learning progressed, subjects exhibited a linear reduction in directional error and selected a motor strategy based on the degree of reward, which improved the performance. These results suggest that when the environment demands high visual attention, error‐based and reinforced motor learning processes are implemented simultaneously, thus enabling efficient predictive behavior. We designed a videogame task to understand how error and reward‐based learning processes evolve in a variable environment. Our results show a decrease of directional error in parallel to a selection of a motor strategy based on the degree of reward, which improved the performance. These results suggest that when the environment demands high visual attention, error‐based and reinforced motor learning processes are implemented simultaneously, thus enabling the efficient predictive behavior. [ABSTRACT FROM AUTHOR]

This paper discusses the development of a coupled Q-learning/fuzzy control algorithm to be applied to the control of solar domestic hot water systems. The controller brings the benefit of showing performance in line with the best reference controllers without the need for devoting time to modelling and simulations to tune its parameters before deployment. The performance of the proposed control algorithm was analysed in detail concerning the input membership function defining the fuzzy controller. The algorithm was compared to four standard reference control cases using three performance figures: the seasonal performance factor of the solar collectors, the seasonal performance factor of the system and the number of on/off cycles of the primary circulator. The work shows that the reinforced learning controller can find the best performing fuzzy controller within a family of controllers. It also shows how to increase the speed of the learning process by loading the controller with partial pre-existing information. The new controller performed significantly better than the best reference case with regard to the collectors’ performance factor (between 15% and 115%), and at the same time, to the number of on/off cycles of the primary circulator (1.2 per day down from 30 per day). Regarding the domestic hot water performance factor, the new controller performed about 11% worse than the best reference controller but greatly improved its on/off cycle figure (425 from 11,046). The decrease in performance was due to the choice of reward function, which was not selected for that purpose and it was blind to some of the factors influencing the system performance factor.

An approach for the binding problem is proposed, based on an autonomous adaptive system designed using artificial neural networks with object handling functions. Object handling functionality, such as object files, has been reported to have a relationship with perception, and working memory. However, in order for a brain-oriented system to decide actions based on object handling, the system must clarify the “binding problem”, or the problem of processing different attributes such as shape, color and location in parallel, then binding these multiple attributes as a single object. The proposed system decides semi-optimum actions by combining nonlinear programming and reinforced learning. By the introduction of artificial neural networks based on dendritic structures of pyramidal neurons in the cerebral cortex, together with a mechanism for dynamically linking nodes to objects, it is shown that deciding actions and learning as a whole system, based on binding object attributes and location, is possible. The proposed features are verified through computer simulation results. [ABSTRACT FROM AUTHOR]

This paper presents a study of the effect of regret on route choice behavior when both descriptional information and experiential feedback on choice outcomes are provided. The relevance of Regret Theory in travel behavior has been well demonstrated in non-repeated choice environments involving decisions on the basis of descriptional information. The relation between regret and reinforced learning through experiential feedbacks is less understood. Using data obtained from a simple route-choice experiment involving different levels of travel time variability, discrete-choice models accounting for regret aversion effects are estimated. The results suggest that regret aversion is more evident when descriptional information is provided ex-ante compared to a pure learning from experience condition. Yet, the source of regret is related more strongly to experiential feedbacks rather than to the descriptional information itself. Payoff variability is negatively associated with regret. Regret aversion is more observable in choice situations that reveal risk-seeking, and less in the case of risk-aversion. These results are important for predicting the possible behavioral impacts of emerging information and communication technologies and intelligent transportation systems on travelers' behavior. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a learning-based model of route-choice behavior when information is provided in real time. In a laboratory controlled experiment, participants made a long series of binary route-choice trials relying on real-time information and learning from their personal experience reinforced through feedback. A discrete choice model with a Mixed Logit specification, accounting for panel effects, was estimated based on the experiment’s data. It was found that information and experience have a combined effect on drivers’ route-choice behavior. Informed participants had faster learning rates and tended to base their decisions on memorization relating to previous outcomes whereas non-informed participants were slower in learning, required more exploration and tended to rely mostly on recent outcomes. Informed participants were more prone to risk-seeking and had greater sensitivity to travel time variability. In comparison, non-informed participants appeared to be more risk-averse and less sensitive to variability. These results have important policy implications on the design and implementation of ATIS initiatives. The advantage of incorporating insights from Prospect Theory and reinforced learning to improve the realism of travel behavior models is also discussed. [Copyright &y& Elsevier]

Energy limitations remain a key concern in the development of Internet of Medical Things (IoMT) devices since most of them have limited energy sources, mainly from batteries. Therefore, providing a sustainable and autonomous power supply is essential as it allows continuous energy sensing, flexible positioning, less human intervention, and easy maintenance. In the last few years, extensive investigations have been conducted to develop energy-autonomous systems for the IoMT by implementing energy-harvesting (EH) technologies as a feasible and economically practical alternative to batteries. To this end, various EH-solutions have been developed for wearables to enhance power extraction efficiency, such as integrating resonant energy extraction circuits such as SSHI, S-SSHI, and P-SSHI connected to common energy-storage units to maintain a stable output for charge loads. These circuits enable an increase in the harvested power by 174% compared to the SEH circuit. Although IoMT devices are becoming increasingly powerful and more affordable, some tasks, such as machine-learning algorithms, still require intensive computational resources, leading to higher energy consumption. Offloading computing-intensive tasks from resource-limited user devices to resource-rich fog or cloud layers can effectively address these issues and manage energy consumption. Reinforcement learning, in particular, employs the Q-algorithm, which is an efficient technique for hardware implementation, as well as offloading tasks from wearables to edge devices. For example, the lowest reported power consumption using FPGA technology is 37 mW. Furthermore, the communication cost from wearables to fog devices should not offset the energy savings gained from task migration. This paper provides a comprehensive review of joint energy-harvesting technologies and computation-offloading strategies for the IoMT. Moreover, power supply strategies for wearables, energy-storage techniques, and hardware implementation of the task migration were provided. [ABSTRACT FROM AUTHOR]

This paper discusses a method in which periodic motion, such as biped walking, is acquired by the framework of reinforced learning. For an object with multiple degrees of freedom, it is in general difficult to acquire the policy by reinforced learning. There must be a framework of learning which is matched to the motion task under consideration. From this viewpoint, this paper proposes “dynamic policy,” that is, a policy with internal state and dynamics. Its learning procedure is then considered. When periodic motion such as biped walking is to be acquired by learning, dynamical entrainment to the control object can be utilized if the policy has internal dynamics. Furthermore, by providing a policy with internal state, the controller is made robust to time delay and noise in the sensor inputs. The use of the policy gradient is discussed as a method of learning the policy without explicitly considering its internal state. As an example, dynamic policy is applied to a three-link biped walking robot model. The dynamic policy in this example is realized by a feedback controller based on a neural oscillator and sensor inputs. By using the policy gradient, appropriate rules for sensor feedback to the neural oscillator are learned by trial and error. It is shown by simulation that a dynamic policy realizing biped walking is acquired. The method is further applied to a five-link biped walking robot in order to investigate the possibility of extension to a system with multiple degrees of freedom. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(4): 25–38, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20441 [ABSTRACT FROM AUTHOR]

A method, named XHJ-method, is proposed in this letter to determine the number of clusters of a data set, which incorporates with the Fuzzy Reinforced Learning Vector Quantization (FRLVQ) technique. The simulation results show that this new method works well for the traditional iris data and an artificial data set, which contains un-equally sized and spaced clusters. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents the development and evaluation of a new approach toward the design of optimised codebooks by vector quantisation (VQ). A strategy of reinforced learning (RL) is proposed which exploits the advantages offered by fuzzy clustering algorithms, competitive learning and knowledge of training vector and codevector configurations. RL is used as a pre-process before using a conventional VQ algorithm such as the generalised Lloyd algorithm (GLA) or the fuzzy k-means (FKM) algorithm. At each iteration of RL, codevectors move intelligently and intentionally toward an improved optimum codebook design. This is distinct from simulated annealing (SA) and genetic algorithm (GA) techniques in which a random variation is introduced in the movement of the codevectors. The new strategy reduces the possibility that, in the final design, codevectors will be overcrowded in the high density distribution regions of the training vector space, and also that too few codevectors will be settled in low density regions. Experiments demonstrate that this results in a more effective representation of the training vectors by the codevectors and that the final codebook is nearer to the optimal solution in applications such as image compression. It has been found that GLA and FKM yield improved quality of codebook design in this application when RL is used as a pre-process. The investigations have also indicated that RL is insensitive to the selection of both the initial codebook and a learning rate control parameter, which is the only additional parameter introduced by RL from the standard FKM. [Copyright &y& Elsevier]

In this study the authors proposed a method for reinforced learning using multistep state estimation based on internal model of control object. It offers a discussion of learning results obtained by application of this model to a vehicle running on a road including various curvatures, for evaluating whether the vehicle would wander off the road. The vehicle's dynamic model acquired by learning was used for estimating rewards and determining the control outputs, based on the estimated states at multiple time intervals. Results of simulation confirmed that safe driving was possible even with road shapes containing nonlearned curvatures and speeds. Since in this control method external information varies depending on the employed control patterns and environmental conditions, it is possible to consider a model with changing view in order to obtain necessary information for driving control. © 2003 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 86(10): 85–95, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.10123 [ABSTRACT FROM AUTHOR]