Search

• Integrated planning for observation and data downlink tasks. • A priority adjustment method for higher revenue rate. • Rapid inference speed for multiple satellites. The autonomous coordination and integrated planning of observation and data downlink missions for the distributed agile Earth observation satellite (AEOS) constellation hold significant importance in practical applications. In order to address this issue, we introduce an abstract universal mission model and present an algorithm rooted in deep reinforcement learning (DRL), termed the Attention-based Distributed Satellite Mission Planning (ADSMP) algorithm, which is designed to generate effective planning solutions. This algorithm employs a neural network that utilizes the attention mechanism, enabling each satellite to independently make decisions with equal intelligence. Furthermore, a mission priority adjustment method is devised to facilitate the coordination of data download and observation scheduling. The ADSMP is trained using the REINFORCE algorithm with Rollout Baseline. By conducting comparative experiments, we demonstrate that the proposed algorithm attains the highest revenue rate in corresponding scenarios, while simultaneously ensuring fast inference speed. [ABSTRACT FROM AUTHOR]

Besides health monitoring, a regular check of dairy heifers' growth rate is desirable, but it is rarely done because procedures that require restraint and handling can be associated with substantial stress for both animals and farmers. Inexperienced heifers, especially if they are highly responsive to humans, may find restraint and handling potentially aversive. This study investigated whether training heifers of different age and responsiveness toward humans (RTH), through operant conditioning, could reduce stress in animals, ease close contact and handling, and be feasible in terms of farmers' effort. We assessed 60 Holstein heifers of 2 age classes (young, n = 29, 291 ± 39 d; old, n = 31, 346 ± 62 d) according to the avoidance distance test and classified them as confident (n = 20), neutral (n = 21), or nonconfident (n = 19). Half of the heifers of each age and RTH class were trained (n = 29), whereas the other half was not (n = 31). The trained heifers were subjected to target training for 8 sessions and positively reinforced with feed to allow being touched on the muzzle, rump, and perineum. If a heifer refused positive reinforcement, the trainer stepped back as negative reinforcement. In the last week of the experiment, the effect of training on the reaction to handling was assessed in all heifers. We measured heart rate, root mean square of successive interbeat interval differences (RMSSD), and fecal cortisol metabolites (FCMet). The presence of behavioral distress signs was recorded as well. The avoidance distance test was performed a second time 24 h after the measuring session. All of the trained heifers, regardless of RTH class, successfully accomplished the target training task in 6 sessions, each spending on average 25.3 s per session. All of the trained heifers allowed touches on the rump and perineum at the end of the fourth session. Training nonconfident heifers required more time compared with the others. Trained heifers showed higher RMSSD than nontrained heifers (14.2 vs. 16.9 ms, respectively), indicating a lower vagal tone, and thus, a slightly lower stress level than nontrained heifers. Training did not lead to differences in HR, FCMet, or presence of stress behavioral signs. Nonconfident heifers had the highest mean baseline FCMet values compared with neutral and confident heifers (38.4 vs. 30.3 vs. 29.1 ng/g, respectively). Nonconfident heifers also showed the lowest value of FCMet 12 h after the measuring session (36.7 vs. 44.6 vs. 49.7 ng/g), likely due to a decreased responsiveness of the adrenal gland to a stressor. The average avoidance distance decreased between the beginning and the end of the experiment, especially for neutral and nonconfident heifers, regardless of whether they were trained or not. These results show how using operant conditioning on some heifers not only decreased their vagal tone, but also reduced the responsiveness to humans of all the animals, trained and not trained; in the latter case, this reduction was through nonassociative learning, such as habituation. [ABSTRACT FROM AUTHOR]

A thorough study was conducted to assess the performance of the strip footing reinforced with geocells in sand, focusing on understanding the enhancement effects and geocell reinforcement mechanisms. Critical factors such as geocell modulus, height, soil relative density and load eccentricity were examined through fully instrumented model tests. Measurements included surface displacement profiles, strains on the geocell layer, subsurface pressure distribution and other relevant parameters. Results revealed that the strip footing on geocell-reinforced sand beds exhibited better performance compared to those on unreinforced soil, characterized by increased load-carrying capacity and reduced settlements. Notably, stiffer geocells improved performance significantly, with a 40% higher modulus enhancing the bearing pressure by up to 25%, due to better confinement and anchorage effects. Conversely, geocells with a lower modulus demonstrated more effective vertical stress distribution. Furthermore, increased geocell height moderately enhanced footing performance by improving confinement, although wall buckling under eccentric loading limited major gains. Dense soils under centric loading exhibited up to a 20% better improvement in bearing pressure than loose soils due to higher strain mobilization within the geocell layer. These findings highlight the crucial role of geocell and soil properties, as well as loading conditions, in optimizing reinforcement effects for strip footings. • Assessed geocell-reinforced strip footings with fully instrumented tests. • Geocell footings outperform unreinforced soils, improving key non-dimensional factors. • Stiffer geocells notably enhance performance due to better confinement and anchorage. • Lower modulus geocells benefit from enhanced vertical stress distribution. • Increased geocell height moderately increased footing performance. [ABSTRACT FROM AUTHOR]

Reinforcement sensitivity is a hypothesized attention-deficit/hyperactivity disorder (ADHD) intermediate phenotype but its role in transgenerational transmission of ADHD-linked psychopathology risk is largely unknown. We examined, in a carefully phenotyped, N = 123 sample of adolescents (M age = 15.27 years, SD = 0.984; 61.78% boys), whether (1) parental psychopathology is differentially associated with fMRI-indexed neural response to reward receipt and (2) both maternal and paternal psychopathology are associated with neural response to reward; across adolescents at-risk for and not at-risk for ADHD. Indices of parental psychopathology were differentially associated with adolescent offspring neural response to reward such that across measures, parental psychopathology was negatively or not associated with offspring superior frontal gyrus (SFG) response to reward receipt in adolescents at-risk for ADHD, but parental psychopathology was positively associated with offspring SFG response in adolescents not at-risk. Further, across measures, greater maternal psychopathology was associated with blunted adolescent SFG response to reward in adolescents at-risk for ADHD whereas greater maternal externalizing problems were linked to enhanced adolescent SFG response in adolescents not at-risk. Across measures, paternal psychopathology was not associated with adolescent response to reward, in either group. ADHD risk confers differential reward-related susceptibility to the effects of parental psychopathology. Results also show this association is nonspecific in terms of parental psychopathology type but is specific to maternal psychopathology. [ABSTRACT FROM AUTHOR]

• Performance of object detection tends to degrade on low-quality images. • Choosing suitable enhancement algorithms can improve the detection performance. • Reinforcement learning can autonomously select algorithms based on the scenario. • Reinforcement learning avoid the issue of data labeling and retraining. The task of object detection for low-quality images in multiple scenarios remains a challenging task. To alleviate this problem, a novel approach to improve the detection performance of the model on low-quality images using deep reinforcement learning is proposed. The approach differs from methods that involve data labeling and retraining, as the detection results of the model are utilized to formulate a reward function. This enables deep reinforcement learning to transform low-quality images into high-quality images through image enhancement, ultimately leading to an improvement in the model's recognition performance on low-quality images. An image enhancement tool chain (IETC) has been prepared, offering a flexible and lightweight algorithm selection for the Dueling Deep Q-Network-based Tool Selector (DDQN-TS). By adding a "Pass" option, Enable DDQN-TS to learn the relative relationship between performing and not performing image enhancement. To mitigate the potential inconsistency between the output of the detection model and the actual situation caused by negative samples, which can lead to incorrect decisions made by DDQN-TS, the introduction of a "thresh" parameter regulates the level of intervention in target detection by DIE. The experimental results demonstrate the effectiveness and portability of the DIE model in environments with noise, fog, and uneven brightness. [ABSTRACT FROM AUTHOR]

Drugs like opioids are potent reinforcers thought to co-opt value-based decisions by overshadowing other rewarding outcomes, but how this happens at a neurocomputational level remains elusive. Range adaptation is a canonical process of fine-tuning representations of value based on reward context. Here, we tested whether recent opioid exposure impacts range adaptation in opioid use disorder, potentially explaining why shifting decision making away from drug taking during this vulnerable period is so difficult. Participants who had recently (<90 days) used opioids (n = 34) or who had abstained from opioid use for ≥ 90 days (n = 20) and comparison control participants (n = 44) completed a reinforcement learning task designed to induce robust contextual modulation of value. Two models were used to assess the latent process that participants engaged while making their decisions: 1) a Range model that dynamically tracks context and 2) a standard Absolute model that assumes stationary, objective encoding of value. Control participants and ≥90-days-abstinent participants with opioid use disorder exhibited choice patterns consistent with range-adapted valuation. In contrast, participants with recent opioid use were more prone to learn and encode value on an absolute scale. Computational modeling confirmed the behavior of most control participants and ≥90-days-abstinent participants with opioid use disorder (75%), but a minority in the recent use group (38%), was better fit by the Range model than the Absolute model. Furthermore, the degree to which participants relied on range adaptation correlated with duration of continuous abstinence and subjective craving/withdrawal. Reduced context adaptation to available rewards could explain difficulty deciding about smaller (typically nondrug) rewards in the aftermath of drug exposure. [ABSTRACT FROM AUTHOR]

• Habit Reversal Therapy (HRT) is an effective behavioral intervention for reducing tics. • The hypothesized premonitory urge-tic cycle and its role in HRT lack support. • A dynamical systems framework can improve tests of the mechanism underlying HRT. • Findings were inconsistent with existing hypotheses regarding HRT change mechanism. • Tic suppression did not disrupt the urge-tic cycle consistently across participants. • Most participants did not experience urge reduction patterns consistent with habituation. Tic disorders are a class of neurodevelopmental disorders characterized by involuntary motor and/or vocal tics. It has been hypothesized that tics function to reduce aversive premonitory urges (i.e., negative reinforcement) and that suppression-based behavioral interventions such as habit reversal training (HRT) and exposure and response prevention (ERP) disrupt this process and facilitate urge reduction through habituation. However, previous findings regarding the negative reinforcement hypothesis and the effect of suppression on the urge-tic relationship have been inconsistent. The present study applied a dynamical systems framework and within-subject time-series autoregressive models to examine the temporal dynamics of urges and tics and assess whether their relationship changes over time. Eleven adults with tic disorders provided continuous urge ratings during separate conditions in which they were instructed to tic freely or to suppress tics. During the free-to-tic conditions, there was considerable heterogeneity across participants in whether and how the urge-tic relationship followed a pattern consistent with the automatic negative reinforcement hypothesis. Further, little evidence for within-session habituation was seen; tic suppression did not result in a reduction in premonitory urges for most participants. Analysis of broader urge change metrics did show significant disruption to the urge pattern during suppression, which has implications for the current biobehavioral model of tics. [ABSTRACT FROM AUTHOR]

• The maximum corrosion inhibition efficiency (97.84 %) was achieved when the MBI/B72 ratio was 3:7. • After MBI/B72 treatment, a denser and smoother membrane layer structure is formed. • The ability of MBI/B72 solution to enhance hardness is somewhere between B72 and MBI. • The penetration properties of this solution are advantageous for the treatment of heavily patinated bronzes. • The corrosion inhibition mechanism was inferred based on the microscopic Raman, IR, and XPS data. The corrosion inhibition effect, reinforcement effect and penetration effect of rusted bronze before and after the compounding of 2-mercaptobenzimidazole (MBI) with B72 (Paraloid B72) were investigated using surface analysis and electrochemical techniques. Fourier transform infrared spectroscopy (FT-IR), Raman spectroscopy (Raman), X-ray photoelectron spectroscopy (XPS) and quantum chemical computing techniques were used to explore the mechanism of corrosion mitigation further. In addition, an electronic universal testing machine was used to test the ability of the material to reinforce different rust compositions before and after compounding. The dynamic potential polarization method was used to determine the composite scheme's optimal corrosion inhibition ratio and study the corrosion inhibition effect. The experimental results showed that the best corrosion inhibition effect was achieved at the ratio of MBI/B72 ratio 3:7, and the corrosion inhibition efficiency was 97.84 % under these conditions. Most of the MBI molecules used by us presented prominent permeation adsorption behavior. They formed uneven films of Cu(I)-MBI and Cu(II)-MBI coordination compounds on the surface of the rust layer. Finally, B72 combines with MBI molecules through hydrogen bonding to form a compact and smooth film, which helps improve corrosion inhibition efficiency and plays a certain role in reinforcement. [ABSTRACT FROM AUTHOR]

The central amygdala (CeA) and bed nucleus of the stria terminalis (BNST) are reciprocally connected nodes of the extended amygdala thought to play an important role in alcohol consumption. Studies of immediate-early genes indicate that BNST and CeA are acutely activated following alcohol drinking and may signal alcohol reward in nondependent drinkers, while stress signaling in the extended amygdala following chronic alcohol exposure drives increased drinking via negative reinforcement. However, the temporal dynamics of neuronal activation in these regions during drinking behavior are poorly understood. In this study, we used fiber photometry and the genetically encoded calcium sensor GCaMP6s to assess acute changes in neuronal activity during alcohol consumption in BNST and CeA before and after a chronic drinking paradigm. Activity was examined in the pan-neuronal population and separately in dynorphinergic neurons. BNST and CeA showed increased pan-neuronal activity during acute consumption of alcohol and other fluid tastants of positive and negative valence, as well as highly palatable chow. Responses were greatest during initial consummatory bouts and decreased in amplitude with repeated consumption of the same tastant, suggesting modulation by stimulus novelty. Dynorphin neurons showed similar consumption-associated calcium increases in both regions. Following three weeks of continuous alcohol access (CA), calcium increases in dynorphin neurons during drinking were maintained, but pan-neuronal activity and BNST-CeA coherence were altered in a sex-specific manner. These results indicate that BNST and CeA, and dynorphin neurons specifically, are engaged during drinking behavior, and activity dynamics are influenced by stimulus novelty and chronic alcohol. • BNST and CeA calcium activity increases during drinking of alcohol and other fluids. • BNST and CeA dynorphin neurons are engaged during alcohol consumption. • BNST and CeA activity is higher when taste stimuli are novel. • BNST and CeA calcium activity during drinking is reduced after chronic alcohol. • BNST-CeA coherence is increased during alcohol drinking bouts in male mice. [ABSTRACT FROM AUTHOR]

In this paper, we propose a frame-part-activated deep reinforcement learning (FPA-DRL) for action prediction. Most existing methods for action prediction utilize the evolution of whole frames to model actions, which cannot avoid the noise of the current action, especially in the early prediction. Moreover, the loss of structural information of human body diminishes the capacity of features to describe actions. To address this, we design a FPA-DRL to exploit the structure of the human body by extracting skeleton proposals and reduce the redundancy of frames under a deep reinforcement learning framework. Specifically, we extract features from different parts of the human body individually, activate the action-related parts in features and the action-related frames in videos to enhance the representation. Our method not only exploits the structure information of the human body, but also considers the attention frame for expressing actions. We evaluate our method on three popular action prediction datasets: UT-Interaction, BIT-Interaction and UCF101. Our experimental results demonstrate that our method achieves the very competitive performance with state-of-the-arts. • We design the part-activated module to enhance the action-related parts of features. • We design the frame-activated module to reduce the redundancy of frames. • We achieved very competitive results of state-of-the-arts on three datasets. [ABSTRACT FROM AUTHOR]

Dairy cattle are often restrained for veterinary procedures, but restraint can cause fear responses that can make the procedure challenging for both the animal and the human handler. Positive reinforcement training (PRT) is used in other species to reduce fear responses and there is now evidence that this can also facilitate handling in cattle. The objectives of this study were to test the effect of PRT on anticipatory and play behavior in dairy heifers. We predicted that heifers trained with PRT would show more anticipatory and play behaviors than control heifers in the period before gaining access to a chute. We used 20 heifers (5 ± 0.6 mo old) that had been habituated to the chute area and had previous experience with handling. Heifers were randomly assigned to 2 treatments: control (n = 10) and PRT (n = 10). Positive reinforcement training heifers were subjected to a training protocol that included standard farm handling techniques, as well as target training with food reinforcement. Control heifers were moved to the chute using standard farm handling techniques only. As predicted, PRT heifers performed more behavioral transitions (7.6 ± 0.77 vs. 4.4 ± 0.57 transitions for control heifers; F 1,9 = 21.99, P < 0.01), and specifically performed more locomotory play such as jumping (2.1 ± 0.30 vs. 0.4 ± 0.19 jumps; F 1,9 = 57.18, P < 0.01) and running (2.0 ± 0.40 s vs. 0.5 ± 0.16 s; F 1,9 = 20.73, P < 0.01). These results indicate that PRT results in heifers having a more positive emotional state in anticipation of handling, and support the use of training to improve the welfare of dairy cattle. [ABSTRACT FROM AUTHOR]

Surface topography modification is a common post-processing step used to enhance surface quality or to add micro/nano surface structures for various functionalities. Laser remelting (LRM) is a new technology aimed to resolve some of the shortcomings of traditional post-processing technologies. In LRM, a high-energy laser beam melts the top surface and offers control of the molten material flow through the control of laser power, scanning speed, melt pool size, and scan trajectory. By controlling the molten material flow, surface polishing by laser remelting (SP-LRM) can be used to produce finishes characterized by areal arithmetical mean height - Sa < 0.1 µm as well as surface microstructures. LRM is faster than traditional methods, creates no harmful byproducts, and allows for polishing high aspect ratio freeform surfaces without harming surrounding areas. The primary limiting factor for wide-rage industrial use of the LRM is represented by thermodynamic process instabilities that are associated with rapid melting and solidification. For instance, deep trenches or grinding marks in the initial surface can cause small deviations of local surface absorptivity to lead to laser melt pool instabilities. The high-speed nature of the LRM process combined with the complex thermodynamic phenomena makes process instabilities difficult to model, detect, predict, and control. This study aimed to use reinforcement learning (RL) in order to build a 'self-learning' controller capable of adapting to any material and control instabilities on-line and in real-time. Along these lines, the study will detail a preliminary application of two RL implementations involving SP-LRM performance simulation to demonstrate the implementation of a double dueling deep Q-network (DDDQN) controller for melt pool temperature distribution control. The results obtained showed that the 3-action implementation achieved 48% less variance and 35% increase in the mean value of the total reward collected compared with the 31-action implementation. [ABSTRACT FROM AUTHOR]

• Emotions play an important role in shaping behaviors within public goods games with and without punishment opportunities. • In standard public goods games, the emotion of anger and having contributed more than others are correlated, and agents adjust their contribution downward overtime to possibly avoid negative emotions. • In public goods games with monetary punishment, saction fosters future cooperation without relying on emotional impact. • In public goods games with non-monetary punishment, sanctioning improves sanctioner's emotion and deteriorates receivers' emotions. • Disgust, rather than angry as conventionally thought explains the sanctioning decision. We consider the emotional correlates of activity in the Public Good game when monetary and non-monetary punishments are available, and when no punishment is possible. In our experiment, emotions are measured using Face Reading software that tracks the emotional content of facial expressions in real time. When no punishment is possible, greater anger and more negative emotional valence correlate with learning that one has contributed more than others. Lower valence and happiness, in turn, are associated with reducing one's cooperation in the next period. When non-monetary punishment in the form of expressed disapproval is possible, positive emotional valence is associated with cooperation, punishment of free-riders, and an increase in cooperation from one period to the next. Negative valence, on the other hand, is associated with the receipt of punishment, suggesting that the expression of disapproval inherent in the non-monetary punishment was well understood and had an effect on the emotions of the recipient. The data support the conjecture that the reinforcement that positive emotion provides is what allows non-monetary punishment to increase cooperation. In contrast, when monetary punishment is available, emotional correlates are less consistent, suggesting that monetary punishment is less reliant on emotions to be effective. [ABSTRACT FROM AUTHOR]

With the rapid development of high-precision ultra-stable spacecraft, the absolute values of coefficients of thermal expansion (CTE) of carbon fiber reinforced SiC matrix composite (C/SiC) are required to be closer to 10−7 K−1 or even lower. In this work, high-modulus and low-expansion M55J carbon fiber was selected to replace T300 carbon fiber as the reinforcement to decrease the CTE of C/SiC. The interfacial region, pore distribution and microstructure evolution during the densification process were investigated. With the replacement of T300 carbon fiber by M55J carbon fiber as the reinforcement, axial thermal residual stress in SiC matrix can increase to 378 ± 26 MPa, resulting in the generation of matrix microcracks. This can reduce the constraint of matrix to carbon fiber and provide expansion space for M55J-C/SiC. Therefore, the average in-plane technical CTE of C/SiC in the range of −20 to 50 °C can be reduced from 1.09 × 10−6 to 0.34 × 10−6 K−1. [ABSTRACT FROM AUTHOR]

The absence of effective quality prediction methods for functional composite materials (FCMs) produced by laser additive remanufacturing (LARM) hampers their application due to the complex cross-scale defects, including surface cracks and thermal damage to the internal reinforcement phase. This paper presents a multi-task deep learning-empowered digital twin for predicting visible and invisible defects in the fabricating process of FCMs. The dimensions of FCM trajectory, thermal damage to the reinforcement phase, and forming cracks were predicted via a parallel multi-task deep learning model. The dynamic visualization of the digital twin is realized through cross-sectional modeling and provides an intuitive and effective perception for monitoring the process. [ABSTRACT FROM AUTHOR]

Human-Robot Collaboration (HRC) has emerged as a pivot in contemporary human-centric smart manufacturing scenarios. However, the fulfilment of HRC tasks in unstructured scenes brings many challenges to be overcome. In this work, mixed reality head-mounted display is modelled as an effective data collection, communication, and state representation interface/tool for HRC task settings. By integrating vision-language cues with large language model, a vision-language-guided HRC task planning approach is firstly proposed. Then, a deep reinforcement learning-enabled mobile manipulator motion control policy is generated to fulfil HRC task primitives. Its feasibility is demonstrated in several HRC unstructured manufacturing tasks with comparative results. [ABSTRACT FROM AUTHOR]

In this paper, a new off-policy two-dimensional (2D) reinforcement learning approach is proposed to deal with the optimal tracking control (OTC) issue of batch processes with network-induced dropout and disturbances. A dropout 2D augmented Smith predictor is first devised to estimate the present extended state utilizing past data of time and batch orientations. The dropout 2D value function and Q-function are further defined, and their relation is analyzed to meet the optimal performance. On this basis, the dropout 2D Bellman equation is derived according to the principle of the Q-function. For the sake of addressing the dropout 2D OTC problem of batch processes, two algorithms, i.e., the off-line 2D policy iteration algorithm and the off-policy 2D Q-learning algorithm, are presented. The latter method is developed by applying only the input and the estimated state, not the underlying information of the system. Meanwhile, the analysis with regard to the unbiasedness of solutions and convergence is separately given. The effectiveness of the provided methodologies is eventually validated through the application of a simulated case during the filling process. • The 2D model-free control method can solve a dropout 2D OTC problem for the batch processes. • The 2D Q-learning method can find the optimal gain of the control law better. • The analysis of unbiasedness and convergence are given by the rigorous theoretical proof. • The proposed method for dropout OTC has the merit to exempt the effect of the probing noises and to produce unbiased solution. [ABSTRACT FROM AUTHOR]

Habenula (HB) function is implicated in substance use disorders and is involved in inhibiting dopamine release in the ventral striatum (VS). While blunted VS reward responsivity is implicated in risk for later substance use, links between HB reinforcement processing and progression of use have not, to our knowledge, been examined among adolescents. In the present study, we longitudinally assessed HB and VS responsivity to social rewards and punishments across adolescence and examined associations with substance use. Within a longitudinal design, 170 adolescents (53.5% female) completed 1 to 3 functional magnetic resonance imaging scans across 6th to 9th grade and reported yearly substance use across 6th to 11th grade. We examined VS and HB responsivity to social reinforcement during a social incentive delay task in which adolescents received social rewards (smiling faces) and punishments (scowling faces). We observed increased VS responsivity to social rewards (vs. reward omissions) and increased VS, but decreased HB, responsivity to social punishment avoidance versus receipt. However, contrary to hypotheses, the HB displayed increased responsivity to social rewards (vs. reward omissions). Further, adolescents reporting regular substance use displayed longitudinally declining HB responsivity to social rewards (vs. reward omissions), whereas adolescents reporting no substance use displayed longitudinally increasing HB responsivity. In contrast, whereas VS responsivity to punishment avoidance versus receipt increased longitudinally among regular substance users, it stayed relatively stable among nonusers. These results suggest that differential HB and VS social reinforcement processing trajectories across adolescence are associated with substance use. [ABSTRACT FROM AUTHOR]

Effective exploration is the key to achieving high returns for reinforcement learning. Agents must explore jointly in multi-agent systems to find the optimal joint policy. Due to the exploration problem and the shared reward, the policy-based multi-agent reinforcement learning algorithms face policy overfitting, which may lead to the joint policy falling into a local optimum. This paper introduces a novel general framework called Learning Joint-Action Intrinsic Reward (LJIR) for improving multi-agent reinforcement learners' joint exploration ability and performance. LJIR observes agents' state and joint actions to learn to construct an intrinsic reward online that can guide effective joint exploration. With the novel combination of Transformer and random network distillation, LJIR selects the novel states to give more intrinsic rewards, which help agents find the best joint actions. LJIR can dynamically adjust the weight of exploration and exploitation during training and keep the policy invariance finally. To ensure LJIR seamlessly adopts existing MARL algorithms, we also provide a flexible combination method for intrinsic and external rewards. Empirical results on the SMAC benchmark show that the proposed method achieves state-of-the-art performance in challenging tasks. [ABSTRACT FROM AUTHOR]

Cannabis is increasingly consumed and increasingly perceived as harmless. Among those whose use develops into a cannabis use disorder (CUD), <5% initiate and engage in treatment. Thus, novel options for low-barrier, appealing treatments are needed to foster engagement in care. We conducted an open trial of a telehealth-delivered multicomponent behavioral economic intervention for non-treatment-engaged adults with CUD. Participants with CUD were recruited from a health system and screened for eligibility. Participants completed behavioral economic indices (cannabis demand, proportionate cannabis-free reinforcement), measures of cannabis use and mental health symptoms, and provided open-ended feedback on the intervention experience. Of the 20 participants who enrolled and engaged in the initial intervention session, 70% (14 out of 20) completed all intervention components. All participants were satisfied/very satisfied with the intervention and 85.7% reported the telehealth delivery made it at least slightly easier/more likely for them to receive substance use care. Baseline to immediate post-treatment, behavioral economic cannabis demand decreased (intensity: Hedges' g = 0.14, maximum total expenditure: Hedges' g = 0.53, maximum expenditure for a single hit: Hedges' g = 0.10) and proportionate cannabis-free reinforcement (Hedges' g = 0.12) increased. Past-month total cannabis use decreased by 8.9% from baseline to post-treatment (Hedges' g = 0.39), along with decreases in recent depression (Hedges' g = 0.50) and anxiety symptoms (Hedges' g = 0.29). These preliminary findings suggest that this behavioral economic intervention was highly acceptable and feasible for adults with untreated CUD. Changes in potential mechanisms of behavior change (cannabis demand, proportionate cannabis-free reinforcement) were consistent with reduced frequency of cannabis use and improved mental health outcomes. • The vast majority of people with a cannabis use disorder receive no treatment. • Novel approaches are critically needed for people with cannabis use disorder. • Behavioral economic mechanisms may be a novel target to reduce use and harms. • A telehealth-delivered behavioral economic intervention was feasible and acceptable. • Preliminary outcomes support additional evaluation. [ABSTRACT FROM AUTHOR]

The performance of recognition systems can be significantly affected by various factors, with facial expression poses and lighting changes being the main confounding factors. In order to minimize their impact, we propose an intriguing method in this study that enables the generation of high-quality images specifically tailored to the target domain. Our objective is to utilize disentangled representation to effectively model the decomposition of data variations and generate neutral facial expression images with frontal posture and adaptive illumination. To achieve this, we incorporate 3D priors in the adversarial learning during the training process, simulating the generation of an analytical 3D face deformation as well as rendering operations. Additionally, we employ contrastive learning to control the disentanglement of the generated faces while preserving the essential properties of facial features. This technique enables us to learn an embedding space in which similar data samples are represented closely, while distinct samples are kept far apart from each other. Furthermore, we conduct an analysis of the learned latent space and introduce several other significant properties that enhance the reinforcement of factor disentanglement. These properties include an imitation learning algorithm, which facilitates the acquisition of meaningful patterns and characteristics. [ABSTRACT FROM AUTHOR]

It is important to understand the relationship between stress and problematic use of social media (PUSM). However, no study to our knowledge has yet investigated the longitudinal relationship between perceived stress and PUSM via positive and negative reinforcement processes. The present study investigated relationships between COVID-19-pandemic-related stress and PUSM and possible moderating effects of motives for using social media (positive and/or negative reinforcement) during and following a COVID-19-pandemic-related lockdown. Six-hundred-and-sixty participants initially completed a survey including self-report measures of PUSM, COVID-19-pandemic-related stress, and motives for using social media (i.e., for negative reinforcement involving coping and conformity or positive reinforcements involving enhancement and social motives). During the COVID-19 outbreak recovery period, 117 participants again completed the survey. Bayesian analyses revealed that PUSM was associated with higher COVID-19-pandemic-related stress levels and use of social media for coping, conformity, and enhancement purposes. Longitudinally, PUSM symptom worsening was associated with increased use of social media for coping motives regardless of levels of perceived stress. Use of social media for conformity and enhancement purposes moderated relationships between stress levels during lockdown and PUSM symptoms worsening after lockdown. Our findings corroborate the hypothesis that negative reinforcement processes may be key factors in PUSM symptom worsening regardless of perceived stress. Concurrently, high levels of stress may worsen PUSM through positive reinforcement processes. • Stressful life events negatively impact problematic use of social media (PUSM). • Negative reinforcement processes relate prospectively to PUSM symptoms. • Positive reinforcement mechanisms moderate the relation between stress and PUSM. [ABSTRACT FROM AUTHOR]

Negative symptoms are closely related to the poor prognosis of schizophrenia, for which there is no effective treatment to date. Behavioral activation (BA), which is an effective treatment for depression, is a behavioral approach that targets low levels of response-contingent positive reinforcement. This study aimed to explore BA as an effective intervention for relieving the negative symptoms of schizophrenia. This was a randomized single-blind controlled trial. Eighty-four patients with schizophrenia were enrolled in community mental health settings. Excluding 14 patients who opted out of the study, 70 were randomly assigned to receive BA in addition to treatment-as-usual (BA + TAU) or treatment-as-usual (TAU) only. Negative symptoms were assessed using the Clinical Assessment Interview for Negative Symptoms (CAINS) and Brief Negative Symptom Scale (BNSS) at baseline, post-treatment, and 6-months follow-up. Significant differences between the BA + TAU and TAU only groups were observed in the measures of negative symptoms post-treatment. The total score of CAINS was significantly decreased after BA treatment (η2 = 0.13). The tendency of the BA + TAU treatment effect was also observed for the BNSS total score and PANSS negative symptom subscale (η 2 = 0.10 and η2 = 0.11, respectively). However, the difference between the two groups was not sustained at the six-month follow-up. Our findings suggest that BA could be a promising time-limited and structured psychosocial intervention for schizophrenia-associated negative symptoms with the merit of easy dissemination. Further studies are needed to examine the factors involved in sustaining improvement. [ABSTRACT FROM AUTHOR]

We report the results of incentivizing students to exercise. We compare a no-incentive control to a per-visit payment and two intermittent incentive schemes: monetary rewards at increasing intervals and monetary rewards with unpredictable timing. Based on the psychology literature, we predicted that irregular incentives would facilitate the maintenance of reinforced behavior over the long run. In line with this prediction, we found that although all incentive schemes worked well during the incentivized period, only the two intermittent schemes were more effective than the control after incentives were removed and over an extended period of time. These results suggest an innovative way to incentivize individuals, which sets out to improve the long-term success of interventions that encourage the formation of habits. [ABSTRACT FROM AUTHOR]

1. Understand the importance for standardized templates in the EMR for documentation and retrieval of serious illness/goals of care conversations. 2. Identify processes that can help sustain behavior change at a systems level. Prompting providers on high-quality Serious Illness Conversation elements through use of a smartphrase in Electronic Medical Record is an easily implementable approach to enhance self-perceived quality of these conversations in a time and resource poor environment. Serious Illness Conversation Guide (SICG) is an evidence-based framework developed as part of Ariadne Labs' Serious Illness Care Program for high-quality conversations. These serious illness (SIC) conversations are crucial to patient safety, satisfaction, and goal-concordant care. Despite the availability of an electronic medical record (EMR)-embedded SmartPhrase based on the SICG framework to document these conversations at our institution, usage in neurocritical care unit was poor (once in 8 weeks) and provider satisfaction with workflow around family meetings was low. Increase use of SICG SmartPhrase and measure the impact. Thirty-six providers (attendings and advanced practice providers) received education about the SmartPhrase via virtual group sessions, tip sheets and videos, e-mail/in-person reminders, and unit flyers. Reinforcement through behavioral nudges (i.e., peer-normative comparison feedback and gift card incentives) was provided during 8-week intervention period. Quantitative data on SmartPhrase usage was collected through EMR reports and manual chart review. Pre- and post-intervention surveys were administered for provider perspectives and feedback. SmartPhrase usage increased to 46 in 8-week intervention period (baseline 1; 4,500% increase). In post-intervention provider survey (n=24), 91.7% reported that using the structured template improved self-perceived SIC quality and patient care, and 100% agreed it was easy to use. Increased use in two non-intervention units suggested a halo effect (from 0 to 29 and 0.67 to 2). Six-month follow-up showed continued use above baseline in all three ICUs suggesting sustainability. These findings suggest that by prompting providers on high-quality SIC elements through system smartphrase, the provider-perceived quality of these conversations increased without time and resource-intensive skills training program. Behavioral nudges were important in overcoming initial hesitancy but ease of use, standardized documentation to offer conversation details and easy retrievability of previous conversations, in addition to improved provider satisfaction with quality, resulted in sustainability of intervention. Quality Improvement / Communication [ABSTRACT FROM AUTHOR]

Background Food reinforcement or one's motivation to eat may be established early in life; it might not be the food reinforcement per se that drives weight gain, but rather the imbalance between food and nonfood reinforcement. Objectives We implemented a 2-y randomized, controlled trial to assess the effects of a music enrichment program (music, n  = 45) compared with an active play date control (control, n  = 45) in 9- to 15-mo-old healthy infants who were strongly motivated to eat. Methods The 12-mo intensive intervention phase included 4 semesters of Music Together® or a play date program (Winter, Spring, Summer, and Fall), comprised of once per week group meetings, followed by a 12-mo maintenance phase with monthly meetings. Parents were encouraged to listen to the Music Together® program CD or play with the play date group's toy with their infants at home, respectively. We performed a modified intention-to-treat analysis using all randomly assigned, non-excluded subjects for the outcome measures [relative reinforcing value of food (RRVfood), food reinforcement, music reinforcement, and weight-for-length z -score (WLZ)]. Results There were significant differential group changes across time for RRVfood (group × month; P  = 0.016; Cohen's f2 = 0.034). The music group had significantly greater RRVfood decreases than the control group from baseline to the end of the intensive intervention phase (music change = −0.211; control change = −0.015; P  = 0.002; Cohen's D  = 0.379). However, these differences were not maintained during months 12–24 (music change = −0.187; control change = −0.143; P  = 0.448; D  = 0.087). We observed an overall moderation effect by sex for food reinforcement and WLZ. Boys in the music group had a significant attenuation in food reinforcement and WLZ compared with boys in the control group. Conclusions This study extends our knowledge in infant eating behavior by providing insight into the role of nonfood alternatives in altering one's motivation to eat. There may be sex differences in altering one's motivation to eat. This trial was registered at clinicaltrials.gov as NCT02936284. [ABSTRACT FROM AUTHOR]

Past research has shown that Intolerance of Uncertainty (IU) affects Pavlovian fear conditioning processes. In particular, extinction of learned fear is delayed in those reporting high IU. Reports of differences during acquisition are less consistent with most of the studies reporting no evidence for effects of IU. This may be due to past studies' focus on first interval electrodermal responses or fear potentiated startle, rather than on indices that may better capture uncertainty – like the response to the absence of a probabilistic unconditional stimulus. The current analysis combined data across three experiments that employed a 50 % reinforcement schedule and assessed electrodermal responses and (in two experiments) ratings of conditional stimulus pleasantness. Participants scoring high on IU showed overall larger electrodermal first interval responses during habituation and acquisition but did not differ from those scoring low on IU in differential conditioning (the difference between CS+ and CS-), as indicated by electrodermal first or second interval responses or ratings of CS pleasantness. However, participants high in IU showed larger differential third interval electrodermal responses to the omission of the electro-tactile unconditional stimulus during acquisition. Some evidence for this difference emerged in each experiment, supporting the reliability of the result. The current results suggest that effects of IU emerge in conditions of high uncertainty in Pavlovian fear learning tasks, such as during the omission of probabilistic unconditional stimuli. • Intolerance of Uncertainty (IU) has been shown to affect fear conditioning in particular during extinction. • Past research has yielded mixed effects for IU during fear acquisition. • Participants high in IU show larger electrodermal responses to the omission of a probabilistic unconditional stimulus. [ABSTRACT FROM AUTHOR]

Mobile charging is feasible to deal with the energy-constrained problem in wireless rechargeable sensor networks (WRSNs). The mobile chargers (MCs) are usually employed to charge the sensors sequentially according to the charging schemes. Existing studies assume that each sensor should be charged to its maximum energy capacity or to a fixed upper threshold before the next one can be charged. However, they neglect to control the charging time adaptively of each sensor according to the charging demand. Hence, in this paper, we assume that the charging time of each sensor can be controlled, and we study the joint optimization of charging sequence and charging time problem (JCSCT). Correspondingly, we propose a novel deep reinforcement learning with hybrid action space approach for JCSCT (DRLH-JCSCT), which utilizes deep q-network (DQN) to generate the charging sequence, and adopts deep deterministic policy gradient (DDPG) to determine the charging time. An attention-based encoder–decoder model is integrated in the actor network of DDPG, and a modified bi-directional gate recurrent unit network (MBGRU) is utilized as the decoder. We also design a novel reward function to evaluate the quality of the charging actions. Simulations demonstrate the improved charging performance of the proposed approach, with a longer network lifetime and fewer failed sensors compared with the existing mobile charging scheduling approaches. [ABSTRACT FROM AUTHOR]

A novel method, the Pareto Envelope Augmented with Reinforcement Learning (PEARL), has been developed to address the challenges posed by multi-objective problems, particularly in the field of engineering where the evaluation of candidate solutions can be time-consuming. PEARL distinguishes itself from traditional policy-based multi-objective Reinforcement Learning methods by learning a single policy, eliminating the need for multiple neural networks to independently solve simpler sub-problems. Several versions inspired from deep learning and evolutionary techniques have been crafted, catering to both unconstrained and constrained problem domains. Curriculum Learning (CL) is harnessed to effectively manage constraints in these versions. PEARL's performance is first evaluated on classical multi-objective benchmarks. Additionally, it is tested on two practical PWR core Loading Pattern (LP) optimization problems to showcase its real-world applicability. The first problem involves optimizing the Cycle length (L C) and the rod-integrated peaking factor (F Δ h) as the primary objectives, while the second problem incorporates the mean average enrichment as an additional objective. Furthermore, PEARL addresses three types of constraints related to boron concentration (C b), peak pin burnup (B u m a x ), and peak pin power (F q). The results are systematically compared against conventional approaches from stochastic optimization. Notably, PEARL, specifically the PEAL-NdS (▪) variant, efficiently uncovers a Pareto front without necessitating additional efforts from the algorithm designer, as opposed to a single optimization with scaled objectives. It also outperforms the classical approach across multiple performance metrics, including the Hyper-volume. Future works will encompass a sensitivity analysis of hyper-parameters with statistical analysis to optimize the application of PEARL and extend it to more intricate problems. [ABSTRACT FROM AUTHOR]

Optimal power flow (OPF) is a crucial task in power system management and control; accurate and time-efficient solutions for OPF are necessary to ensure cost-efficient and reliable power system operation. We introduce a novel solution to solving alternating current OPF (ACOPF), a nonlinear and nonconvex optimization problem, by combining the speed of a trained deep learning model with the accuracy of iterative solvers. The proposed framework uses a graph neural network (GNN) to exploit the graph structure of a power system in conjunction with proximal policy optimization, a deep reinforcement learning algorithm, to compute initial guesses for an interior point solver (IPS), providing a warm start, allowing the solver to converge in fewer iterations. Existing literature that explores warm start ACOPF solutions using machine learning choose to compute initial guesses that are trained to be feasible and cost-minimizing. Our approach trains the GNN-based reinforcement learning agent to produce an output that minimizes IPS convergence time by designing a reward function that is a function of the IPS convergence time. We evaluate the proposed framework using IEEE test case environments, using PyPower's IPS-based ACOPF solver and a GNN-based framework that computes ACOPF solutions directly as baselines, demonstrating significantly improved convergence times. • Initial guesses for interior point methods significantly affect their convergence properties. • Graph neural networks are advantageous for optimal power flow. • A loss function that considers interior point solver computation time can help compute better initial guesses. [ABSTRACT FROM AUTHOR]

Problematic pornography use (PPU) is characterized by compulsive and uncontrolled pornography consumption due to a disordered reward processing system. Past studies have shown that reward involves multiple stages, but the connection between these specific stages and PPU remains unclear. In this research, an electrophysiological incentive delay task (IDT) was employed, using both erotic and monetary incentives as rewards, to decompose the distinct phases of both anticipatory and consummatory aspects of reward processing. ERP components linked to reward anticipation (cue-P3 and contingent negative variation, CNV), as well as reward feedback (reward positivity; RewP and feedback-P3; fb-P3), were investigated in a group of 31 individuals with PPU and 33 control participants. In contrast to the control group, who reacted faster to monetary cues, individuals with PPU responded more quickly to erotic cues. RT differences between erotic and monetary cues correlated with CNV amplitudes during anticipation, although no significant CNV differences across groups. In the consummatory phase, controls had higher RewP for large-monetary than large-erotic rewards, not seen in the PPU group. Conversely, the PPU group had increased fb-P3 for large-erotic over large-monetary rewards, a trend absents in controls. Our findings indicate an imbalanced sensitivity in PPU to erotic versus non-erotic rewards, evident in both the anticipatory and consummatory aspects of reward processing. This imbalance could lead to a preference for erotic rewards, possibly fostering addictive sexual behaviors. • Anticipatory and consummatory reward mechanisms involve several distinct phases. • ERP can differentiate between these specific phases of reward processing. • Abnormalities in reward processing in PPU are specific to certain stages. • PPU impacts both the anticipatory and consummatory aspects of reward processing. [ABSTRACT FROM AUTHOR]

This study aimed to quantify preferences for the characteristics of a financial incentives program that would motivate adolescent engagement in type 1 diabetes (T1D) self-care. We performed a discrete choice experiment with 12–18 year-olds with T1D from two pediatric hospital endocrinology clinics (n = 317). We identified key attributes of incentives: (1) monthly value of the reward, (2) payment structure, and (3) difficulty of incentivized behaviors. In twelve choice questions, adolescents chose the incentive option from a pair of profiles that was more likely to motivate them to increase adherence to recommended self-care. Options presented were tailored to adolescents' T1D technology use and perceived difficulty of completing each behavior. We analyzed data using a conditional logit model. The value of the reward accounted for 60.8% of preferences. Adolescents were willing to accept lower value rewards when incentive payments used positive vs. negative reinforcement (−$10.88 (95% CI: −$12.60, −9.24)) and preferred higher incentives for performing hard vs. easier behaviors (+$14.92 (95% CI: +$12.66, +$17.28)). Stated preferences can inform intervention design. Future research will evaluate the external validity of the discrete choice experiment-informed intervention design by assessing adolescent health and behavioral outcomes in a randomized controlled trial. [ABSTRACT FROM AUTHOR]

This paper uses reinforcement learning techniques to introduce a novel approach to the controller design for a wide range of serial chain robots and cable-driven serial chain robots. The method addresses the regulation problem by designing a regulator and steady-state torques without requiring knowledge of the Coriolis vector, a complex part of the robot's dynamic model. The approach relies on Denavit–Hartenberg parameters to determine regulator and desired steady-state torques, using an actor–critic neural network for adaptation and a multi-layer perceptron neural network for desired steady-state torque. Stability is ensured through Lyapunov functions, with only the mass matrix and gravity vector required. The proposed control scheme offers adaptability, robustness, scalability, and optimization features compared to traditional controllers. Simulations on a 4-degrees-of-freedom cable-driven serial chain robot validate the proposed control method's effectiveness. A comparative benchmark is performed, demonstrating that the reinforcement learning controller outperforms other controllers in terms of regulation error, reducing it by at least 17.14% compared to deep deterministic policy gradient and robust integral of the sign of the error methods. • Regulation control system for Serial Chain Robots (SCRs). • Exploring RL as a computationally intensive alternative to traditional optimal control. • Reduced reliance on robot dynamics knowledge. • Limiting computational torque. • Neural Network-based adaptation laws and stability guaranteeing. [ABSTRACT FROM AUTHOR]

Reinforcement corrosion is a popular hidden disease in concrete constructions and seriously threats to the structure safety, but effectively detecting it has still been challenging for its hidden nature. This work proposes a new principle of internal reinforcement corrosion detection: magnetic resonance eddy penetrating imaging (MREPI), which shows a large eddy detection depth because of resonance enhancement. Theoretical and experimental results demonstrate the effectiveness of MREPI for imaging internal steel reinforcement. A scanning MREPI system for reinforcement testing is built and the specific images of reinforcement corrosion under up to 50 mm concrete are successfully obtained in this work. The position, orientation and diameter of reinforcement all can be determined. Importantly, by analyzing MREPI data, corrosion depth and width of the reinforcement can be quantitively characterized. These results present an innovative approach for non-destructively testing reinforcement corrosion in concrete constructions. • Develepe a penestrating imaging system based on magnetic resonance enhanced eddy effect. • Fine imaging of reinforcement and reinforcement corrosion under at least 50 mm thick layer of concrete. • Quantitively estimate locaton, diameter, corrosion depth and corrosion width of reinforcement under concrete. [ABSTRACT FROM AUTHOR]

Financial multivariate time series anomaly detection has received widespread attention, which is crucial for the reliable and safe operation of the financial system. Based on this, multi-variable time series anomaly detection methods based on different network architectures have emerged, but these models are either based on a single anomaly hypothesis (such as point anomalies) or based on labeled data (such as supervised methods). This goes against the multimodal nature and complexity of real-world data, making these methods ineffective. To remedy these shortcomings, we propose a model selection framework based on curiosity search and experience replay mechanisms.This framework includes multiple anomaly assumptions and base models in a carefully designed search space. The agent explores this search space to guide each base model, leveraging its strengths in exotic types of time series data. The framework subsequently employs reinforcement learning to select the optimal model from these candidate models. Through multi-dimensional, multi-index experimental verification of four different types of data sets, our model search framework can select the optimal base model suitable for the current timestamp from the base model pool, thereby achieving superior detection performance and better than the current A model selection framework based on reinforcement learning.Additionally, we introduce self-imitation learning and utilize an experience replay mechanism to enhance the effectiveness of samples, thereby saving detection time. To better extend our approach to more complex real-world applications, we conduct interpretability analysis on the output of anomalies. This research assists in troubleshooting and analyzing anomalies. [ABSTRACT FROM AUTHOR]

Ketamine (KET), a non-competitive N-methyl- d -aspartate (NMDA) receptor antagonist, has rapid onset of antidepressant effects in Treatment-Resistant Depression patients and repeated infusions are required to sustain its antidepressant properties. However, KET is an addictive drug, and so more preclinical and clinical research is needed to assess the safety of recurring treatments in both sexes. Thus, the aim of this study was to investigate the reinforcing properties of various doses of KET (0-, 0.125-, 0.25-, 0.5 mg/kg/infusion) and assess KET's cue-induced reinstatement and neuronal activation in both sexes of Long Evans rats. Neuronal activation was assessed using the protein expression of the immediate early gene cFos in the nucleus accumbens (Nac), an important brain area implicated in reward, reinforcement and reinstatement to most drug-related cues. Our findings show that KET has reinforcing effects in both male and female rats, albeit exclusively at the highest two doses (0.25 and 0.5 mg/kg/infusion). Furthermore, we noted sex differences, particularly at the highest dose of ketamine, with female rats displaying a higher rate of self-administration. Interestingly, all groups that self-administered KET reinstated to drug-cues. Following drug cue-induced reinstatement test in rats exposed to KET (0.25 mg/kg/infusion) or saline, there was higher cFos protein expression in KET-treated animals compared to saline controls, and higher cFos expression in the core compared to the shell subregions of the Nac. As for reinstatement, there were no notable sex differences reported for cFos expression in the Nac. These findings reveal some sex and dose dependent effects in KET's reinforcing properties and that KET at all doses induced similar reinstatement in both sexes. This study also demonstrated that cues associated with ketamine induce comparable neuronal activation in the Nac of both male and female rats. This work warrants further research into the potential addictive properties of KET, especially when administered at lower doses which are now being used in the clinic for treating various psychopathologies. [Display omitted] • Ketamine is reinforcing in both male and female rats. • At the highest dose, ketamine is more reinforcing in female rats. • No sex differences in Ketamine's cue-induced reinstatement or in cFos activation in nucleus accumbens. • Ketamine cues increase cFos expression in the core subregions of the Nac. [ABSTRACT FROM AUTHOR]

We propose a reinforcement learning strategy to control wind turbine energy generation by actively changing the rotor speed, the rotor yaw angle and the blade pitch angle. A double deep Q-learning with a prioritized experience replay agent is coupled with a blade element momentum model and is trained to allow control for changing winds. The agent is trained to decide the best control (speed, yaw, pitch) for simple steady winds and is subsequently challenged with real dynamic turbulent winds, showing good performance. The double deep Q-learning is compared with a classic value iteration reinforcement learning control and both strategies outperform a classic PID control in all environments Furthermore, the reinforcement learning approach is well suited to changing environments including turbulent/gusty winds, showing great adaptability. Finally, we compare all control strategies with real winds and compute the annual energy production. In this case, the double deep Q-learning algorithm also outperforms classic methodologies. • Reinforcement learning (Double deep Q-learning) is developed to maximize wind energy energy in changing turbulent winds. • BEMT allows three control parameters (rotational speed, blade pitch and rotor yaw). • Description of states, actions, rewards, training and validation are included. • Comparisons with a classic PID control and Value Iteration reinforcement learning are included. • Double deep Q-learning outperforms the other control mechanisms tested. [ABSTRACT FROM AUTHOR]

• This study applied computational advances in the measurement of value-based choice. • Decisions about alcohol and soft drinks were made after alcohol value manipulation. • Alcohol devaluation resulted in higher soft-drink evidence accumulation rates. • Findings demonstrate partial support for predictions from conceptual VBDM accounts. Devaluation of alcohol leads to reductions in alcohol choice and consumption; however, the cognitive mechanisms that underpin this relationship are not well-understood. In this study we applied a computational model of value-based decision-making (VBDM) to decisions made about alcohol and alcohol-unrelated cues following experimental manipulation of alcohol value. Method: Using a pre-registered within-subject design, thirty-six regular alcohol consumers (≥14 UK units per week) completed a two-alternative forced choice task where they chose between two alcohol images (in one block) or two soft drink images (in a different block) after watching videos that emphasised the positive (alcohol value), and separately, the negative (alcohol devalue) consequences of alcohol. On each block, participants pressed a key to select the image depicting the drink they would rather consume. A drift–diffusion model (DDM) was fitted to reaction time and choice data to estimate evidence accumulation (EA) processes and response thresholds during the different blocks in each experimental condition. Findings: In the alcohol devalue condition, soft drink EA rates were significantly higher compared to alcohol EA rates (p = 0.04, d = 0.31), and compared to soft drink EA rates in the alcohol value condition (p = 0.01, d = 0.38). However, EA rates for alcoholic drinks and response thresholds (for either drink type) were unaffected by the experimental manipulation. Conclusions: In line with behavioural economic models of addiction that emphasise the important role of alternative reinforcement, experimentally manipulating alcohol value is associated with changes in the internal cognitive processes that precede soft drink choice. [ABSTRACT FROM AUTHOR]

This paper proposes a new Bayesian inference-based learning automaton with a multi-objective brainstorm algorithm to find the best solutions for combinatorial optimization problems such as vehicle routing problems. Initially, a robust deep k-means algorithm is used to solve the consumer allocation problem to various clusters. Then, a multi-objective brainstorm optimization algorithm is used to determine non-dominated solutions and thereby, the Pareto front of the solution space can be formed. The proposed method uses an external archive to store the attained non-dominated solutions. In addition, a reinforcement learning algorithm is used for improving the overall convergence of the brainstorm optimization algorithm, and it can be used to discover good designs for the parameters of the brainstorm algorithm and ensure the best transition decisions by balancing the exploration and exploitation phases. Here, the control of two phases can be handled by Q-table in reinforcement learning. Subsequently, the Bayesian inference-based learning automaton is employed to learn the optimal actions by placing the agent in a complex environment. The proposed method is implemented in the Python working platform, and the performance of the proposed approach is validated based on Solomon's benchmark problems. Additionally, the effectiveness of the proposed method can be validated by comparing it with other algorithms in terms of Pareto optimal results, hypervolume, generational distance, inverted generation distance, space and spread. As a result, the proposed method has obtained better generational distance and inverted generation distance of 0.0669 and 0.0004 than other competitive methods, and it is superior for solving vehicle routing problems. ● Integrate the machine learning techniques into meta-heuristics for solving CVRP with Time Windows. ● Formulate a MOP for Vehicle Routing based on the constraints of total route distance and total customer waiting time. ● Modify the CRLA using learning automaton to improve the overall meta-heuristics convergence by learning the actions optimally. ● Validate the performance of the proposed algorithm on VRP benchmarks. [ABSTRACT FROM AUTHOR]

Under partial shading condition (PSC), for efficient usage, solar photovoltaic (PV) systems are required to operate at global maximum power point (GMPP). A maximum power point tracking (MPPT) controller is usually employed for this task. In this work, for MPPT under PSC, a recurrent deep reinforcement learning (DRL) approach has been developed and investigated. The optimized neural network, augmented by long short-term memory (LSTM), is trained using the proximal policy optimization (PPO) DRL algorithm. The MPPT performance of the proposed scheme was assessed using the GMPP attainment accuracy (%), on several realistic static and dynamic test cases. The presented investigations revealed the proposed controller's average MPPT accuracy of 97.79 % for the considered random static PSCs. Similarly, average MPPT accuracies of 95.69 % under changing irradiance, 95.70 % under change in temperature and 95.77 % under changes in both irradiance and temperature were recorded. These accuracies are found to be significantly superior to recent reported works. The resulting performance enhancement is attributed to the inclusion of LSTM, which allowed the agent to retain information about past states and actions, enabling it to make informed decisions. Therefore, based on the presented investigations, it is concluded that the proposed PPO-LSTM approach is an excellent MPPT alternative for PV systems. • A PPO-LSTM DRL approach is investigated for GMPPT in PV systems. • This approach employed previous actions and observations leading to enhanced GMPPT. • A novel, sparse, and standalone reward function is introduced. • A parametric episode sampling was implemented for effective training of RL agent. • The proposed approach is model-free, requiring no control engineering expert knowledge. [ABSTRACT FROM AUTHOR]

Flow control and shape optimisation are fundamental problems in fluid mechanics, particularly in certain scenarios involving ocean engineering. Attempts to manage the flow field via reinforcement learning are based on newly developed deep-learning techniques. By utilising an adaptive optimisation process in the flow around two square cylinders (the main square cylinder with a smaller square cylinder in the front) for the position of the front square cylinder, the flow state that minimises the oscillation of the flow field in the wake can be obtained through deep reinforcement learning. Furthermore, as the training process for this reinforcement learning is time consuming, the flow simulation component of the process is replaced with a feature detection model based on a convolutional neural network, which effectively accelerates the training process. This approach to simulating the optimal position-finding procedure with acceleration can be extended to other similar situations and practical engineering projects. • Flow control using reinforcement learning models to reduce flow-field fluctuation. • Introduction of a flow feature model to rapidly predict the flow amount. • Acceleration using two-way coupling with the flow feature model. [ABSTRACT FROM AUTHOR]

Hybrid timber composites are becoming increasingly widespread to efficiently enhance timber performance. While carbon fibre reinforcement has been explored for strength in timber structures, its impact on vibration performance remains understudied and is crucial as mass timber buildings extend their spans and heights. This work explores the effect of carbon fibre reinforcement on the vibration performance parameters of CLT floor systems. Natural frequency, mode shape, damping and stiffness parameters are evaluated from experimental testing using the Natural Excitation Technique (NExT) combined with the Eigensystem Realization Algorithm (ERA) and robust techniques for selection of physical dynamic properties, verified against numerical finite element models. The findings reveal that, in the case of three-ply Cross-laminated Timber (CLT) panels with depths ranging from 115 mm to 135 mm, introducing reinforcement at ratios below 0.67% led to an enhancement in frequencies, registering an increase of 6% to 11%, although not explicitly accounting for changes in environmental conditions. Moreover, these reinforced panels showed a relevant increase in effective flexural stiffness, ranging from 15% to 22%, when compared to their unreinforced counterparts. This means an improved maximum span of around 4%–6% within current design limits for timber floors influenced by vibration performance. These findings enhance the understanding and design of high-performance composite timber floors, promoting the use of renewable building materials in construction. • Stiffness of panels increase significantly with a small reinforcement ratio. • CFRP reinforcement allows for longer spans or smaller deflections while keeping the same panel thickness. • Floor-like assemblies also show an increase in natural frequencies due to the CFRP reinforcement. [ABSTRACT FROM AUTHOR]

This study addresses the elevator dispatching problem using deep reinforcement learning, with a specific emphasis on traffic pattern awareness. Previous studies on reinforcement learning-based elevator dispatching have largely focused on training separate models for single traffic patterns, such as up-peak, down-peak, lunch-peak, and inter-floor. This separate training approach not only introduces practical complexities by requiring an auxiliary model to predict traffic patterns for guiding dispatching decisions but is also computationally burdensome. In contrast, our goal is to develop a unified, traffic pattern-aware dispatching model. We formulate the elevator dispatching problem as a Semi-Markov Decision Process (SMDP) with novel state representation, action space, and reward function designs. To solve the formulated SMDP, we propose a Dueling Double Deep Q-Network (D3QN) architecture associated with the training algorithm. To ensure traffic pattern awareness, we train our model in a unified 'All in One' traffic scenario, employing two practical techniques to enhance the training process: (1) temporal grouping with gradient surgery and (2) incorporation of passenger arrival information. Empirical evaluations confirm the superiority of our model over multiple benchmarks, including those relying on separate, pattern-specific models. Remarkably, our unified model demonstrates robust performance across unseen traffic scenarios and performs exceptionally well in single traffic patterns despite being trained solely on the unified 'All in One' scenario. The short inference time for decision-making further solidifies the model's practical viability. Additionally, the incremental benefits contributed by each of our introduced techniques are also investigated. Our code is available at https://github.com/jswan95/RL-based-traffic-pattern-aware-elevator-dispatching [ABSTRACT FROM AUTHOR]

Statistical texture features are essential for metal surface defect detection. However, low contrast and cluttered backgrounds exacerbate the intrinsic blurriness and variability of defect textures, hampering accurate identification and localization. To tackle these challenges, we propose a Statistical Texture Feature Enhancement Network (STFE-Net) based on YOLO model, a multi-stage approach based on pooling attention mechanism and cross-scale shallow feature reinforcement. Firstly, we enhance the image contrast and propose a Stacked Pooling Attention Residual (SPA-Res) module to capture better texture features; then, we embed the Shallow Feature Weighted (SFW) branches to achieve cross-layer shallow feature reinforcement and add feature layer (AFL) to extract rich semantic features; and finally, we propose an Efficient Complete Intersection over Union (ECIoU) loss function and employ four detection heads to achieve precise localization of defects. On the NEU-DET dataset, our model achieves the state-of-the-art performance. Especially, the mean Average Precision (mAP@IoU = 0.5) of our model is 82.3 %, which is 6.2 % higher than baseline network. With the input resolution of 640 × 640, STFE-Net achieves 54 FPS inference speed with a GeForce GTX 1080. In addition, the experimental results on the DAGM dataset and GC10-DET dataset demonstrate that the proposed model has excellent generalization ability. [ABSTRACT FROM AUTHOR]

This paper aims to address the bidding strategy optimization in the real-time multi-participant electricity market with short-term load dynamics. In order to avoid the sub-optimal solution and the dependence on the complete information in traditional mathematical programming methods, an electricity market bidding strategy optimization algorithm based on deep reinforcement learning (DRL) is developed. While conventional reinforcement learning algorithms (e.g., Q-learning and deep Q-learning) are only capable of handling simple problems in discrete state spaces, the proximal policy optimization (PPO) algorithm is implemented in the bidding strategy optimization since it can optimize the bidding strategy in the continuous action and state spaces. In order to substantiate the aforementioned perspective, this paper conducts a two-part experimental study. First, experiments which consider the fixed demand load of market participants show that the developed method can reach the Nash equilibrium just like the bi-level optimization, and higher profits can be achieved by adjusting hyperparameters. Then, complex experiments which consider the time-varying demand load verify the DRL-based electricity market bidding strategy performs better than bi-level optimization-based methods and increases the profits of generators. • A PPO-EMBSA is proposed to overcome the limitations of the MPEC-based in two distinct aspects. First, PPOEMBSA does not need to transform the lower-level problem of the bi-level programming to the corresponding KKT condition. Second, agents select appropriate strategies by PPO-EMBSA in an environment with incomplete information and optimizes the present strategy for future rewards. • The PPO-EMBSA improves the accuracy and the stability of RL-based-electricity market bidding strategy. • Experiment results show that the PPO-EMBSA derives appropriate bidding strategies in situations of both fixed demand load and time-varying load, and the proposed method is more competitive than bidding algorithms based on Q-learning, DQN and A3C. In particular, additional ablation studies discuss the sensitivity of PPO-EMBSA to some hyper-parameters. [ABSTRACT FROM AUTHOR]

Reinforcement learning has achieved promising results on robotic control tasks but struggles to leverage information effectively from multiple sensory modalities that differ in many characteristics. Recent works construct auxiliary losses based on reconstruction or mutual information to extract joint representations from multiple sensory inputs to improve the sample efficiency and performance of reinforcement learning algorithms. However, the representations learned by these methods could capture information irrelevant to learning a policy and may degrade the performance. We argue that compressing information in the learned joint representations about raw multimodal observations is helpful, and propose a multimodal information bottleneck model to learn task-relevant joint representations from egocentric images and proprioception. Our model compresses and retains the predictive information in multimodal observations for learning a compressed joint representation, which fuses complementary information from visual and proprioceptive feedback and meanwhile filters out task-irrelevant information in raw multimodal observations. We propose to minimize the upper bound of our multimodal information bottleneck objective for computationally tractable optimization. Experimental evaluations on several challenging locomotion tasks with egocentric images and proprioception show that our method achieves better sample efficiency and zero-shot robustness to unseen white noise than leading baselines. We also empirically demonstrate that leveraging information from egocentric images and proprioception is more helpful for learning policies on locomotion tasks than solely using one single modality. [ABSTRACT FROM AUTHOR]

The reinforcing efficacy, or behavior-strengthening effect, of a substance is a critical determinant of substance use typically quantified by measuring behavioral allocation to the substance under schedules of reinforcement with escalating response requirements. Although responses on these tasks are often used to indicate stable reinforcing effects or trait-level abuse potential for an individual, task designs often demonstrate within-person variability across varying degrees of a constraint within experimental procedures. As a result, quantifying behavioral allocation is an effective approach for measuring the impact of contextual and psychosocial factors on substance reward. We review studies using laboratory self-administration, behavioral economic purchase tasks, and ambulatory assessments to quantify the impact of various contextual and psychosocial factors on behavioral allocation toward consumption of a substance. We selected these assessment approaches because they cover the translational spectrum from experimental control to ecological relevance, with consistent support across these approaches representing greater confidence in the effect. Conceptually, we organized factors that influence substance value into two broad categories: factors that influence the cost/benefit ratio of the substance (social context, stress and affect, cue exposure), and factors that influence the cost/benefit ratio of an alternative (alternative non-drug reinforcers, alternative drug reinforcers, and opportunity costs). We conclude with an overview of future research directions and considerations for clinical application. • Drug reward can be quantified as behavioral allocation toward substances. • Drug reward fluctuates across context and with the introduction of alternatives. • Presence of peers, affect, stress, and cues increases drug value. • Drug and non-drug alternatives and opportunity costs decrease drug value. • Treatments may be enhanced by targeting both drug and non-drug reward. [ABSTRACT FROM AUTHOR]

In the present work, the aluminum-graphite metal matrix composites were fabricated via accumulative roll bonding (ARB). The graphite powder particles of <50 μm were introduced into aluminum in three different weight percentages. The effect of graphite on the evolution of microstructure, interface formation, and mechanical properties was studied. The microstructural investigation revealed that the amount of graphite played a vital role. A better particle distribution was observed for 1.25 and 1.5 wt%. However, a further increase in the reinforcement to 2.2 wt% shows longer flaky graphite with comparatively sharing a larger interface area with the matrix aluminum. An FE-SEM/EDS (Field Emission-Scanning electron microscopy/Energy dispersive Spectroscopy) line mapping at the aluminum-graphite interface confirmed an intact sound interface with better adherence properties. The ultimate tensile strength of the composites drastically improved with an increasing number of ARB passes. The FE-SEM taken over the fractured surface of the composite revealed that the ARBed Al-1100 and the composites failed in ductile mode. It was observed that the graphite particle was the primary source of nucleation in the composites. The voids have nucleated at the particle-matrix interface, followed by their growth and coalescence. [Display omitted] • ARB process ensures a sound Al-graphite interface, avoiding undesirable phases. • Higher graphite content results in flaky particles with increased interface area. • Significant enhancement in tensile strength observed with appropriate graphite reinforcement. • Void nucleation primarily occurred at the particle-matrix interface, followed by growth and coalescence. [ABSTRACT FROM AUTHOR]