Search

Function approximation has gained significant attention in reinforcement learning research as it effectively addresses problems with large-scale, continuous state, and action space.Although the function approximation algorithm based on gradient descent method is one of the most widely used methods in reinforcement learning, it requires careful tuning of the step size parameter as an inappropriate value can lead to slow convergence, unstable convergence, or even divergence.To address these issues, an improvement was made around the temporal-difference (TD) algorithm based on function approximation.The weight update method was enhanced using both the least squares method and gradient descent, resulting in the proposed weights gradient descent (WGD) method.The least squares were used to calculate the weights, combining the ideas of TD and gradient descent to find the error between the weights.And this error was used to directly update the weights.By this method, the weights were updated in a new manner, effectively reducing the consumption of computing resources by the algorithm enhancing other gradient descent-based function approximation algorithms.The WGD method is widely applicable in various gradient descent-based reinforcement learning algorithms.The results show that WGD method can adjust parameters within a wider space, effectively reducing the possibility of algorithm divergence.Additionally, it achieves better performance while improving the convergence speed of the algorithm.

Cadmium (Cd) is a common environmental pollutant that accumulates mainly in the kidneys and thus endangers the physiological health of aquatic animals. Selenium (Se) is a natural antidote to heavy metals that antagonises heavy metal toxicity and enhances the antioxidant capacity of organisms. Lactobacillus plantarum (L. plantarum) can reduce the toxicity of heavy metals through adsorption, reduction and metabolism. Studies have confirmed that the biological synthesis of Se nanoparticles (Bio-SeNPs) using bacterial microorganisms is simple, safe and less toxic than the synthesis of inorganic and organic Se, but the effect on Cd-induced immunosuppression is un-known. One hundred and eighty Bulatmai barbel (Luciobarbus capito: L. capito) plants were randomly divided into control (C), Cd and Cd + Se-enriched L. plantarum groups (S1L1-Cd) and fed for 28 days. The analysis methods included histopathology, test kits, transcriptomics and real-time quantitative PCR. The addition of selenium-enriched L. plantarum significantly attenuated cadmium-induced pathological changes such as glomerular atrophy, detachment of renal tubular epithelial cells, mild swelling, and interstitial inflammatory cell infiltration. Cd stress can lead to significant decreases in RBC, HCT, WBC, LZM, C3, and IgM levels, and the addition of Se-enriched L. plantarum can significantly reverse the changes in these indicators. Transcriptomic analysis revealed 488 DEGs in the Cd groups, 301 of which were upregulated and 187 of which were downregulated. There were 1474 DEGs in the S1L1-Cd group, of which 720 were upregulated and 754 were downregulated. In addition, GO enrichment analysis revealed that the biological regulation of the most differentially expressed genes involved metal ion binding, ATP binding and nucleotide inclusion. KEGG enrichment analysis revealed six of the most enriched pathways: oxidative phosphorylation, Huntington disease, retrograde endocannabinoid signalling, natural killer cell-mediated cyto-toxicity, the IL-17 signalling pathway, and leukocyte transient migration. Moreover, we selected 12 DEGs for qRT-PCR, which showed that the qRT-PCR results were consistent with our RNA-Seq results. Our results suggest that Se-enriched L. plantarum can enhance immunity and alleviate Cd exposure-mediated immunosuppression in L. capito. [ABSTRACT FROM AUTHOR]

As an emerging technology to convert environmental high‐entropy energy into electrical energy, triboelectric nanogenerator (TENG) has great demands for further enhancing the service lifetime and output performance in practical applications. Here, an ultra‐robust and high‐performance rotational triboelectric nanogenerator (R‐TENG) by bearing charge pumping is proposed. The R‐TENG composes of a pumping TENG (P‐TENG), an output TENG (O‐TENG), a voltage‐multiplying circuit (VMC), and a buffer capacitor. The P‐TENG is designed with freestanding mode based on a rolling ball bearing, which can also act as the rotating mechanical energy harvester. The output low charge from the P‐TENG is accumulated and pumped to the non‐contact O‐TENG, which can simultaneously realize ultralow mechanical wear and high output performance. The matched instantaneous power of R‐TENG is increased by 32 times under 300 r/min. Furthermore, the transferring charge of R‐TENG can remain 95% during 15 days (6.4 × 106 cycles) continuous operation. This work presents a realizable method to further enhance the durability of TENG, which would facilitate the practical applications of high‐performance TENG in harvesting distributed ambient micro mechanical energy. [ABSTRACT FROM AUTHOR]

Astronauts exhibit neurological dysfunction during long-duration spaceflight, and the specific mechanisms may be closely related to the cumulative effects of these neurological injuries in the space radiation environment. Here, we investigated the interaction between astrocytes and neuronal cells exposed to simulated space radiation. we selected human astrocytes (U87 MG) and neuronal cells (SH-SY5Y) to establish an experimental model to explore the interaction between astrocytes and neuronal cells in the CNS under simulated space radiation environment and the role of exosomes in the interactions. We found that γ-ray caused oxidative and inflammatory damage in human U87 MG and SH-SY5Y. The results of the conditioned medium transfer experiments showed that astrocytes exhibited a protective effect on neuronal cells, and neuronal cells influenced the activation of astrocytes in oxidative and inflammatory injury of CNS. We demonstrated that the number and size distribution of exosomes derived from U87 MG and SH-SY5Y cells were changed in response to H2O2, TNF-α or γ-ray treatment. Furthermore, we found that exosome derived from treated nerve cells influenced the cell viability and gene expression of untreated nerve cells, and the effect of exosomes was partly consistent with that of the conditioned medium. Our findings demonstrated that astrocytes showed a protective effect on neuronal cells, and neuronal cells influenced the activation of astrocytes in oxidative and inflammatory damage of CNS induced by simulated space radiation. Exosomes played an essential role in the interaction between astrocytes and neuronal cells exposed to simulated space radiation. [ABSTRACT FROM AUTHOR]

With the advancement of internet of things and wireless sensing technology, the need for personnel track and position monitoring are greatly increasing. Commercial global positioning system (GPS) and Global system for mobile communications (GSM) position methods cannot meet the demand for low power consumption and three‐dimensional indoor position monitoring. Herein, a self‐powered position monitoring system (SPMS) based on an insole‐type wearable triboelectric nanogenerator (IW‐TENG) and Bluetooth beacon for human kinetic energy harvesting and sustainable position monitoring is proposed. The SPMS consists of an insole‐type wearable triboelectric nanogenerator, an energy management module (EMM), and a Bluetooth low energy (BLE) beacon module. Benefitting from the multilayers structure, a maximum energy of 120 µJ per step can be harvested by the IW‐TENG with EMM for a typical adult. By every 3–4 steps, SPMS can transmit Bluetooth signals to obtain the location information. A single BLE signal transmission consumes only 136 µJ, and the transmission distance can reach 100 m. The SPMS is applied to intelligent attendance in office rooms and shows great prospects in intelligent navigation and trajectory tracking in local area network. [ABSTRACT FROM AUTHOR]