Search

Reinforcement learning has been widely applied in exploration, navigation, manipulation, and other fields. Most of the relevant techniques generate kinematic commands (e.g., move, stop, turn) for agents based on the current state information. However, recent dense action representations based research, such as spatial action maps, pointing way-points to the agent in the same domain as its observation of the state shows great promise in mobile manipulation tasks. Inspired by that, we make the first step towards using a spatial action maps based method to effectively explore novel environmental spaces. To reduce the chance of redundant exploration, the visit frequency map (VFM) and its corresponding reward function are introduced to direct the agent to actively search previously unexplored areas. In the experimental section, our work was compared to the same method without VFM and the method based on traditional steering commands with the same input data in various environments. The results show conclusively that our method is more efficient than other methods.

Programmed cell death (PCD) is an essential feature during the development of the central nervous system in Drosophila as well as in mammals. During metamorphosis, a group of peptidergic neurons (vCrz) are eliminated from the larval central nervous system (CNS) via PCD within 6-7 h after puparium formation. To better understand this process, we first characterized the development of the vCrz neurons including their lineages and birth windows using the MARCM (Mosaic Analysis with a Repressible Cell Marker) assay. Further genetic and MARCM analyses showed that not only Myoglianin (Myo) and its type I receptor Baboon is required for neuron cell death, but also this death signal is extensively regulated by endocytic trafficking in Drosophila melanogaster. We found that clathrin-mediated membrane receptor internalization and subsequent endocytic events involved in Rab5-dependent early endosome and Rab11-dependent recycling endosome differentially participate in TGF-β [beta] signaling. Two early endosome-enriched proteins, SARA and Hrs, are found to act as a cytosolic retention factor of Smad2, indicating that endocytosis mediates TGF-β [beta] signaling through regulating the dissociation of Smad2 and its cytosolic retention factor.