Search

Deep reinforcement learning (DRL) combines reinforcement learning algorithms with deep neural networks (DNNs). Spiking neural networks (SNNs) have been shown to be a biologically plausible and energy efficient alternative to DNNs. Since the introduction of surrogate gradient approaches that allowed to overcome the discontinuity in the spike function, SNNs can now be trained with the backpropagation through time (BPTT) algorithm. While largely explored on supervised learning problems, little work has been done on investigating the use of SNNs as function approximators in DRL. Here we show how SNNs can be applied to different DRL algorithms like Deep Q-Network (DQN) and Twin-Delayed Deep Deteministic Policy Gradient (TD3) for discrete and continuous action space environments, respectively. We found that SNNs are sensitive to the additional hyperparameters introduced by spiking neuron models like current and voltage decay factors, firing thresholds, and that extensive hyperparameter tuning is inevitable. However, we show that increasing the simulation time of SNNs, as well as applying a two-neuron encoding to the input observations helps reduce the sensitivity to the membrane parameters. Furthermore, we show that randomizing the membrane parameters, instead of selecting uniform values for all neurons, has stabilizing effects on the training. We conclude that SNNs can be utilized for learning complex continuous control problems with state-of-the-art DRL algorithms. While the training complexity increases, the resulting SNNs can be directly executed on neuromorphic processors and potentially benefit from their high energy efficiency.

Background Pericardial patches are often used for repair of congenital cardiac defects. The aim of this study was to describe our initial experience with the use of equine pericardium and its safety and advantages and disadvantages compared to bovine pericardium. Methods We designed a retrospective cohort study of 111 patients who were surgically treated for congenital heart disease between 2017 and 2020. Equine pericardium was used in 58 patients and bovine pericardium was used in 53 patients. Recorded variables included demographic data, preoperative cardiac pathology, site of patch insertion, morbidity and mortality. Results The overall survival rate was 94.5% and no deaths were related to patch insertion. None of our patients were reoperated on for patch related complications. Postoperative transcatheter intervention was needed in 2 patients (1.8%): one for dilatation of aortic arch stenosis after repair of hypoplastic left heart syndrome with equine pericardium and one for dilatation of pulmonary artery branches after repair of tetralogy of Fallot using bovine pericardium. Conclusions Equine pericardium is a safe patch material for reconstruction in congenital heart surgery. It may be preferable to bovine pericardium in cases requiring a complex shape or a pliable patch as in in arch reconstruction or for valve reconstruction.

Background: St. Thomas (ST) and Del Nido (DN) cardioplegic solutions are widely used for myocardial protection during cardiac surgery. In 2016, our university hospital shifted from modified St. Thomas to Del Nido solution for both adult and pediatric cardiac surgery. This retrospective study was conducted to compare ST and DN solutions regarding surgical workflow and clinical outcome in pediatric and adult patients undergoing cardiac surgery. Methods: We reviewed 220 patients who underwent cardiac surgery requiring cardioplegic arrest. Patients were categorized in 2 groups: ST (n = 110) and DN (n = 110). Each group included 60 pediatric and 50 adult patients. Demographic, intraoperative, and postoperative variables were collected. Results: In pediatric patients, no significant difference was found between the 2 groups regarding clamping time, bypass time, need for defibrillation, inotropic score, postoperative ejection fraction (EF), period of mechanical ventilation, intensive care unit stay, or postoperative arrhythmias. One patient in the ST group required mechanical support by extracorporeal membrane oxygenation. We had 5 cases of pediatric mortality (3 in DN and 2 in ST, P = .64). In adult patients, significantly fewer patients in the DN group needed defibrillation than in the ST group. No significant difference was found regarding clamping time, inotropic score, or intraaortic balloon pump use. Mortality in adult patients was 6 cases (4 in ST group and 2 in DN group). Conclusion: DN cardioplegia solution is as safe as ST solution in pediatric and adult cardiac surgery. It has comparable results of myocardial protection and clinical outcome, with superiority regarding uninterrupted surgery and lower rate of defibrillation.

Deep neural networks (DNNs) set the benchmark in many tasks in perception and control. Spiking versions of DNNs, implemented on neuromorphic hardware can enable orders of magnitude lower power consumption and low latency during network use. In this paper, we explore behavior and generalization capability of spiking, quantized spiking, and hardware implementation of deep Q-networks in two classical reinforcement learning tasks. We found that spiking neural networks have slightly decreased performance compared to non-spiking network, but we can avoid performance degradation from quantization and in-chip implementation. We conclude that since hardware implementation leads to lower power consumption and low latency, neuromorphic approach is a promising avenue for deep Q-learning. Furthermore, online learning, enabled in neuromorphic chips, can be used to compensate for the performance decrease in environments with parameter variations.

Effectiveness of project-based learning at the level of health and sports awareness for students of the Collegeof Applied Studies and Community Service. the research aims to identify the effectiveness of projectbasedlearning at the level of health and sports awareness for students of the College of Applied Studiesand Community Service. researchers used the experimental method for its suitability to the research. Theresearch sample was deliberately chosen from first-level students, who are (113) students from the Collegeof Applied Studies and Community Service, whose ages range from (18-20) years. Measurements weretaken and data collected through questionnaire forms.Among the most important findings of the research is the existence of a statistically significant relationshipat the level of (0.05) in all aspects of students’ health and sports awareness, the effectiveness of the projectlearning method as one of the health education methods in the learning process.Through this, the following recommendations were reached to activate the project learning method fordifferent academic courses and use it on different groups in society, to pay attention to the practical aspectas a basic method in teaching health and fitness, which helps to confirm the practical practice of informationrelated to sports and health in the daily life of students andintegrating educational messages in the curriculacourses at different academic levels.

The energy efficiency of buildings drives the replacement of traditional construction materials with lightweight insulating materials. However, energy-efficient but combustible insulation might contribute to the building’s fire load. Therefore, it is necessary to analyse the reaction-to-fire properties of various insulating materials to provide a better understanding of designing a fire-safe structure. In this study, reaction-to-fire tests were carried out to assess the fire behaviour of lightweight polystyrene insulating panels commonly employed in high-rise buildings. The flammability characteristics of expanded polystyrene (EPS) and extruded polystyrene (XPS) were determined using a cone calorimeter under two distinct external irradiance regimes, 35 kW/m2 and 50 kW/m2, to approximate small to medium fire exposure situations. To investigate the effect of a fire-rated (FR) foil layer on a sandwich panel, three distinct test configurations were used: (i) sample without FR layer (standard sample), (ii) sample with FR layer (FR foil), and (iii) damaged layer (foil and vent) for EPS. Except for the smoke toxicity index (STI), the overall fire performance of EPS is superior to that of XPS. The findings of this study are useful in analysing fire performance and fire safety design for lightweight insulation panels.

The main purpose of the human arm is to reach a target and perform a manipulation task. Human babies learn to move their arms by imitating and doing motor babbling through trial and error. This learning is believed to result from changes in synaptic efficacy triggered by complex mechanisms involving neuro-modulators in which dopamine plays a key role. After learning, humans are able to reuse and adapt the motions without performing complex calculations. In contrast, classical robotics achieve target reaching by mathematically computing each time the inverse kinematics (IK) of the joint angles leading to a particular target, then validating the configuration and generating a trajectory. This process is computational intensive and becomes more complex with the amount of degrees of freedom (DoF). In this work, we propose a spiking neural network architecture to learn target reaching motions with a robotic arm using reinforcement learning (RL), which is closely related to the way babies learn. To make our approach scalable, we sub-divide the kinematics structure of the robot and create one sub-network per joint. We generate training data offline by generating random reaching motions with an IK calculation outside of the network. After learning, the IK is no longer required, and the model is implicitly learned in the weights of the network. Mimicking the learning mechanisms of the brain, we use the spike time dependent plasticity (STDP) learning rule modulated by dopamine, representing a reward. The approach is evaluated with a simulated Universal Robot UR5 with six DoF. The network successfully learns to reach multiple targets and by changing the reward function on-the-fly it is able to learn different control functions. With a standard computer our network was able to control a robotic kinematics chain up to 13 DoF in real time. A key aspect of our approach is that in contrast to deep RL our SNN does not need much data to learn new behaviors. We believe that model free motion controllers inspired on the human brain mechanisms can improve the way robots are programmed by making the process more adaptive and flexible.

Learning-based methods have demonstrated clear advantages in controlling robot tasks, such as the information fusion abilities, strong robustness, and high accuracy. Meanwhile, the on-board systems of robots have limited computation and energy resources, which are contradictory with state-of-the-art learning approaches. They are either too lightweight to solve complex problems or too heavyweight to be used for mobile applications. On the other hand, training spiking neural networks (SNNs) with biological plausibility has great potentials of performing fast computation and energy efficiency. However, the lack of effective learning rules for SNNs impedes their wide usage in mobile robot applications. This paper addresses the problem by introducing an end to end learning approach of spiking neural networks for a lane keeping vehicle. We consider the reward-modulated spike-timing-dependent-plasticity (R-STDP) as a promising solution in training SNNs, since it combines the advantages of both reinforcement learning and the well-known STDP. We test our approach in three scenarios that a Pioneer robot is controlled to keep lanes based on an SNN. Specifically, the lane information is encoded by the event data from a neuromorphic vision sensor. The SNN is constructed using R-STDP synapses in an all-to-all fashion. We demonstrate the advantages of our approach in terms of the lateral localization accuracy by comparing with other state-of-the-art learning algorithms based on SNNs.

Background: CT scanning of ancient human remains has the potential to provide insights into health and diseases. While Egyptian mummies have undergone CT scans in prior studies, a systematic survey of the orthopedic conditions afflicting a group of these ancient individuals has never been carried out. Methods: We performed whole body CT scanning on 52 ancient Egyptian mummies using technique comparable to that of medical imaging. All of the large joints and the spine were systematically examined and osteoarthritic (OA) changes were scored 0-4 using Kellgren and Lawrence classification. Results: The cruciate ligaments and menisci could be identified frequently. There were much more frequent OA changes in the spine (25 mummies) than in the large joints (15 cases of acromioclavicular and/or glenohumeral joint OA changes, five involvement of the ankle, one in the elbow, four in the knee, and one in the hip). There were six cases of scoliosis. Individual mummies had the following conditions: juvenile aseptic necrosis of the hip (Perthes disease), stage 4 osteochondritis dissecans of the knee, vertebral compression fracture, lateral patella-femoral joint hyper-compression syndrome, severe rotator cuff arthropathy, rotator cuff impingement, hip pincer impingement, and combined fracture of the greater trochantor and vertebral bodies indicating obvious traumatic injury. This report includes the most ancient discovery of several of these syndromes. Conclusions: Ancient Egyptians often suffered painful orthopedic conditions. The high frequency of scoliosis merits further study. The pattern of degenerative changes in the spine and joints may offer insights into activity levels of these people. Anat Rec, 298:1036–1046, 2015. © 2015 Wiley Periodicals, Inc.

The field of neurorobotics encompasses the intersection of computational neuroscience and robotics. The TUM led Neurorobotics subproject of the Human Brain Project is actively researching concepts within the field and developing the tools to allow researchers to fully explore simulated robotics driven by computational neuroscience models. Further, the development of biologically inspired, tendon driven robotics systems provides a unique research platform. These efforts allow researchers to explore the interesting space from computational neuroscience to intelligent robots and back.

CT scanning of ancient human remains has the potential to provide insights into health and diseases. While Egyptian mummies have undergone CT scans in prior studies, a systematic survey of the orthopedic conditions afflicting a group of these ancient individuals has never been carried out.We performed whole body CT scanning on 52 ancient Egyptian mummies using technique comparable to that of medical imaging. All of the large joints and the spine were systematically examined and osteoarthritic (OA) changes were scored 0-4 using Kellgren and Lawrence classification.The cruciate ligaments and menisci could be identified frequently. There were much more frequent OA changes in the spine (25 mummies) than in the large joints (15 cases of acromioclavicular and/or glenohumeral joint OA changes, five involvement of the ankle, one in the elbow, four in the knee, and one in the hip). There were six cases of scoliosis. Individual mummies had the following conditions: juvenile aseptic necrosis of the hip (Perthes disease), stage 4 osteochondritis dissecans of the knee, vertebral compression fracture, lateral patella-femoral joint hyper-compression syndrome, severe rotator cuff arthropathy, rotator cuff impingement, hip pincer impingement, and combined fracture of the greater trochantor and vertebral bodies indicating obvious traumatic injury. This report includes the most ancient discovery of several of these syndromes.Ancient Egyptians often suffered painful orthopedic conditions. The high frequency of scoliosis merits further study. The pattern of degenerative changes in the spine and joints may offer insights into activity levels of these people.

Artificial neural networks have been employed in many areas of cognitive systems research, ranging from low-level control tasks to high-level cognition. However, there is only few work on the use of spiking neural networks in these fields. Unlike ar- tificial neurons, spiking neuron models are designed to approx- imate the dynamics of biological neurons. In this work, we developed a virtual environment to explore solving navigation tasks using spiking neural networks. We first used an exper- imental setup inspired by Floreano and Mattiussi (2001) and compared the results to validate the developed environment. An evolutionary approach is used to set the parameters of a spiking neural network controlling a robot to navigate without collisions. In a second set of experiments, we trained the net- work via reinforcement learning which was implemented as a reward-based STDP protocol. Our results validate the correct- ness of the developed virtual environment and demonstrate the usefulness of using such a platform. The virtual environment guarantees the reproducibility of our experiments and can be easily adapted for future research.

Background Deep hypothermic circulatory arrest (DHCA) is a technique used in the repair of complex congenital cardiac lesions that require aortic arch or pulmonary vein repair. DHCA has been linked to adverse outcomes and neurologic complications. Selective cerebral perfusion (SCP) may be added to DHCA to prevent neurological complication. Air embolism and hyperperfusion injury may be encountered. The aim of this study was to evaluate the safety and efficacy of simple DHCA and to outline the early outcomes especially the neurological ones. Methods Twenty nine patients underwent surgical repair of congenital cardiac lesion with DHCA at a single institution from January 2010 to November 2015. DHCA was conducted with a target esophageal temperature of 18° and placement of an ice pack on the head. No selective perfusion was done. Demographic, operative and postoperative data were reviewed. Mortality, any neurological complications including seizers, coma, and stroke were recorded. Results The mean age was 20.6 ± 8.2 months (range: 9 days to 154 months). The majority were males (20, 69%). The mean weight was 5.57 ± 4.2 kg (range: 2.3–17.5 kg). DHCA time was 20.03 min (range 3–52 min). There were three (10.3%) deaths. Two deaths occurred after Norwood operation, and one after interrupted aortic arch repair. None of the deaths were related to neurological injury. None of the patients developed seizers, coma, abnormal movement or neurological deficits. Conclusions Simple DHCA without SCP is a safe, expeditious and reliable method for brain protection during repair of complex cardiac lesions, with acceptable outcomes.