Search

Reinforcement learning (RL) exhibits impressive performance when managing complicated control tasks for robots. However, its wide application to physical robots is limited by the absence of strong safety guarantees. To overcome this challenge, this paper explores the control Lyapunov barrier function (CLBF) to analyze the safety and reachability solely based on data without explicitly employing a dynamic model. We also proposed the Lyapunov barrier actor-critic (LBAC), a model-free RL algorithm, to search for a controller that satisfies the data-based approximation of the safety and reachability conditions. The proposed approach is demonstrated through simulation and real-world robot control experiments, i.e., a 2D quadrotor navigation task. The experimental findings reveal this approach's effectiveness in reachability and safety, surpassing other model-free RL methods., Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Robot Dynamics

Due to the increase in bacterial resistance, improving the anti-infectious immunity of the host is rapidly becoming a new strategy for the prevention and treatment of bacterial pneumonia. However, the specific lung immune responses and key immune cell subsets involved in bacterial infection are obscure. Actinobacillus pleuropneumoniae (APP) can cause porcine pleuropneumonia, a highly contagious respiratory disease that has caused severe economic losses in the swine industry. Here, using high-dimensional mass cytometry, the major immune cell repertoire in the lungs of mice with APP infection was profiled. Various phenotypically distinct neutrophil subsets and Ly-6C+ inflammatory monocytes/macrophages accumulated post-infection. Moreover, a linear differentiation trajectory from inactivated to activated to apoptotic neutrophils corresponded with the stages of uninfected, onset, and recovery of APP infection. CD14+ neutrophils, which mainly increased in number during the recovery stage of infection, were revealed to have a stronger ability to produce cytokines, especially IL-10 and IL-21, than their CD14- counterparts. Importantly, MHC-II+ neutrophils with antigen-presenting cell features were identified, and their numbers increased in the lung after APP infection. Similar results were further confirmed in the lungs of piglets infected with APP and Klebsiella pneumoniae infection by using a single-cell RNA-seq technique. Additionally, a correlation analysis between cluster composition and the infection process yielded a dynamic and temporally associated immune landscape where key immune clusters, including previously unrecognized ones, marked various stages of infection. Thus, these results reveal the characteristics of key neutrophil clusters and provide a detailed understanding of the immune response to bacterial pneumonia., Pattern Recognition and Bioinformatics

As global warming continues to intensify, the relationship between diurnal temperature range (DTR) and vegetation productivity continues to change over time. However, the impact of DTR changes on vegetation activities remains uncertain. Thus, further study about how DTR changes affect the physiological activities of plants is also urgently needed. In this study, we employed copula function theory to analyze the impact of DTR on Normalized Difference Vegetation Index (NDVI) values during the spring, summer, and autumn seasons from 1982 to 2014 for various land types in the Inner Mongolia Plain (IMP), China. The results showed that the relationship between DTR and NDVI in the IMP was characterized by correlation at the upper tail and asymptotical independence at the lower tail. This demonstrated that the DTR had little effect on NDVI when they reached their minimum value. However, it has a significant impact on NDVI at its maximum values. This study provides valuable insight into the dynamic impact of monthly DTR on different land use types under climate change., Hydraulic Structures and Flood Risk

Extreme surges and rainfall represent major driving factors for compound flooding in estuary regions along the Chinese coast. The combined effect of extreme surges and rainfall (that is, compound floods) might lead to greater impacts than if the drivers occurred in isolation. Hence, understanding the frequency and severity of compound flooding is important for improving flood hazard assessment and compound flood resilience in coastal cities. In this study, we examined the dependence between extreme surges and corresponding rainfall events in 26 catchments along the Chinese coastline during typhoon and non-typhoon seasons using copula functions, to identify where the two drivers more often occur together and the implication for flood management in these locations. We found that the interaction between flood drivers is statistically significant in 10 catchments located around Hainan Island (south) and Shanghai, where surge peaks occur mainly during the typhoon season and around the Bohai Sea (north), where surge peaks occur mainly during the non-typhoon season. We further applied the copula-based framework to model the dependence between surge peaks and associated rainfall and estimate their joint and conditional probability in two specific locations—Hainan Island and the Bohai Sea, where the correlation between flood drivers is statistically significant. We observed that in Hainan Island where most of the surge peaks occur during the typhoon season, extreme rainfall events during the typhoon season are generally more intense compared to annual maxima rainfall. In contrast, around the Bohai Sea where surge peaks occur mainly outside the typhoon season, rainfall is less intense than annual maxima rainfall. These results show that the interaction between extreme surges and rainfall can provide valuable insight when designing coastal and urban infrastructure, especially in highly populated urban areas prone to both coastal and pluvial flooding, such as many Chinese co, Hydraulic Structures and Flood Risk, Coastal Engineering

Reconstruction of high-fidelity 3D objects or scenes is a fundamental research problem. Recent advances in RGB-D fusion have demonstrated the potential of producing 3D models from consumer-level RGB-D cameras. However, due to the discrete nature and limited resolution of their surface representations (e.g., point or voxel based), existing approaches suffer from the accumulation of errors in camera tracking and distortion in the reconstruction, which leads to an unsatisfactory 3D reconstruction. In this article, we present a method using on-the-fly implicits of Hermite Radial Basis Functions (HRBFs) as a continuous surface representation for camera tracking in an existing RGB-D fusion framework. Furthermore, curvature estimation and confidence evaluation are coherently derived from the inherent surface properties of the on-the-fly HRBF implicits, which are devoted to a data fusion with better quality. We argue that our continuous but on-the-fly surface representation can effectively mitigate the impact of noise with its robustness and constrain the reconstruction with inherent surface smoothness when being compared with discrete representations. Experimental results on various real-world and synthetic datasets demonstrate that our HRBF-fusion outperforms the state-of-the-art approaches in terms of tracking robustness and reconstruction accuracy., Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright older of this work and the author uses the Dutch legislation to make this work public., Urban Data Science, Materials and Manufacturing

This project aims to improve employees’ social well-being in offices through informal interactions. It consists of 3 phases in total, context research, idea generation and concept development. Experiencing prototyping was mainly used in the design process. In the first phase, observation and interviews were conducted in 2 offices in Beijing to know about the context and discover problems and design opportunities. Another round of research was carried out in StudioLab for verification of the insights from Beijing offices. The results, combining with the results of the literature review, led to the design goal defined. Then the project started to focus on stimulating meaningful chats among employees to increase their sense of belonging. Ideas were brainstormed and selected, prototypes and storyboards were made to let participants evaluate the concepts through interviews. After 3 cycles of idea generation, the final concept direction was defined. After that, 2 cycles of concept development were conducted to iterate the concept. Again, prototypes were made and interviews were done to evaluate the concept and get feedback from participants. In the end, the final design was defined, final prototypes were made and final evaluation was conducted. Overall, it is a design project exploring solutions for solving one of employees’ social problems in offices, lacking the sense of belonging., Design for Interaction

This project aims to improve employees’ social well-being in offices through informal interactions. It consists of 3 phases in total, context research, idea generation and concept development. Experiencing prototyping was mainly used in the design process. In the first phase, observation and interviews were conducted in 2 offices in Beijing to know about the context and discover problems and design opportunities. Another round of research was carried out in StudioLab for verification of the insights from Beijing offices. The results, combining with the results of the literature review, led to the design goal defined. Then the project started to focus on stimulating meaningful chats among employees to increase their sense of belonging. Ideas were brainstormed and selected, prototypes and storyboards were made to let participants evaluate the concepts through interviews. After 3 cycles of idea generation, the final concept direction was defined. After that, 2 cycles of concept development were conducted to iterate the concept. Again, prototypes were made and interviews were done to evaluate the concept and get feedback from participants. In the end, the final design was defined, final prototypes were made and final evaluation was conducted. Overall, it is a design project exploring solutions for solving one of employees’ social problems in offices, lacking the sense of belonging., Design for Interaction

Humans are capable of attributing latent mental contents such as beliefs, or intentions to others. The social skill is critical in everyday life to reason about the potential consequences of their behaviors so as to plan ahead. It is known that humans use this reasoning ability recursively, i.e. considering what others believe about their own beliefs. In this paper, we start from level-1 recursion and introduce a probabilistic recursive reasoning (PR2) framework for multi-agent reinforcement learning. Our hypothesis is that it is beneficial for each agent to account for how the opponents would react to its future behaviors. Under the PR2 framework, we adopt variational Bayes methods to approximate the opponents' conditional policy, to which each agent finds the best response and then improve their own policy. We develop decentralized-training-decentralized-execution algorithms, PR2-Q and PR2-Actor-Critic, that are proved to converge in the self-play scenario when there is one Nash equilibrium. Our methods are tested on both the matrix game and the differential game, which have a non-trivial equilibrium where common gradient-based methods fail to converge. Our experiments show that it is critical to reason about how the opponents believe about what the agent believes. We expect our work to contribute a new idea of modeling the opponents to the multi-agent reinforcement learning community., Robot Dynamics

Bas-relief is characterized by its unique presentation of intrinsic shape properties and/or detailed appearance using materials raised up in different degrees above a background. However, many bas-relief modeling methods could not manipulate scene details well. We propose a simple and effective solution for two kinds of bas-relief modeling (i.e., structure-preserving and detail-preserving), which is different from the prior tone mapping alike methods. Our idea originates from an observation on typical 3D models which are decomposed into a piecewise smooth base layer and a detail layer in normal field. Proper manipulation of the two layers contributes to both structure-preserving and detail-preserving bas-relief modeling. We solve the modeling problem in a discrete geometry processing setup that uses normal-based mesh processing as a theoretical foundation. Specifically, using the two-step mesh smoothing mechanism as a bridge, we transfer the bas-relief modeling problem into a discrete space, and solve it in a least-squares manner. Experiments and comparisons to other methods show that (i) geometry details are better preserved in the scenario with high compression ratios, and (ii) structures are clearly preserved without shape distortion and interference from details., Accepted author manuscript, Materials and Manufacturing

Humans are capable of attributing latent mental contents such as beliefs, or intentions to others. The social skill is critical in everyday life to reason about the potential consequences of their behaviors so as to plan ahead. It is known that humans use this reasoning ability recursively, i.e. considering what others believe about their own beliefs. In this paper, we start from level-1 recursion and introduce a probabilistic recursive reasoning (PR2) framework for multi-agent reinforcement learning. Our hypothesis is that it is beneficial for each agent to account for how the opponents would react to its future behaviors. Under the PR2 framework, we adopt variational Bayes methods to approximate the opponents' conditional policy, to which each agent finds the best response and then improve their own policy. We develop decentralized-training-decentralized-execution algorithms, PR2-Q and PR2-Actor-Critic, that are proved to converge in the self-play scenario when there is one Nash equilibrium. Our methods are tested on both the matrix game and the differential game, which have a non-trivial equilibrium where common gradient-based methods fail to converge. Our experiments show that it is critical to reason about how the opponents believe about what the agent believes. We expect our work to contribute a new idea of modeling the opponents to the multi-agent reinforcement learning community., Robot Dynamics

Bas-relief is characterized by its unique presentation of intrinsic shape properties and/or detailed appearance using materials raised up in different degrees above a background. However, many bas-relief modeling methods could not manipulate scene details well. We propose a simple and effective solution for two kinds of bas-relief modeling (i.e., structure-preserving and detail-preserving), which is different from the prior tone mapping alike methods. Our idea originates from an observation on typical 3D models which are decomposed into a piecewise smooth base layer and a detail layer in normal field. Proper manipulation of the two layers contributes to both structure-preserving and detail-preserving bas-relief modeling. We solve the modeling problem in a discrete geometry processing setup that uses normal-based mesh processing as a theoretical foundation. Specifically, using the two-step mesh smoothing mechanism as a bridge, we transfer the bas-relief modeling problem into a discrete space, and solve it in a least-squares manner. Experiments and comparisons to other methods show that (i) geometry details are better preserved in the scenario with high compression ratios, and (ii) structures are clearly preserved without shape distortion and interference from details., Accepted author manuscript, Materials and Manufacturing

This paper proposes a novel hybrid model for learning discrete and continuous dynamics of car-following behaviors. Multiple modes representing driving patterns are identified by partitioning the model into groups of states. The model is visualizable and interpretable for car-following behavior recognition, traffic simulation, and human-like cruise control. The experimental results using the next generation simulation datasets demonstrate its superior fitting accuracy over conventional models., Accepted author manuscript, Cyber Security

Car-following is the most general behavior in highway driving. It is crucial to recognize the cut-in intention of vehicles from an adjacent lane for safe and cooperative driving. In this paper, a method of behavior estimation is proposed to recognize and predict the lane change intentions based on the contextual traffic information. A model predictive controller is designed to optimize the acceleration sequences by incorporating the lane-change intentions of other vehicles. The public data set of next generation simulation is labeled and then published as a benchmarking platform for the research community. Experimental results demonstrate that the proposed method can accurately estimate vehicle behavior and therefore outperform the traditional car-following control., Accepted author manuscript, Cyber Security

Car-following is the most general behavior in highway driving. It is crucial to recognize the cut-in intention of vehicles from an adjacent lane for safe and cooperative driving. In this paper, a method of behavior estimation is proposed to recognize and predict the lane change intentions based on the contextual traffic information. A model predictive controller is designed to optimize the acceleration sequences by incorporating the lane-change intentions of other vehicles. The public data set of next generation simulation is labeled and then published as a benchmarking platform for the research community. Experimental results demonstrate that the proposed method can accurately estimate vehicle behavior and therefore outperform the traditional car-following control., Accepted author manuscript, Cyber Security

Learning driving behavior is fundamental for autonomous vehicles to “understand” traffic situations. This paper proposes a novel method for learning a behavioral model of car-following using automata learning algorithms. The model is interpretable for car-following behavior analysis. Frequent common state sequences are extracted from the model and clustered as driving patterns. The Next Generation SIMulation dataset on the I-80 highway is used for learning and evaluating. The experimental results demonstrate high accuracy of car-following model fitting., Cyber Security

Learning driving behavior is fundamental for autonomous vehicles to “understand” traffic situations. This paper proposes a novel method for learning a behavioral model of car-following using automata learning algorithms. The model is interpretable for car-following behavior analysis. Frequent common state sequences are extracted from the model and clustered as driving patterns. The Next Generation SIMulation dataset on the I-80 highway is used for learning and evaluating. The experimental results demonstrate high accuracy of car-following model fitting., Cyber Security