Search

Semantic scene understanding is an essential task for self-driving vehicles and mobile robots. In our work, we aim to estimate a semantic segmentation map, in which the foreground objects are removed and semantically inpainted with background classes, from a single RGB image. This semantic foreground inpainting task is performed by a single-stage convolutional neural network (CNN) that contains our novel max-pooling as inpainting (MPI) module, which is trained with weak supervision, i.e., it does not require manual background annotations for the foreground regions to be inpainted. Our approach is inherently more efficient than the previous two-stage state-of-the-art method, and outperforms it by a margin of 3% IoU for the inpainted foreground regions on Cityscapes. The performance margin increases to 6% IoU, when tested on the unseen KITTI dataset. The code and the manually annotated datasets for testing are shared with the research community at https://github.com/Chenyang-Lu/semantic-foreground-inpainting., Comment: RA-L and ICRA'20

We propose a novel single-step training strategy that allows convolutional encoder-decoder networks that use skip connections, to complete partially observed data by means of hallucination. This strategy is demonstrated for the task of completing 2-D road layouts as well as 3-D vehicle shapes. As input, it takes data from a partially observed domain, for which no ground truth is available, and data from an unpaired prior knowledge domain and trains the network in an end-to-end manner. Our single-step training strategy is compared against two state-of-the-art baselines, one using a two-step auto-encoder training strategy and one using an adversarial strategy. Our novel strategy achieves an improvement up to +12.2% F-measure on the Cityscapes dataset. The learned network intrinsically generalizes better than the baselines on unseen datasets, which is demonstrated by an improvement up to +23.8% F-measure on the unseen KITTI dataset. Moreover, our approach outperforms the baselines using the same backbone network on the 3-D shape completion benchmark by a margin of 0.006 Hamming distance.

Internet of Things (IoT) employs a large number of spatially distributed wireless sen-sors to monitor physical environments, e.g., temperature, humidity, and air pressure, have found wide applications including environmental monitoring, health care monitoring, smart cities and precision agriculture. A wireless sensor can collect, analyze, and transmit measurements of its environment. To date, wireless sensors used in IoT are predominately based on electronic devices that may suffer from electromagnetic interference in many circumstances. Immune to the electromagnetic interference, optical sensors provide a significant advantage in harsh environments. Furthermore, by introducing optical resonance to enhanced light-matter interactions, optical sensors based on resonators exhibit small footprints, extreme sensitivity and versatile functionalities, which can signifi-cantly enhance the capability and flexibility of wireless sensors. Here we provide the first demonstration of a wireless photonic sensor node based on whispering-gallery-mode (WGM) optical resonators. The sensor node is controlled via a customized iOS app. Its per-formance was studied in two practical scenarios: (1) real-time measurement of air tempera-ture over 12 hours and (2) aerial mapping of temperature distribution by a sensor node mounted on an unmanned drone. Our work demonstrates the capability of WGM optical sensors in practical applications and may pave the way for large-scale deployments of WGM sensors in IoT.

In this work, we research and evaluate end-to-end learning of monocular semantic-metric occupancy grid mapping from weak binocular ground truth. The network learns to predict four classes, as well as a camera to bird's eye view mapping. At the core, it utilizes a variational encoder-decoder network that encodes the front-view visual information of the driving scene and subsequently decodes it into a 2-D top-view Cartesian coordinate system. The evaluations on Cityscapes show that the end-to-end learning of semantic-metric occupancy grids outperforms the deterministic mapping approach with flat-plane assumption by more than 12% mean IoU. Furthermore, we show that the variational sampling with a relatively small embedding vector brings robustness against vehicle dynamic perturbations, and generalizability for unseen KITTI data. Our network achieves real-time inference rates of approx. 35 Hz for an input image with a resolution of 256x512 pixels and an output map with 64x64 occupancy grid cells using a Titan V GPU.

Hospital readmission rate is high for heart failure patients. Early detection of deterioration will help doctors prevent readmissions, thus reducing health care cost and providing patients with just-in-time intervention. Wearable devices (e.g., wristbands and smart watches) provide a convenient technology for continuous outpatient monitoring. In the paper, we explore the feasibility of monitoring outpatients using Fitbit Charge HR wristbands and the potential of machine learning models to predicting clinical deterioration (readmissions and death) among outpatients discharged from the hospital. We developed and piloted a data collection system in a clinical study which involved 25 heart failure patients recently discharged from a hospital. The results from the clinical study demonstrated the feasibility of continuously monitoring outpatients using wristbands. We observed high levels of patient compliance in wearing the wristbands regularly and satisfactory yield, latency and reliability of data collection from the wristbands to a cloud-based database. Finally, we explored a set of machine learning models to predict deterioration based on the Fitbit data. Through 5-fold cross validation, K nearest neighbor achieved the highest accuracy of 0.8800 for identifying patients at risk of deterioration using the health data from the beginning of the monitoring. Machine learning models based on multimodal data (step, sleep and heart rate) significantly outperformed the traditional clinical approach based on LACE index. Moreover, our proposed weighted samples one class SVM model can reach high accuracy (0.9635) for predicting the deterioration happening in the future using data collected by a sliding window, which indicates the potential for allowing timely intervention.

Wireless networked control systems (WNCS) are composed of spatially distributed sensors, actuators, and con- trollers communicating through wireless networks instead of conventional point-to-point wired connections. Due to their main benefits in the reduction of deployment and maintenance costs, large flexibility and possible enhancement of safety, WNCS are becoming a fundamental infrastructure technology for critical control systems in automotive electrical systems, avionics control systems, building management systems, and industrial automation systems. The main challenge in WNCS is to jointly design the communication and control systems considering their tight interaction to improve the control performance and the network lifetime. In this survey, we make an exhaustive review of the literature on wireless network design and optimization for WNCS. First, we discuss what we call the critical interactive variables including sampling period, message delay, message dropout, and network energy consumption. The mutual effects of these communication and control variables motivate their joint tuning. We discuss the effect of controllable wireless network parameters at all layers of the communication protocols on the probability distribution of these interactive variables. We also review the current wireless network standardization for WNCS and their corresponding methodology for adapting the network parameters. Moreover, we discuss the analysis and design of control systems taking into account the effect of the interactive variables on the control system performance. Finally, we present the state-of-the-art wireless network design and optimization for WNCS, while highlighting the tradeoff between the achievable performance and complexity of various approaches. We conclude the survey by highlighting major research issues and identifying future research directions., Comment: 37 pages, 17 figures, 4 tables

Wireless sensor-actuator networks (WSANs) are gaining momentum in industrial process automation as a communication infrastructure for lowering deployment and maintenance costs. In traditional wireless control systems the plant controller and the network manager operate in isolation, which ignore the significant influence of network reliability on plant control performance. To enhance the dependability of industrial wireless control, we propose a holistic cyber-physical management framework that employs run-time coordination between the plant control and network management. Our design includes a holistic controller that generates actuation signals to physical plants and reconfigures the WSAN to maintain desired control performance while saving wireless resources. As a concrete example of holistic control, we design a holistic manager that dynamically reconfigures the number of transmissions in the WSAN based on online observations of physical and cyber variables. We have implemented the holistic management framework in the Wireless Cyber-Physical Simulator (WCPS). A systematic case study has been presented based on two 5-state plants sharing a 16-node WSAN. Simulation results show that the holistic management design has significantly enhanced the resilience of the system against both wireless interferences and physical disturbances, while effectively reducing the number of wireless transmissions., Comment: Submitted to TCPS