Search

Abstract Muscular dystrophies (MD) are a group of genetic neuromuscular disorders that cause progressive weakness and loss of muscles over time, influencing 1 in 3500–5000 children worldwide. New and exciting treatment options have led to a critical need for a clinical post-marketing surveillance tool to confirm the efficacy and safety of these treatments after individuals receive them in a commercial setting. For MDs, functional gait assessment is a common approach to evaluate the efficacy of the treatments because muscle weakness is reflected in individuals’ walking patterns. However, there is little incentive for the family to continue to travel for such assessments due to the lack of access to specialty centers. While various existing sensing devices, such as cameras, force plates, and wearables can assess gait at home, they are limited by privacy concerns, area of coverage, and discomfort in carrying devices, which is not practical for long-term, continuous monitoring in daily settings. In this study, we introduce a novel functional gait assessment system using ambient floor vibrations, which is non-invasive and scalable, requiring only low-cost and sparsely deployed geophone sensors attached to the floor surface, suitable for in-home usage. Our system captures floor vibrations generated by footsteps from patients while they walk around and analyzes such vibrations to extract essential gait health information. To enhance interpretability and reliability under various sensing scenarios, we translate the signal patterns of floor vibration to pathological gait patterns related to MD, and develop a hierarchical learning algorithm that aggregates insights from individual footsteps to estimate a person’s overall gait performance. When evaluated through real-world experiments with 36 subjects (including 15 patients with MD), our floor vibration sensing system achieves a 94.8% accuracy in predicting functional gait stages for patients with MD. Our approach enables accurate, accessible, and scalable functional gait assessment, bringing MD progressive tracking into real life.

Crowd monitoring for sports games is important to improve public safety, game experience, and venue management. Recent crowd-crushing incidents (e.g., the Kanjuruhan Stadium disaster) have caused 100+ deaths, calling for advancements in crowd-monitoring methods. Existing monitoring approaches include manual observation, wearables, video-, audio-, and WiFi-based sensing. However, few meet the practical needs due to their limitations in cost, privacy protection, and accuracy.

Optical metamaterials manipulate light through various confinement and scattering processes, offering unique advantages like high performance, small form factor and easy integration with semiconductor devices. However, designing metasurfaces with suitable optical responses for complex metamaterial systems remains challenging due to the exponentially growing computation cost and the ill‐posed nature of inverse problems. To expedite the computation for the inverse design of metasurfaces, a physics‐informed deep learning (DL) framework is used. A tandem DL architecture with physics‐based learning is used to select designs that are scientifically consistent, have low error in design prediction, and accurate reconstruction of optical responses. The authors focus on the inverse design of a representative plasmonic device and consider the prediction of design for the optical response of a single wavelength incident or a spectrum of wavelength in the visible light range. The physics‐based constraint is derived from solving the electromagnetic wave equations for a simplified homogenized model. The model converges with an accuracy up to 97% for inverse design prediction with the optical response for the visible light spectrum as input, and up to 96% for optical response of single wavelength of light as input, with optical response reconstruction accuracy of 99%.

This study presents the first rapid seismic multi-hazard and impact estimation system integrating advanced causal inference and remote sensing techniques, which jointly estimates regional-scale and high-resolution maps of seismic multi-hazards and building damage from InSAR imageries.

Quantitative analysis of human gait is critical for the early discovery, progressive tracking, and rehabilitation of neurological and musculoskeletal disorders, such as Parkinson’s disease, stroke, and cerebral palsy. Gait analysis typically involves estimating gait characteristics, such as spatiotemporal gait parameters and gait health indicators (e.g., step time, length, symmetry, and balance). Traditional methods of gait analysis involve the use of cameras, wearables, and force plates but are limited in operational requirements when applied in daily life, such as direct line-of-sight, carrying devices, and dense deployment. This paper introduces a novel approach for gait analysis by passively sensing floor vibrations generated by human footsteps using vibration sensors mounted on the floor surface. Our approach is low-cost, non-intrusive, and perceived as privacy-friendly, making it suitable for continuous gait health monitoring in daily life. Our algorithm estimates various gait parameters that are used as standard metrics in medical practices, including temporal parameters (step time, stride time, stance time, swing time, double-support time, and single-support time), spatial parameters (step length, width, angle, and stride length), and extracts gait health indicators (cadence/walking speed, left–right symmetry, gait balance, and initial contact types). The main challenge we addressed in this paper is the effect of different floor types on the resultant vibrations. We develop floor-adaptive algorithms to extract features that are generalizable to various practical settings, including homes, hospitals, and eldercare facilities. We evaluate our approach through real-world walking experiments with 20 adults with 12,231 labeled gait cycles across concrete and wooden floors. Our results show 90.5% (RMSE 0.08s), 71.3% (RMSE 0.38m), and 92.3% (RMSPE 7.7%) accuracy in estimating temporal, spatial parameters, and gait health indicators, respectively.

Fine-grained air pollution monitoring has attracted increasing attention worldwide. Even with an increasing amount of both static and mobile sensing systems, an inference algorithm is still essential to achieve a comprehensive understanding of the urban atmospheric environment. Conventional physical model-based methods are unable to involve all the influencing factors with limited prior knowledge, and datadriven methods lacking physical interpretation may result in bad generalization ability. This paper presents a multi-task learning scheme, which combines the physical model and the data-driven model with both merits. It enhances the data learning of a neural network with the aid of prior knowledge on atmospheric dispersion, and also controls the impact of the knowledge with a tunable weighting coefficient. Evaluations over a real-world deployment in Foshan, China show that, with the resolution of 500m×500m×15min, the proposed method outperforms the state-of-the-art ones with 7.9% error reduction and 6.2% correlation increase. Benefited from the physical knowledge, the neural network obtains stable performance with lower variance, as well as higher robustness against negative background conditions.

Non-intrusive monitoring of fine-grained activities of daily living (ADL) enables various smart healthcare applications. For example, ADL pattern analysis for older adults at risk can be used to assess their loss of safety or independence. Prior work in the area of ADL recognition has focused on coarse-grained ADL recognition at the context-level (e.g., cooking, cleaning, sleeping), and/or activity duration segmentation (hourly or minutely). It also typically relies on a high-density deployment of a variety of sensors. In this work, we target a finer-grained ADL recognition at the action-level to provide more detailed ADL information, which is crucial for enabling the assessment of patients' activity patterns and potential changes in behavior. To achieve this fine-grained ADL monitoring, we present a heterogeneous multi-modal cyber-physical system, where we use (1) distributed vibration sensors to capture the action-induced structural vibrations and their spatial characteristics for information aggregation, and (2) single point electrical sensor to capture appliance usage with high temporal resolution. To evaluate our system, we conducted real-world experiments with multiple human subjects to demonstrate the complementary information from these two sensing modalities. Our system achieved an average 90% accuracy in recognizing activities, which is up to 2.6× higher than baseline systems considering each state-of-the-art sensing modality separately.

A common pain point for physical retail stores is live inventory monitoring, i.e., knowing how many items of each product are left on the shelves. About 4% of sales are lost due to an average 5–10% out-of-shelf stockout rate, while additional supplies existed in the warehouse. Traditional techniques rely on manual inspection, per-item tagging using RFIDs, or human-in-the-loop systems, such as Amazon Go. These approaches, while effective, either have poor accuracy, long delays between results or are cost prohibitive. In this paper, we present FAIM (Autonomous Inventory Monitoring Framework) for cashier-less stores. To the best of our knowledge, this is the first fully autonomous system that fuses multiple sensing modalities. Utilizing weight difference on a shelf, visual item recognition in customers' hands and prior knowledge of item layout FAIM monitors products picked up or returned without human-in-the-loop. We present results from a real-world setup with 85 items (33 unique products) replicating the layout of a local 7-Eleven store. To evaluate our system we characterize the similarity of the unique products across three physical features (i.e., weight, color, and location). Our results show that the fused approach provides up to 92.6% item identification accuracy, a 2× reduction in error compared to reported self-checkout stations.

Sensor-based occupant tracking has the potential to enhance knowledge of the utilization of buildings. Occupancy-tracking strategies using footstep-induced floor vibrations may be beneficial for thermal-load prediction, security enhancement, and care-giving without undermining privacy. Current floor-vibration-based occupant-tracking methodologies are based on data-driven techniques that do not include a physics-based model of the structural behavior of the floor slab. These techniques suffer from ambiguous interpretations when signals are affected by complex configurations of structural and non-structural elements such as beams and walls. Using a physics-based model for data-interpretation enables deployment of sparse number of sensors in contexts of non-uniform structural configurations. In this paper, an application of physics-based data interpretation using error-domain model falsification (EDMF) is presented to track an occupant within an office environment through footstep-induced floor vibrations. EDMF is a population-based approach that incorporates various sources of uncertainty, including bias, arising from measurements and modeling. EDMF involves the rejection of simulated model responses that contradict footstep-induced floor vibration measurements. Thus, EDMF provides a set of candidate locations from an initial population of possible occupant locations. A sequential analysis that accommodates information from previous footsteps is then used to enhance candidate locations and identify trajectories among candidates. In this way, incorporating structural behavior in interpreting vibration measurements induced by occupant footsteps has the potential to identify accurately the trajectory of an occupant in buildings with complex configurations, thereby providing tracking information without undermining privacy.

This paper introduces an indirect train traffic monitoring method to detect and infer real-time train events based on the vibration response of a nearby building. Monitoring and characterizing traffic events are important for cities to improve the efficiency of transportation systems (e.g., train passing, heavy trucks, and traffic). Most prior work falls into two categories: (1) methods that require intensive labor to manually record events or (2) systems that require deployment of dedicated sensors. These approaches are difficult and costly to execute and maintain. In addition, most prior work uses dedicated sensors designed for a single purpose, resulting in deployment of multiple sensor systems. This further increases costs. Meanwhile, with the increasing demands of structural health monitoring, many vibration sensors are being deployed in commercial buildings. Traffic events create ground vibration that propagates to nearby building structures inducing noisy vibration responses. We present an information-theoretic method for train event monitoring using commonly existing vibration sensors deployed for building health monitoring. The key idea is to represent the wave propagation in a building induced by train traffic as information conveyed in noisy measurement signals. Our technique first uses wavelet analysis to detect train events. Then, by analyzing information exchange patterns of building vibration signals, we infer the category of the events (i.e., southbound or northbound train). Our algorithm is evaluated with an 11-story building where trains pass by frequently. The results show that the method can robustly achieve a train event detection accuracy of up to a 93% true positive rate and an 80% true negative rate. For direction categorization, compared with the traditional signal processing method, our information-theoretic approach reduces categorization error from 32.1 to 12.1%, which is a 2.5× improvement.

This paper presents a collaboratively adaptive vibration monitoring system that captures high-fidelity structural vibration signals induced by pedestrians. These signals can be used for various human activities’ monitoring by inferring information about the impact sources, such as pedestrian footsteps, door opening and closing, and dragging objects. Such applications often require high-fidelity (high resolution and low distortion) signals. Traditionally, expensive high resolution and high dynamic range sensors are adopted to ensure sufficient resolution. However, for sensing systems that use low-cost sensing devices, the resolution and dynamic range are often limited; hence this type of sensing methods is not well explored ubiquitously. We propose a low-cost sensing system that utilizes (1) a heuristic model of the investigating excitations and (2) shared information through networked devices to adapt hardware configurations and obtain high-fidelity structural vibration signals. To further explain the system, we use indoor pedestrian footstep sensing through ambient structural vibration as an example to demonstrate the system performance. We evaluate the application with three metrics that measure the signal quality from different aspects: the sufficient resolution rate to present signal resolution improvement without clipping, the clipping rate to measure the distortion of the footstep signal, and the signal magnitude to quantify the detailed resolution of the detected footstep signal. In experiments conducted in a school building, our system demonstrated up to 2× increase on the sufficient resolution rate and 2× less error rate when used to locate the pedestrians as they walk along the hallway, compared to a fixed sensing setting.

Individual perspiration level indicates a person’s physical status as well as their comfort level. Therefore, continuous perspiration level measurement enables people to monitor these conditions for applications including fitness assessment, athlete physical status monitoring, and patient/elderly care. Prior work on perspiration (sweat) sensing required the user either to be static or to wear the adhesive sensor directly on the skin, which limits users’ mobility and comfort. In this paper, we present a novel conductive thread-based textile sensor that measures an individual’s on-cloth sweat quantity. The sensor consists of three conductive threads. Each conductive thread is surrounded by a braided cotton cover. An additional braided cotton cover is placed outside the three conductive threads, holding them in a position that is stable for measurement. the sensor can be embedded at various locations on a person’s clothing. When the person sweats, the cotton braids absorb the sweat and change the conductivity (resistance) between conductive threads. We used a voltage dividing circuit to measure this resistance as the sensor output (DC). We then conducted a sensor calibration to map this measured voltage to the quantity of electrolyte solution (with the same density as sweat) applied to the sensor. We used this sensor to measure individuals’ perspiration quantity and infer their perceived perspiration levels. The system is able to limit the average prediction error to 0.4 levels when compared to five pre-defined perceived perspiration levels.

Robust and accurate building energy load forecasting is important for helping building managers and utilities to plan, budget, and strategize energy resources in advance. With recent prevalent adoption of smart-meters in buildings, a significant amount of building energy consumption data became available. Many studies have developed physics-based white box models and data-driven black box models to predict building energy consumption; however, they require extensive prior knowledge about building system, need a large set of training data, or lack robustness to different forecasting scenarios. In this paper, we introduce a new building energy forecasting method based on Gaussian Process Regression (GPR) that incorporates physical insights about load data characteristics to improve accuracy while reducing training requirements. The GPR is a non-parametric regression method that models the data as a joint Gaussian distribution with mean and covariance functions and forecast using the Bayesian updating. We model the covariance function of the GPR to reflect the data patterns in different forecasting horizon scenarios, as prior knowledge. Our method takes advantage of the modeling flexibility and computational efficiency of the GPR while benefiting from the physical insights to further improve the training efficiency and accuracy. We evaluate our method with three field datasets from two university campuses (Carnegie Mellon University and Stanford University) for both short- and long-term load forecasting. The results show that our method performs more accurately, especially when the training dataset is small, compared to other state-of-the-art forecasting models (up to 2.95 times smaller prediction error).