Search

Human activity recognition (HAR) is a key enabling technology for a wide range of emerging applications. Although multimodal sensing systems are essential for capturing complex and dynamic human activities in real-world settings, they bring several new challenges including limited labeled multimodal data. In this paper, we propose Cosmo, a new system for contrastive fusion learning with small data in multimodal HAR applications. Cosmo features a novel two-stage training strategy that leverages both unlabeled data on the cloud and limited labeled data on the edge. By integrating novel fusion-based contrastive learning and quality-guided attention mechanisms, Cosmo can effectively extract both consistent and complementary information across different modalities for efficient fusion. Our evaluation on a cloud-edge testbed using two public datasets and a new multimodal HAR dataset shows that Cosmo delivers significant improvement over state-of-the-art baselines in both recognition accuracy and convergence delay. [ABSTRACT FROM AUTHOR]

A COVID 19 outbreak caused by the new SARSCoV2 virus was declared by the World Health Organization (WHO) in March 2020. Since then, other studies have used Chest Xray or CT scans to identify this infection. Often, one aspect of the study is that these XRAY or CT scans of Covid patients have to be enhanced. The purpose of picture fusion is to merge complimentary, multi-sensor and/or multi-view images. Our major purpose of our work is to assist doctors speed up treatments in order to give their patients the most effective remedies as soon as possible. This study employs two multi-view data sets, which are merged using hybrid methodology and divided into two phases, as input images for our system. In first stage we use two fusion rules of Dual tree Complex Wavelet Transform (DT-CWT) and Discrete Cosine Transform (DCT) separately on both the images. In second stage we use fusion rule based on Singular Value Decomposition (SVD) on those fused images acquired from first stage. The performance of fused image is carried out by standard deviation (SD), root mean square (RMSE), peak signal to noise ratio (PSNR), percentage fit error (PEF), mean absolute error (MAE), mutual information (MI), quality index (QI) and measure of structural similarity (SSIM). [ABSTRACT FROM AUTHOR]

Information fusion methodologies are one of the most relevant issues at the core of multimodal systems. More than the simple juxtapositions of various sensory modalities at the interface, their possible synergistic combinations should be considered, in order to overcome individual weaknesses and obtain the sum of their expressive forces. The current paradigm in the association of data in interactive processes of a multimodal nature proposes the existence of three relevant aggregating levels: the level of fusion of characteristics - early fusion; the level of fusion of semantics - late fusion; and a hybrid fusion, that implies the simultaneous use of the fusion levels of characteristics and semantics. On the part of most top authors, there is a frequent disregard for the potential of the association of raw signals, immediately after their acquisition - the so-called (early fusion), notably when coming from different modalities or input channels. This article proposes to clarify, through literature review, the importance of this type of fusion for the integration of subjectivity in the scope of artistic explorations, so that one can consider combinations of creative scope that contemplate error as a form of communication and expression. [ABSTRACT FROM AUTHOR]

Recent developments in Internet of Things for data collection and monitoring have increased its potential applications in monitoring environmental phenomenon for data dependent climate action. Internet of things devices are convenient for real time monitoring and generate huge amount of data. The collected data can be uncertain and erroneous, making traditional data analysis approaches redundant. Moreover, specific approaches can be best suited to a dataset depending upon its nature. The present work aims to tackle these challenges in the experimental dataset consisting of vital parameters for water quality assessment of Ganga River. The dataset was collected over a period of one year. The experiment based comparative analysis were performed and the results show that the Dempster-Shafer evidence theory based data fusion approach outperforms other similar methods and improves the classification accuracy. [ABSTRACT FROM AUTHOR]

The Fusion Science Demonstrator in the European Open Science Cloud for Research Pilot Project aimed to demonstrate that the fusion community can make use of distributed cloud resources. We developed a platform, Prominence, which enables users to transparently exploit idle cloud resources for running scientific workloads. In addition to standard HTC jobs, HPC jobs such as multi-node MPI are supported. All jobs are run in containers to ensure they will reliably run anywhere and are reproduceable. Cloud infrastructure is invisible to users, as all provisioning, including extensive failure handling, is completely automated. On-premises cloud resources can be utilised and at times of peak demand burst onto external clouds. In addition to the traditional "cloud-bursting" onto a single cloud, Prominence allows for bursting across many clouds in a hierarchical manner. Job requirements are taken into account, so jobs with special requirements, e.g. high memory or access to GPUs, are sent only to appropriate clouds. Here we describe Prominence, its architecture, the challenges of using many clouds opportunistically and report on our experiences with several fusion use cases. [ABSTRACT FROM AUTHOR]

Super-resolution image fusion aims to produce an image with finer spectral and spatial details than the input images. However, the super-resolution fusion is mainly applied to enhance a visual representation of the images and its potential benefits to the final thematic classification is an open question. In this paper, we present an experimental investigation of the remote sensing image classification performance in the case of the multi-sensor super-resolution image fusion. The research aims to compare classification performance obtained for the fused image and the low resolution input ones using different standard-of-the-art classifiers and feature extraction methods. Input data are supposed to be multispectral data obtained in visible and near infrared spectral ranges by the different remote sensing systems. To perform a multi-sensor super-resolution image fusion, we used a gradient-descent optimization approach with a B-TV regularization successfully adapted for remote sensing images with different spatial and spectral sampling characteristics by the authors of the paper. As for features, we applied brightness in spectral channels, attribute profiles and local feature attribute profiles. The classification was performed using support vector machines and random forest classifiers that have been proved to be very effective for remote sensing data classification. The experimental research included the multi-sensor input data simulation for four remote sensing systems, the super-resolution image fusion of all simulated images and the thematic classification of the fused image and the images obtained as an average input for each of the simulated imaging systems. The spatial resolution of the fused image was in 2, 3, 4 and 5 times better than the spatial resolution of the modeled input images. The average bandwidth of the fused image was 29 nm whereas for the input low resolution images it was in the range from 37 to 83 nm. Experimental results have shown that random forest classification is better to use with fusion, whereas support vector machines demonstrated better results without fusion. The feature extraction test showed that extended attribute profiles enhance the random forest classification accuracy of the fused image. Thus, the classification results have shown that super-resolution image fusion leads to the classification accuracy increase in the case of random forest classifier and there is no need to apply fusion in the case of support vector machines. [ABSTRACT FROM AUTHOR]

Geospatial technologies have been widely used in various fields since their inception. In particular, the science of remote sensing has helped in the analysis and solving of many earth's related issues. One primary application of remote sensing has been in the field of environmental studies and monitoring. This paper addresses the application of data fusion of remotely sensed satellites' data which could be useful to the remote sensing analyst. Data fusion is an important emerging area which finds its application in every field of research. In this paper, fusion of IKONOS sensor satellite imageries comprising of a panchromatic image and a multispectral image is addressed. The fusion method is built based on the pixel value of the images. An objective function is formulated using Lagrange's multiplier which minimizes the spectral inconsistency and variance differences. Evaluation of the results of the proposed technique using independent criteria is also discussed. [ABSTRACT FROM AUTHOR]

Many of the indoor localization algorithms rely on predictor-corrector methods. In the prediction phase the algorithm estimates the user’s position in the next timeframe and the correction phase uses some kind of location specific measurement to refine the position. For indoor navigation systems developed for human users the most obvious prediction method is determining the heading of user using the built-in compass of the mobile phone and detecting and analyzing the walking motion step-by-step. Knowing the actual position of the person and combining the step and heading information the next estimated position can be determined. The step detection is based on the acquired data from a mobile phone sensor filtered and processed by an appropriate algorithm in order to determine the user motion state. The paper presents the development and results of such an algorithm. [ABSTRACT FROM AUTHOR]

Modern aerospace industry is migrating from reactive to proactive and predictive maintenance to increase platform operational availability and efficiency, extend its useful life cycle and reduce its life cycle cost. Multiphysics modeling together with data-driven analytics generate a new paradigm called “Digital Twin.” The digital twin is actually a living model of the physical asset or system, which continually adapts to operational changes based on the collected online data and information, and can forecast the future of the corresponding physical counterpart. This paper reviews the overall framework to develop a digital twin coupled with the industrial Internet of Things technology to advance aerospace platforms autonomy. Data fusion techniques particularly play a significant role in the digital twin framework. The flow of information from raw data to high-level decision making is propelled by sensor-to-sensor, sensor-to-model, and model-to-model fusion. This paper further discusses and identifies the role of data fusion in the digital twin framework for aircraft predictive maintenance. [ABSTRACT FROM AUTHOR]

The paper presents the evaluation with experimental data of an ANFIS based INS/GPS integrated navigator. The experimental model of the INS/GPS navigator has been manufactured in the Aerospace Engineering laboratories of the University of Craiova, Romania, during a research project. This hardware structure was developed around an inertial module (inertial detection unit) incorporating three analogous miniaturized inertial sensors (LPY510AL - Biaxial analog gyro, LPR510AL - Biaxial analog gyro, MMA7361L - Three-axial analog accelerometer), by adding a GPS module (u-blox NEO-6M) and a dsPIC33EP512MU810 microcontroller. The data from inertial sensors were acquired with a sample rate of 100 samples/s, while the GPS receiver provided data once per second. To equalize the sampling rate of the two systems before the data fusion, an ANFIS extrapolator for the GPS lost data has been used. Boarded on a testing car, in a first phase the system provided the experimental data necessary to tune the ANFIS based data fusion algorithm. The training data were provided by the system clock, by the six channels of the inertial detection unit, by the attitude information and by the estimates of the positioning and speed errors obtained at the output of the Kalman filter included in the integrated navigator. The ANFIS structure includes six channels, each one with a Fuzzy Inference System (FIS), estimating the navigation errors for global position components (latitude, longitude and altitude) and for vehicle speed in North, East and Down directions. The navigator has been successfully tested with experimental data acquired in three different scenarios: open sky detection (access to GPS signal all the time), GPS signal interruption, INS detection only. [ABSTRACT FROM AUTHOR]

The traditional attitude detection technology can't tackle the problem of attitude detection of the polyhedral robot. Thus we propose a novel algorithm of multi-sensor data fusion which is based on Kalman filter. In the algorithm a tetrahedron robot is investigated. We devise an attitude detection system for the polyhedral robot and conduct the verification of data fusion algorithm. It turns out that the minimal attitude detection system we devise could capture attitudes of the tetrahedral robot in different working conditions. Thus the Kinematics model we establish for the tetrahedron robot is correct and the feasibility of the attitude detection system is proven. [ABSTRACT FROM AUTHOR]

This paper presents a Wireless Sensor Network (WSN) system which was created as a project about protecting wildlife using sensor networks following the assistance of the department of Electrical and Computer Engineering of the Democritus University of Thrace. An automated process was implemented, regarding the recognition of a passenger (ie human, wolf, bear, etc.) traversing a box-shaped underground passage, such as the ones located along main highways fusing Width, Height and Weight values. These were measured using low-cost distance (beam) and weight (S-type load) micro-sensors and stored in a central repository. Moreover, the information provided by the WSN was analyzed, via a variety of methods including a neural pattern recognition network as well as clustering algorithms, which were able to recognize the kind of passenger, with certainty scores over 90%. The main concern, regarding the future, is the evaluation of these passages in respect to their effectiveness, i.e. whether they are frequently utilized by animals. This information was further analysed by appropriate information systems, in order to provide insights about the effectiveness of such mitigation structures. [ABSTRACT FROM AUTHOR]

The paper exposes the validation of an error model associated with an inertial navigation system. The final target is to use this model in an integrated ING/GPS architecture based on a neural-network data fusion algorithm. As a start point, the navigator mechanization and its error model are highlighted, both from the mathematical point of view, as well as from their Matlab/Simulink software implementation point of view. The model allows the evaluation of the attitude errors, in terms of pitch, roll and yaw angles errors, but also allows the evaluation of the global positioning errors, in terms of latitude, longitude and altitude errors, and of the vehicle speed errors, in terms of north, east and down speed components errors. To validate the error model a software procedure is developed. It implies the using of few standardized errors affecting the miniaturized inertial sensors, software modelled and interfaced in order to allow the user to run various testing scenarios. The numerical and graphical results obtained by applying this procedure prove the validity of the error model. In the same time, are created the premises to use the developed testing procedure to evaluate the performance limits of an inertial navigation system when the sensors in the detection unit are affected by certain errors. [ABSTRACT FROM AUTHOR]

The Harghita Mts are located in the central part of Romania and represent the youngest part of the East Carpathians Neogene-Quaternary magmatic belt. The Ciomadul magmatic structure, situated at the southern end of the Harghita Mts, includes the St. Ana Lake and several isolated domes, Puturosul and Balvanyos representing the easternmost outcropping volcanics. Traces of active post volcanic processes are widespread between Tusnad and Balvanyos localities, mostly represented by mineral groundwater springs (displaying a large variety of chemical characteristics) and moffeta (generated by gas emergences rich in CO2). The special interest in this post volcanic activities is given by the perspective of being used as a tool for economic development of the region, once famous for the curative properties of the mineralized groundwater resources and the abundant number of springs (naturally outcropping or tapped). The possibility to revive the great potential of this area is diminished by the small quantities of mineral groundwater offered by the springs nowadays still active and the unsuccessful wells drilled for mineral water exploitation during the last several decades. Benefiting of the specific low electric resistivity values of mineral groundwater in this post volcanic area (mostly ranging between 0.5 and 15 Om) it was recently considered that modern geophysical techniques, such as the Electric Resistivity Tomography (ERT), may contribute to locate underground aquifers bearing higher water resources. ERT survey lines were deployed in the Balvanyos, St. Ana Lake and Mohos Peat Bog areas trending N-S, E-W, NW-SE and NE-SW in an attempt to detect structural features involved in groundwater vertical displacement (faults, fractures, tectonic contacts) or its underground accumulations. The high variability of electric resistivity values illustrated in the ERT cross-sections may be interpreted in terms of groundwater quality, rarely here to be used for domestic consumption, but mostly for various medical treatments. [ABSTRACT FROM AUTHOR]

Two heads are better than one, and the many are smarter than the few. Integrating knowledge from multiple sources has shown to increase retrieval and classification accuracy in many domains. The recent explosion of crowdsourced information, such as on websites hosting chords and tabs for popular songs, calls for sophisticated algorithms for data-driven quality assessment and data integration to create better, and more reliable data. In this paper, we propose to integrate the heterogeneous output of multiple automatic chord extraction algorithms using data fusion. First we show that data fusion creates significantly better chord label sequences from multiple sources, outperforming its source material, majority voting and random source integration. Second, we show that data fusion is capable of assessing the quality of sources with high precision from source agreement, without any ground-truth knowledge. Our study contributes to a growing body of work showing the benefits of integrating knowledge from multiple sources in an advanced way. [ABSTRACT FROM AUTHOR]

In this paper, we present the mathematical models and algorithms that describe the problem of attitude determination for a small satellite using measurements from three angular rate sensors (ARS) and aiding measurements from multiple GPS receivers/antennas rigidly attached to the platform of the satellite. [ABSTRACT FROM AUTHOR]

First developed to predict continuous variables, Bayesian Maximum Entropy (BME) has become a complete framework in the context of space-time prediction since it has been extended to predict categorical variables and mixed random fields. This method proposes solutions to combine several sources of data whatever the nature of the information. However, the various attempts that were made for adapting the BME methodology to categorical variables and mixed random fields faced some limitations, as a high computational burden. The main objective of this paper is to overcome this limitation by generalizing the Bayesian Data Fusion (BDF) theoretical framework to categorical variables, which is somehow a simplification of the BME method through the convenient conditional independence hypothesis. The BDF methodology for categorical variables is first described and then applied to a practical case study : the estimation of soil drainage classes using a soil map and point observations in the sandy area of Flanders around the city of Mechelen (Belgium). The BDF approach is compared to BME along with more classical approaches, as Indicator CoKringing (ICK) and logistic regression. Estimators are compared using various indicators, namely the Percentage of Correctly Classified locations (PCC) and the Average Highest Probability (AHP). Although BDF methodology for categorical variables is somehow a simplification of BME approach, both methods lead to similar results and have strong advantages compared to ICK and logistic regression. [ABSTRACT FROM AUTHOR]

Predictive maintenance, one of the important topics in industrial data analysis, mainly focuses on predicting when the machine or device will be broken or malfunctioned based on the traceability of sensor data collected from machines or devices. In the previous research works, time-series data analysis of the health status of the machine under fixed maintenance mode or fixed recession cycle is studied. In this research, the data fusion model is proposed to improve the overall accuracy through the combination of various-data driven technologies with the time series prediction method. The experimental result based on hydraulic system condition monitoring data shows that the proposed fusion model can provide better maintenance decision making on the machine/device maintenance plan based on the relatively small or data with incomplete maintenance cycle. [ABSTRACT FROM AUTHOR]

As the most stressed part of a gas turbine the first row of turbine blades is not only a challenge for the materials used. Also the testing of these parts have to meet the highest standards. Computed tomography (CT) as the technique which could reveal the most details also provides the biggest challenges [1]: A full penetration of large sized turbine blades is often only possible at high X-ray voltages causing disproportional high costs. A reduction of the X-ray voltage is able to reduce these arising costs but yields non penetration artifacts in the reconstructed CT image. In most instances, these artifacts manifests itself as blurred and smeared regions at concave edges due to a reduced signal to noise ratio. In order to complement the missing information and to increase the overall image quality of our reconstruction, we use further imaging modalities such as a 3-D Scanner and ultrasonic imaging. A 3-D scanner is easy and cost effective to implement and is able to acquire all relevant data simultaneously with the CT projections. If, however, the interior structure is of supplemental interest, an ultrasonic imaging method is additionally used. We consider this data as a priori knowledge to employ them in an iterative reconstruction. To do so, standard iterative reconstruction methods are modified to incorporate the a priori data in a regularization approach in combination with minimizing the total variation of our image. Applying this procedure on turbine blades, we are able to reduce the apparent artifacts almost completely. [ABSTRACT FROM AUTHOR]

In automated NDE a region of an inspected component is often interrogated several times, be it within a single data channel, across multiple channels or over the course of repeated inspections. The systematic combination of these diverse readings is recognized to provide a means to improve the reliability of the inspection, for example by enabling noise suppression. Specifically, such data fusion makes it possible to declare regions of the component defect-free to a very high probability whilst readily identifying indications. Registration, aligning input datasets to a common coordinate system, is a critical pre-computation before meaningful data fusion takes place. A novel scheme based on a multi-objective optimization is described. The developed data fusion framework, that is able to identify and rate possible indications in the dataset probabilistically, based on local data statistics, is outlined. The process is demonstrated on large data sets from the industrial ultrasonic testing of aerospace turbine disks, with major improvements in the probability of detection and probability of false call being obtained. [ABSTRACT FROM AUTHOR]

A novel, autonomous reconfigurable robotic inspection system for quantitative NDE mapping is presented. The system consists of a fleet of wireless (802.11g) miniature robotic vehicles, each approximately 175 x 125 x 85 mm with magnetic wheels that enable them to inspect industrial structures such as storage tanks, chimneys and large diameter pipe work. The robots carry one of a number of payloads including a two channel MFL sensor, a 5 MHz dry coupled UT thickness wheel probe and a machine vision camera that images the surface. The system creates an NDE map of the structure overlaying results onto a 3D model in real time. The authors provide an overview of the robot design, data fusion algorithms (positioning and NDE) and visualization software. [ABSTRACT FROM AUTHOR]

To resolve the problem of low identification merely relying on the EEG or EMG from the patients with perceptual-motor dysfunction, this paper proposes a quantitative description method to extract instantaneous feature of the EEG and EMG based on local mean decomposition and multiscale entropy. In the method, EEG and EMG are adaptively decomposed into product functions (PFs), and the PFs in different characteristic frequency bands are quantitatively described by the multiscale entropy to consist a multiple features information space (MFIS). Then, a fusion model based on Extreme Learning Machine (ELM) is established to realize the mapping from the EEG-EMG IMFS to human movement intention (HMI) by analyzing the MFIS. The HMI can be identified by the proposed model. 20 patients with perceptual-motor dysfunction of lower limbs (including 10 stroke patients in chronic phase (stroke) and 10 patients with peripheral nerve injury (PNI)) and 10 healthy people (HP) are enrolled. All subjects performed a knee extension and flexion task in which EEG and EMG of the quadriceps femoris and gastrocnemius muscle are synchronously recorded. Applying the above method, the feature of EEG-EMG are extracted and training in three groups (group 1: only EEG as input; group 2: only EMG as input; group 3: EEG-EMG as input) are performed. The results illustrate that the identification accuracy for group 1 can arrive 69.2% (stroke), 75.4% (PNI), 85.3% (HP); the identification accuracy for group 2 can arrive 80.5% (stroke), 79.6% (PNI), 93.4% (HP); the identification accuracy for group 3 can arrive 84.4% (stroke), 85.3% (PNI), 98.9% (HP). Conclusion is obtained that the human movement intention is reflected complementarily by EEG and EMG features. Especially for the perceptual-motor dysfunction patients, the algorithm can improve the identification accuracy on the human movement intention, and can be helpful in rehabilitation training process based on biological feedback control. [ABSTRACT FROM AUTHOR]

SGL for USaR is an EC-funded project which aims to solve issues related to human detection and hazard recognition in Urban Search and Rescue (USaR) operations. Prototype of a sensor network developed in the project combines on-site optical, acoustic and chemical analysis into robust communication networks. [ABSTRACT FROM AUTHOR]

Wavelet transform is an effective method of image fusion. But the decomposition level is an important factor which affects the image fusion effect and computing complexity. By using some evaluation methods of image fusion, such as entropy, standard deviation, mutual information, and quality measure methods was proposed by Wang & Bovik and Xing. Evaluation of image fusion was implemented with different wavelet type and different decomposition level. Experiment results indicate that 1-level is optimized decomposition level for different wavelet basis. [ABSTRACT FROM AUTHOR]

Splicing face pictures seamlessly is an important step in the 3D modeling process of human faces. Among the existing methods, the most popular way is the Multi-Degree Analysis Method. While this method offers a flexible, convenient multi-resolution format that mirrors the multiple scales of processing in the human visual system, it reduces the picture's overall clarity. This paper presents our approach of utilizing the iterative algorithm based on the n * n wave-low-pass filter to deepen the integration. The comparative study, design details, and initial results of our implementation are discussed in this paper. Experiments show that our approach has such benefits as easily implemented, more effective and avoiding the picture degradation. For a possible light imbalance of spliced pictures, two solutions are provided to enhance the fusion algorithm to reach an optimal effect of splicing human facial pictures. [ABSTRACT FROM AUTHOR]

A variable-weight state vector data fusion algorithm from active and passive radars is introduced to the target tracking system. Each radar adopts variable weight coefficient in the data fusion process instead of traditional invariable-weight method ignoring random errors caused by target distance. Simulation shows that the variable-weight data fusion algorithm has a good tracking performance and stability when random errors caused by target distance are relative large and cannot be ignored. [ABSTRACT FROM AUTHOR]