Search

Localisation and positioning of devices in indoor scenarios is one of the major research lines in the scientific community. While outdoor position solutions have already converged into an integration of GNSS networks, and scenarios with no GNSS connectivity still represent a challenge. A novel technique has been recently presented in , which allows the positioning of devices by overhearing network operations. Nevertheless, implementations of the time difference computations done at an operating system kernel level concluded that the results obtained were affected by non-linear latency. The operating system attempt finished by proposing to implement the same mechanism at firmware level, thus reducing the impact of the system latency. This bachelor thesis focuses on a first implementation of a passive positioning algorithm on a commercial off-the-shelf device through the use of time differences of arrival (TDOA) computed by exploiting the nature of the wireless shared medium access in Wi-Fi networks. The raw TDOA algorithm is integrated within a commercial wireless network card (WNIC) firmware. Since manufacturers keep the firmware integrated on the WNIC devices closed-software, the implementation is done on top of the open code project, OpenFWWF, released in 2018. The implemented algorithm exploits the request-response operations of the IEEE 802.11 medium access control (MAC), as the request to send (RTS) and clear to send(CTS) mechanism, to compute the time difference of arrival between the two frames. Additionally a platform to retrieve and control the access of the TDOA measures is designed and integrated as a kernel module. The platform is designed with scalability in mind, allowing for multiple user-level applications to leverage the TDOA measurements to provide localisation services. Lastly, a first behavioral analysis of the platform and algorithm versus the distance between network elements is done, in order to validate the implementation and its ability in capturing the time differences over increased distances.

For complex satellite missions that rely on agile and high-precision manoeuvres, the low-friction aspect of the space environment is a critical component in understanding the attitude control dynamics of the spacecraft. The Kinesthetic Node and Autonomous Table-Top Emulator (KNATTE) is a three-degree-of-freedom frictionless vehicle that serves as the foundation of a multipurpose platform for real-time spacecraft hardware-in-the-loop experiments, and allows emulation of these conditions in two dimensions with the purpose of validating various guidance, navigation, and control algorithms. The data acquisition of the vehicle depends on a computer vision system (CVS) that yields position and attitude data, but also suffers from unpredictable blackout events. To complement such measurements, KNATTE incorporates an inertial measurement unit (IMU) that yields accelerometer, gyroscope, and magnetometer data. This study describes a multisensor data fusion approach to obtain accurate attitude information by combining the measurements from the CVS and the IMU using nonlinear Kalman filter algorithms. To do this, the data fusion algorithms are developed and tested in a Matlab/Simulink environment. After that, the algorithms are adapted to the KNATTE platform and their performance is confirmed in various conditions. Through this work, the accuracy and efficiency of the approach can be checked by numerical simulation and real-time experiments. In addition, the quality of the CVS measurements are further improved by the introduction of a neural network to the image processing pipeline of the original system.

Multisensor data fusion demand in Earth observations is constantly increasing thanks to technological advances and the willingness to explore the Earth in a multidisciplinary way. Recently hyperspectral imaging has become a promising tool for Earth monitoring purposes but has also emerged as suitable for fusion with other remote sensors for various applications. This dissertation examines different types of multisensor data fusion, such as feature-level and application-level fusion, where each application is based on hyperspectral imaging at the airborne scale. In feature-level data fusion, hyperspectral imaging is combined with LiDAR (Light Detection and Ranging) to analyze urban environments, mainly focusing on urban land cover classification and implementing deep learning algorithms. In contrast, application-level data fusion presents the integration of hyperspectral imaging with magnetic data for material characterization of geologic complexes in remote and harsh environments, such as Greenland. This PhD thesis focused on enhancing analysis outcomes by combining hyperspectral imaging with other sensors and precisely selecting applications in which one sensor is insufficient to obtain the required parameters. The analysis of feature-level data fusion for hyperspectral and LiDAR data began with a detailed review of sensor key characteristics most representative of urban land cover analysis. These features were intended to segment land cover classes by considering 2D and 3D convolutional operations, where 2D convolutions involve spatial information and 3D convolutions add a spectral dimension allowing the inclusion of information about the interrelation of hyperspectral bands. The study on feature-level data fusion was completed with a multitemporal analysis, where a general framework was proposed towards automatical updating a local urban database. The other part of the dissertation was based on the fusion of sensors operating in different feature vectors with a common factor: identifying iron and its magnetic properties. Iron in hyperspectral imaging also has distinct absorption features recognizable at the relatively low spatial resolution. Moreover, it is the only chemical element capable of maintaining magnetic properties, which is the main aim of magnetic surveys. This dissertation has contributed new approaches to various feature-level and application-level multisensor data fusion exploitations confirming the great potential and versatility and showing future directions of multidisciplinary research using remote sensing methods for Earth observation. This work is part of the project “FKB maskinlæring” funded byRFF “Oslo og Akershus Regionale forskningsfond” (295836). In addition, the research was funded by the EnMAP scientific preparation program under the Space Agency at DLR with resources from the German Federal Ministry of Economic Affairs and Climate Action, grant number 50EE1529.

Multi-modal fusion has been proved to help enhance the performance of scene classification tasks. This paper presents a 2D-3D Fusion stage that combines 3D Geometric Features with 2D Texture Features obtained by 2D Convolutional Neural Networks. To get a robust 3D Geometric embedding, a network that uses two novel layers is proposed. The first layer, Multi-Neighbourhood Graph Convolution, aims to learn a more robust geometric descriptor of the scene combining two different neighbourhoods: one in the Euclidean space and the other in the Feature space. The second proposed layer, Nearest Voxel Pooling, improves the performance of the well-known Voxel Pooling. Experimental results, using NYU-Depth-V2 and SUN RGB-D datasets, show that the proposed method outperforms the current state-of-the-art in RGB-D indoor scene classification task., Comment: Information Fusion 2021

Multi-exposure image fusion has emerged as an increasingly important and interesting research topic in information fusion. It aims at producing an image with high quality by fusing a set of differently exposed images. In this article, we present a pixel-level method for multi-exposure image fusion based on an information-theoretic approach. In our scheme, an information channel between two source images is used to compute the Rényi entropy associated with each pixel in one image with respect to the other image and hence to produce the weight maps for the source images. Since direct weight-averaging of the source images introduce unpleasing artifacts, we employ Laplacian multi-scale fusion. Based on this pyramid scheme, images at every scale are fused by weight maps, and a final fused image is inversely reconstructed. Multi-exposure image fusion with the proposed method is easy to construct and implement and can deliver, in less than a second for a set of three input images of size 512×340, competitive and compelling results versus state-of-art methods through visual comparison and objective evaluation This work is supported by the National Natural Science Foundation of China under grant No.61702359, and by grant PID2019-106426RB-C31 and by grant PID2019-106426RB-C31 from the Agencia Estatal de Investigación (AEI) from Spanish Government. Part of this work was supported by a grant of the Romanian Ministry of Education and Research, CNCS – UEFISCDI, project number PN-III-P1-1.1-TE-2019-1111, within PNCDI III

The aim of this project is to modify, adapt, correct and test two target tracking algorithms to check their feasibility for future implementation in Advanced Driving Assistance Systems (ADAS). These systems, which range from automatic brake action to direct intervention in vehicle steering, require constant real-time monitoring of the environment (other cars, pedestrians, wild animals, etc.) and, in this respect, tracking algorithms have a crucial role to play, as they allow the continuous estimation of a target's trajectory in an accurate and efficient way. This project is the continuation of a project initiated by the Wireless Communications Research Unit of the Institute of Telecommunications of the TU Wien. As a starting point, two algorithms designed and implemented by the researchers working on the original project have been used. The first of these algorithms is a Particle Filter (PF) implemented in Python, developed to track a single target, while the second consists of a complex algorithm combining a Multiple Hypothesis Tracking (MHT) algorithm coupled to a Particle Filter (PF), also implemented in Python, with the intention of performing multiple target tracking. The project has been developed as follows. First, a random trajectory of a target was simulated in Matlab, using a random walk. Then, a Frequency Modulated Continuous Wave (FMCW) radar simulator, implemented in Matlab, developed by researchers at TU Wien, was used to perform the corresponding measurements. For the measurement process, a system consisting of four FMCW radars, placed in a square arrangement, was simulated. Finally, all data coming from the four radars was introduced into the two algorithms and combined by means of sensor fusion techniques in order to improve the quality of the trajectory estimates. El objetivo de este proyecto es modificar, adaptar, corregir y probar dos algoritmos de seguimiento de objetivos para comprobar su viabilidad de cara a su futura implantación en sistemas avanzados de asistencia a la conducción (ADAS). Estos sistemas, que van desde la actuación automática de los frenos hasta la intervención directa en la dirección del vehículo, requieren una monitorización constante en tiempo real del entorno (otros coches, peatones, animales salvajes, etc.) y, en este sentido, los algoritmos de seguimiento tienen un papel crucial, ya que permiten la estimación continua de la trayectoria de un objetivo de forma precisa y eficiente. Este proyecto es la continuación de un proyecto iniciado por la Unidad de Investigación de Comunicaciones Inalámbricas del Instituto de Telecomunicaciones de la TU Wien. Como punto de partida, se han utilizado dos algoritmos diseñados e implementados por los investigadores que trabajan en el proyecto original. El primero de estos algoritmos es un Filtro de Partículas (PF) implementado en Python, desarrollado para el seguimiento de un único objetivo, mientras que el segundo consiste en un complejo algoritmo que combina un algoritmo de Seguimiento de Hipótesis Múltiples (MHT) acoplado a un Filtro de Partículas (PF), también implementado en Python, con la intención de realizar el seguimiento de múltiples objetivos. El proyecto se ha desarrollado de la siguiente manera. En primer lugar, se simuló una trayectoria aleatoria de un objetivo en Matlab, utilizando un randomwalk. A continuación, se utilizó un simulador de radar de Onda Continua Modulada en Frecuencia (FMCW), implementado en Matlab, desarrollado por investigadores de TU Wien, para realizar las mediciones correspondientes. Para el proceso de medición, se simuló un sistema formado por cuatro radares FMCW, colocados en una disposición cuadrada. Por último, todos los datos procedentes de los cuatro radares se introdujeron en los dos algoritmos y se combinaron mediante técnicas de fusión de sensores para mejorar la calidad de las estimaciones de la trayectoria. L'objectiu d'aquest projecte és modificar, adaptar, corregir i provar dos algorismes de seguiment d'objectius per comprovar-ne la viabilitat de cara a la implantació futura en sistemes avançats d'assistència a la conducció (ADAS). Aquests sistemes, que van des de l'actuació automàtica dels frens fins a la intervenció directa a la direcció del vehicle, requereixen una monitorització constant en temps real de l'entorn (altres cotxes, vianants, animals salvatges, etc.) i, en aquest sentit, els algorismes de seguiment tenen un paper crucial, ja que permeten l'estimació contínua de la trajectòria d'un objectiu de manera precisa i eficient. Aquest projecte és la continuació d'un projecte iniciat per la Unitat de Recerca de Comunicacions Sense Fils de l'Institut de Telecomunicacions de la TU Wien. Com a punt de partida, s'han utilitzat dos algorismes dissenyats i implementats pels investigadors que treballen al projecte original. El primer d'aquests algoritmes és un Filtre de Partícules (PF) implementat a Python, desenvolupat per al seguiment d'un únic objectiu, mentre que el segon consisteix en un complex algorisme que combina un algorisme de seguiment d'hipòtesis múltiples (MHT) acoblat a un Filtre de Partícules (PF), també implementat a Python, amb la intenció de fer el seguiment de múltiples objectius. El projecte s'ha desenvolupat de la manera següent. En primer lloc, es va simular una trajectòria aleatòria d'un objectiu a Matlab, fent servir un randomwalk. A continuació, es va utilitzar un simulador de radar d'Onda Contínua Modulada en Freqüència (FMCW), implementat a Matlab, desenvolupat per investigadors de TU Wien, per realitzar els mesuraments corresponents. Per al procés de mesura, es va simular un sistema format per quatre radars FMCW, col·locats en una disposició quadrada. Finalment, totes les dades procedents dels quatre radars es van introduir als dos algoritmes i es van combinar mitjançant tècniques de fusió de sensors per millorar la qualitat de les estimacions de la trajectòria.

This paper addresses the robust Kalman filtering problem for multisensor time-varying systems with uncertainties of noise variances. Using the minimax robust estimation principle, based on the worst-case conservative system with the conservative upper bounds of noise variances, the robust local time-varying Kalman filters are presented. Further, the batch covariance intersection (BCI) fusion and a fast sequential covariance intersection (SCI) fusion robust time-varying Kalman filters are presented. They have the robustness that the actual filtering error variances or their traces are guaranteed to have a minimal upper bound for all admissible uncertainties of noise variances. Their robustness is proved based on the proposed Lyapunov equations approach. The concepts of the robust and actual accuracies are presented, and the robust accuracy relations are proved. It is also proved that the robust accuracies of the BCI and SCI fusers are higher than that of each local Kalman filter, the robust accuracy of the BCI fuser is higher than that of the SCI fuser, and the actual accuracies of each robust Kalman filter are higher than its robust accuracy for all admissible uncertainties of noise variances. The corresponding steady-state robust local and fused Kalman filters are also presented for multisensor time-invariant systems, and the convergence in a realization between the local and fused time-varying and steady-state Kalman filters is proved by the dynamic error system analysis (DESA) method and dynamic variance error system analysis (DVESA) method. A simulation example is given to verify the robustness and the correctness of the robust accuracy relations.

The correlations between smartphone sensors, algorithms, and relevant techniques are major components facilitating indoor localization and tracking in the absence of communication and localization standards. A major research gap can be noted in terms of explaining the connections between these components to clarify the impacts and issues of models meant for indoor localization and tracking. In this paper, we comprehensively study the smartphone sensors, algorithms, and techniques that can support indoor localization and tracking without the need for any additional hardware or specific infrastructure. Reviews and comparisons detail the strengths and limitations of each component, following which we propose a handheld-device-based indoor localization with zero infrastructure (HDIZI) approach to connect the abovementioned components in a balanced manner. The sensors are the input source, while the algorithms are used as engines in an optimal manner, in order to produce a robust localizing and tracking model without requiring any further infrastructure. The proposed framework makes indoor and outdoor navigation more user-friendly, and is cost-effective for researchers working with embedded sensors in handheld devices, enabling technologies for Industry 4.0 and beyond. We conducted experiments using data collected from two different sites with five smartphones as an initial work. The data were sampled at 10 Hz for a duration of five seconds at fixed locations; furthermore, data were also collected while moving, allowing for analysis based on user stepping behavior and speed across multiple paths. We leveraged the capabilities of smartphones, through efficient implementation and the optimal integration of algorithms, in order to overcome the inherent limitations. Hence, the proposed HDIZI is expected to outperform approaches proposed in previous studies, helping researchers to deal with sensors for the purposes of indoor navigation—in terms of either positioning or tracking—for use in various fields, such as healthcare, transportation, environmental monitoring, or disaster situations.

In IoT environments, voluminous amounts of data are produced every single second. Due to multiple factors, these data are prone to various imperfections, they could be uncertain, conflicting, or even incorrect leading to wrong decisions. Multisensor data fusion has proved to be powerful for managing data coming from heterogeneous sources and moving towards effective decision-making. Dempster–Shafer (D–S) theory is a robust and flexible mathematical tool for modeling and merging uncertain, imprecise, and incomplete data, and is widely used in multisensor data fusion applications such as decision-making, fault diagnosis, pattern recognition, etc. However, the combination of contradictory data has always been challenging in D–S theory, unreasonable results may arise when dealing with highly conflicting sources. In this paper, an improved evidence combination approach is proposed to represent and manage both conflict and uncertainty in IoT environments in order to improve decision-making accuracy. It mainly relies on an improved evidence distance based on Hellinger distance and Deng entropy. To demonstrate the effectiveness of the proposed method, a benchmark example for target recognition and two real application cases in fault diagnosis and IoT decision-making have been provided. Fusion results were compared with several similar methods, and simulation analyses have shown the superiority of the proposed method in terms of conflict management, convergence speed, fusion results reliability, and decision accuracy. In fact, our approach achieved remarkable accuracy rates of 99.32% in target recognition example, 96.14% in fault diagnosis problem, and 99.54% in IoT decision-making application.

Multisensor data fusion is extensively used to merge data from heterogeneous sensors in a smart environment. However, sensors provide noisy and uncertain information which is a big challenge for researchers. Since uncertainty in the data is a central constraint for data fusion and decision-making systems. Dempster-Shafer's evidence theory is an appropriate method for modeling and fusing uncertain information. In this paper, a novel data fusion scheme is proposed based on the modified belief entropy of the basic probability assignments (BPAs) to quantify the uncertainty in the information and fused them by Dempster-Shafer evidence theory. The proposed DFUDS (data fusion based on measuring uncertainty in Dempster-Shafer) scheme considers the available redundant information in the body of evidence (BoEs). The BoEs obtained from the sensor data are processed by proposed belief entropy, and fuse all pieces of evidence by Dempster's rule of combination to transfer the conflicting data into decision-making results. Extensive computer simulation results show that the proposed scheme outperforms in terms of the degree of uncertainty, evidence, reasoning, and decision accuracy under active contexts of the smart environment.

Technological innovations and advanced multidisciplinary research increase the demand for multisensor data fusion in Earth observations. Such fusion has great potential, especially in the remote sensing field. One sensor is often insufficient in analyzing urban environments to obtain comprehensive results. Inspired by the capabilities of hyperspectral and Light Detection and Ranging (LiDAR) data in multisensor data fusion at the feature level, we present a novel approach to the multitemporal analysis of urban land cover in a case study in Høvik, Norway. Our generic workflow is based on bitemporal datasets; however, it is designed to include datasets from other years. Our framework extracts representative endmembers in an unsupervised way, retrieves abundance maps fed into segmentation algorithms, and detects the main urban land cover classes by implementing 2D ResU-Net for segmentation without parameter regularizations and with effective optimization. Such segmentation optimization is based on updating initial features and providing them for a second iteration of segmentation. We compared segmentation optimization models with and without data augmentation, achieving up to 11% better accuracy after segmentation optimization. In addition, a stable spectral library is automatically generated for each land cover class, allowing local database extension. The main product of the multitemporal analysis is a map update, effectively detecting detailed changes in land cover classes.

In the recent years, remote sensing has faced a huge evolution. The constantly growing availability of remote sensing data has opened up new opportunities and laid the foundations for many new challenges. The continuous space missions and new constellations of satellites allow in fact more and more frequent acquisitions, at increasingly higher spatial resolutions, and at an almost total coverage of the globe. The availability of such an huge amount data has highlighted the need for automatic techniques capable of processing the data and exploiting all the available information. Meanwhile, the almost unlimited potential of machine learning has changed the world we live in. Artificial neural Networks have break trough everyday life, with applications that include computer vision, speech processing, autonomous driving but which are also the basis of commonly used tools such as online search engines. However, the vast majority of such models are of the supervised type and therefore their applicability rely on the availability of an enormous quantity of labeled data available to train the models themselves. Unfortunately, this is not the case with remote sensing, in which the enormous amounts of data are opposed to the almost total absence of ground truth. The purpose of this thesis is to find the way to exploit the most recent deep learning techniques, defining a common thread between two worlds, those of remote sensing and deep learning, which is often missing. In particular, this thesis proposes three novel contributions which face current issues in remote sensing. The first one is related to multisensor image registration and combines generative adversarial networks and non-linear optimization of crosscorrelation-like functionals to deal with the complexity of the setting. The proposed method was proved able to outperform state of the art approaches. The second novel contribution faces one of the main issues in deep learning for remote sensing: the scarcity of ground truth data for semantic segmentation. The proposed solution combines convolutional neural networks and probabilistic graphical models, two very active areas in machine learning for remote sensing, and approximate a fully connected conditional random field. The proposed method is capable of filling part of the gap which separate a densely trained model from a weakly trained one. Then, the third approach is aimed at the classification of high resolution satellite images for climate change purposes. It consist of a specific formulation of an energy minimization which allows to fuse multisensor information and the application a markov random field in a fast and efficient way for global scale applications. The results obtained in this thesis shows how deep learning methods based on artificial neural networks can be combined with statistical analysis to overcome their limitations, going beyond the classic benchmark environments and addressing practical, real and large-scale application cases.

Průnik Kovariancí je pravidlo lineární fúze pro slučování odhadů. Pokud není vzájemná korelační matice chyb dvou odhadů známa, je toto pravidlo optimální ve smyslu mezí. Článek rozpracovává případ, kdy jsou některé prvky této matice známy. Zavádí techniky pro konstruování třídy horních mezí sdružené matice střední kvadratické chyby. Všechny konfigurace fúzí až čtyř odhadů jsou uvažovány výslovně. Představené techniky jsou též použitelné pro fúze více než čtyř odhadů. Covariance Intersection fusion is a linear rule for combining estimates. If the cross-correlation matrix of the errors of two estimates is unknown, the rule is bound-optimal. This paper elaborates the case when some elements of the cross-correlation matrix are known. Techniques for constructing a family of upper bounds of the joint mean square error matrix are introduced. All configurations for the fusion of up to four estimates are considered explicitly. The techniques are also applicable for the fusion of more than four estimates.

Remote sensing using multisensor platforms has been systematically applied for monitoring and optimizing human activities. Several advanced techniques have been developed to enhance and extract the spatially and spectrally semantic information in the hyperspectral image (HSI) and light detection and ranging (LiDAR) data processing and analysis. However, an abundance of redundant information and sometimes a lack of discriminative features reduce the efficiency and effectiveness of multisource classification methods. This article proposes a fractional Gabor convolutional network (FGCN), focusing on efficient feature fusion and comprehensive feature extraction. First, the proposed FGCN uses Octave convolution layers to perform multisource information fusion and preserve discriminative information. Second, fractional Gabor convolutional (FGC) layers are proposed to extract multiscale, multidirectional, and semantic change features. The completeness and discrimination of the multisource features using different FGC kernels are improved, which yield robust feature extraction against semantic changes. Finally, the fractional Gabor feature and spectral feature are combined with two weighting factors which can be learned during the network training. Experimental results and comparisons with state-of-the-art multisource classification methods indicate the effectiveness of the proposed FGCN. With the FGCN, we can obtain an 89.90x0025; overall accuracy on the challenging Muufl Gulfport (MUUFL) data set, with an improvement of 3x0025; over state-of-the-art methods.

RadioSense project report presents the results and the validation testing activities carried out inside a human robot collaborative (HRC) workspace. In particular the results target the verification of the multisensory platform for worker monitoring and protection in collaborative robot (cobots) industrial environments. The deliverable describes experiments using different sensors and the data fusion (MDF) platform previously introduced and based on an edge-cloud architecture that supports the combination of multiple sensing technologies to enable the passive and anonymous detection of workers. Considering the HRC environment and the related risk mitigation policies, we evaluate the system robustness in terms of the minimum protective separation distance as it provides a standardized indicator of the minimum human-robot separation below which the robot must stop or replan its activities. For all the test, we focused on a typical speed separation monitoring (SSM) operation scheme where the robot(s) and the operator(s) may move concurrently in the shared workspace.

Perception and correct understanding of the road scene is crucial for any application of assisted and automated driving. In order to guarantee safety of the passenger and other road users, planning and navigation must be made on the basis of a reliable environment representation. Multi-sensor data and prior information is used to build this representation which incorporates identification of road users and road structure. For the latter, the focus is put on the drivablespace and lane repartition. On highways, urban streets and generally all over the road network, the drivable space is organized in oriented corridors which enablesafer and predictable navigation for everyone. In the development of intelligentvehicles, identifying the lane repartition and building an accurate road representation outlines the lane boundaries detection task. Depending on the specifics of the target automated system, car manufacturers integrate in currently commercialized vehicles ready-to-use lane detection solutions from Tier-1 suppliers generally featuring single and vision-based smart sensors. Such solutions may not be adequate in highly automated systems where the driver is allowed to divert their attention from the driving task and become passenger. This thesis addresses the problem of lane boundaries identification relying on multi-sensor fusion of smart camera data (specifically, frontal and AVM cameras) and HD-maps. In the first part, an appropriate modeling for smart sensor measurements which is independent from the sensor nature is proposed. Uncertain detections of markings, barriers and other road elements contribute to the tracking of the surrounding lane boundaries using a novel clothoid-spline model. The second part focuses on the integration of prior information coming from digital maps. Similarly to the modeling of smart sensors, the involved uncertainties in the usage of map-providers have been taken into account to support the lane boundaries estimation. For the testing of the proposed approaches, a custom dataset of road data has been recorded using both off-the-shelf smart sensors and live streamed HD-maps. Validated and tuned tracking solutions are then integrated in close-loop experimentations on Renault prototype vehicle of SAE Level 3 of automation., La perception et la compréhension correcte de la scène routière sont cruciales pour toute application de conduite assistée et automatisée. Afin de garantir la sécurité du passager et des autres usagers de la route, la planification et la navigation doivent être effectuées sur la base d’une représentation fiable de l’environnement. Les données multi-capteurs et les informations préalables sont utilisées pour construire cette représentation qui intègre l’identification des usagers de la route et de la structure de la route. Pour cette dernière, l’accent est mis sur l’espace de conduite et la répartition en voies. Sur les autoroutes, les rues urbaines et plus généralement sur l’ensemble du réseau routier, l’espace carrossable est organisé en couloirs orientes qui permettent une navigation plus sure et prévisible pour tous. Dans le cadre du développement des véhicules intelligents, l’identification de la répartition des voies et la construction d’une représentation précise de la route constituent la tache de détection des limites des voies. En fonction des spécificités du système automatise cible, les constructeurs automobiles intègrent dans les véhicules actuellement commercialises des solutions de détection des voies prêtes à l’emploi provenant de fournisseurs Tier-1 et comprenant généralement des capteurs intelligents simples et bases sur la vision. Ces solutions peuvent ne pas être adéquates dans les systèmes hautement automatisés où le conducteur est autorisé à détourner son attention de la tache de conduite pour devenir un passager. Cette thèse aborde le problème de l’identification des limites de voies en s’appuyant sur la fusion multi-capteurs des données des caméras intelligentes (en particulier, les cameras frontales et AVM) et des cartes HD. Dans la première partie, une modélisation appropriée pour les mesures des capteurs intelligents, indépendante de la nature du capteur, est proposée. Les détections incertaines des marquages, des barrières et d’autres éléments de la route contribuent au suivi des limites des voies environnantes à l’aide d’un nouveau modèle de splines de clothoides. La deuxième partie se concentre sur l’intégration d’informations préalables provenant de cartes numériques. Comme pour la modélisation des capteurs intelligents, les incertitudes liées à l’utilisation des fournisseurs de cartes ont été prises en compte pour aider à l’estimation des limites de la voie. Pour tester les approches proposées, un ensemble sur mesure de données routières a été enregistré en utilisant à la fois des capteurs intelligents disponibles dans le commerce et des cartes HD live stream. Les solutions de suivi validées et tunées sont ensuite intégrées dans des expérimentations en boucle fermée sur un véhicule prototype Renault de niveau 3 d’automatisation SAE., La percezione e la corretta comprensione della scena stradale e fondamentale per qualsiasi applicazione di guida assistita e automatizzata. Per garantire la sicurezza del passeggero e degli altri utenti della strada, la pianificazione e la navigazione devono essere effettuate sulla base di una rappresentazione affidabile dell’ambiente. Dati di origine multi-sensore e informazioni disponibili a priori sono utilizzati per costruire questa rappresentazione che incorpora l’identificazione degli utenti della strada e la struttura della strada stessa. Per quest’ultima, l’attenzione e posta sullo spazio percorribile e sulla ripartizione in corsie. Sulle autostrade, le strade urbane e in generale su tutta la rete stradale, lo spazio percorribile e organizzato in corridoi orientati che permettono una navigazione piu sicura e prevedibile per tutti. Nello sviluppo di veicoli intelligenti, l’identificazione della ripartizione in corsie e la costruzione di una rappresentazione accurata della strada delinea il compito di rilevamento dei confini delle corsie o lane boundaries detection. A seconda delle specifiche del sistema automatizzato di destinazione, le case automobilistiche integrano nei veicoli attualmente commercializzati soluzioni di rilevamento di corsia pronte all’uso da fornitori Tier-1, generalmente composte di singoli sensori intelligenti e basate sulla visione computerizzata. Tali soluzioni potrebbero non essere adeguate in sistemi altamente automatizzati dove al guidatore e permesso di distogliere l’attenzione dal compito di guida e di diventare passeggero. Questa tesi di dottorato affronta il problema dell’identificazione dei limiti di corsia basandosi sulla fusione multi-sensore di dati provenienti da telecamere intelligenti (in particolare, telecamere frontali e AVM) e mappe HD. Nella prima parte, viene proposta una modellazione appropriata per le misure dei sensori intelligenti che e indipendente dalla natura del sensore. I rilevamenti incerti di marcature, barriere e altri elementi stradali contribuiscono alla stima dei limiti delle corsie circostanti utilizzando un nuovo modello di spline di clotoidi. La seconda parte si concentra sull’integrazione di informazioni provenienti da mappe digitali. Analogamente alla modellazione dei sensori intelligenti, le incertezze coinvolte nell’uso di map-providers sono state prese in considerazione per supportare l’identificazione dei limiti di corsia. Per testare gli approcci proposti, e stato registrato un dataset personalizzato di dati stradali utilizzando sia sensori intelligenti off-the-shelf che mappe HD in live streaming. Le soluzioni di tracking convalidate e correttamente regolate sono poi integrate in sperimentazioni a circuito chiuso su un veicolo prototipo Renault di livello 3 di automazione SAE.

The process of combining data and estimates is inherent in estimation problems. This paper focuses on the linear fusion under the assumption that only some elements of the cross-correlation matrix of the estimation errors are known. Configurations of the knowledge are discussed individually for up to five estimates. For an arbitrary number of estimates, a general construction of upper bounds of the joint mean square error matrix is proposed. Last, the relation with the Split Covariance Intersection fusion is discussed. © 2022 International Society of Information Fusion.

This work deals with the development of an adaptive multisensor data fusion technique for the accurate estimation of the trains position and velocity. The proposed technique will work with the Train Collision Avoidance System (TCAS) used in Indian railways during Global Positioning System (GPS) outages. The determination of accurate position of trains is a challenging task for the TCAS during GPS outages. The accuracy of the proposed Volterra Recursive Least Square (VRLS) based adaptive multisensor data fusion technique is evaluated by generating two kinematic profiles for a passenger train running between Silchar–Lumding broad gauge route in Indian railways. The effect of accelerometer bias is also considered during the analysis. It is observed that the developed technique can provide a better estimate of the position and velocity for the TCAS especially during GPS outages and without using any additional railway infrastructure. The simulation results indicate that the proposed technique is superior to the earlier works in terms of achieving better positional accuracy in presence of accelerometer bias.

La estimación de profundidad usando diferentes sensores es uno de los desafíos clave para dotar a las máquinas autónomas de sólidas capacidades de percepción robótica. Ha habido un avance sobresaliente en el desarrollo de técnicas de estimación de profundidad unimodales basadas en cámaras monoculares, debido a su alta resolución o sensores LiDAR, debido a los datos geométricos precisos que proporcionan. Sin embargo, cada uno de ellos presenta inconvenientes inherentes, como la alta sensibilidad a los cambios en las condiciones de iluminación en el caso delas cámaras y la resolución limitada de los sensores LiDAR. La fusión de sensores se puede utilizar para combinar los méritos y compensar las desventajas de estos dos tipos de sensores. Sin embargo, los métodos de fusión actuales funcionan a un alto nivel. Procesan los flujos de datos de los sensores de forma independiente y combinan las estimaciones de alto nivel obtenidas para cada sensor. En este proyecto, abordamos el problema en un nivel bajo, fusionando los flujos de sensores sin procesar, obteniendo así estimaciones de profundidad que son densas y precisas, y pueden usarse como una fuente de datos multimodal unificada para problemas de estimación de nivel superior. Este trabajo propone un modelo de campo aleatorio condicional (CRF) con múltiples potenciales de geometría y apariencia que representa a la perfección el problema de estimar mapas de profundidad densos a partir de datos de cámara y LiDAR. El modelo se puede optimizar de manera eficiente utilizando el algoritmo Conjúgate Gradient Squared (CGS). El método propuesto se evalúa y compara utilizando el conjunto de datos proporcionado por KITTI Datset. Adicionalmente, se evalúa cualitativamente el modelo, usando datos adquiridos por el autor de esté trabajo Multi-modal depth estimation is one of the key challenges for endowing autonomous machines with robust robotic perception capabilities. There has been an outstanding advance in the development of uni-modal depth estimation techniques based on either monocular cameras, because of their rich resolution or LiDAR sensors due to the precise geometric data they provide. However, each of them suffers from some inherent drawbacks like high sensitivity to changes in illumination conditions in the case of cameras and limited resolution for the LiDARs. Sensor fusion can be used to combine the merits and compensate the downsides of these two kinds of sensors. Nevertheless, current fusion methods work at a high level. They processes sensor data streams independently and combine the high level estimates obtained for each sensor. In this thesis, I tackle the problem at a low level, fusing the raw sensor streams, thus obtaining depth estimates which are both dense and precise, and can be used as a unified multi-modal data source for higher level estimation problems. This work proposes a Conditional Random Field (CRF) model with multiple geometry and appearance potentials that seamlessly represents the problem of estimating dense depth maps from camera and LiDAR data. The model can be optimized efficiently using the Conjugate Gradient Squared (CGS) algorithm. The proposed method was evaluated and compared with the state-of-the-art using the commonly used KITTI benchmark dataset. In addition, the model is qualitatively evaluated using data acquired by the author of this work. Maestría Magíster en Ingeniería de Desarrollo de Productos

While wireless sensor networks (WSNs) have been traditionally tasked with single applications, in recent years we have witnessed the emergence of WSNs that allow the sensing and communication infrastructure to be shared among multiple applications thus optimizing the use of resources. As the number of applications in a WSN increases, a growing amount of sensor-generated data will be produced, from which useful information can be extracted. A major requirement in these networks is to save energy in order to extend their operational lifetime. However, wireless sensors and actuators commonly rely on batteries as their energy sources, whose replacement is undesirable or unfeasible. Among the methods employed to extend network lifetime, Multisensor data fusion (MDF) is one of the most widely used. Traditional MDFs are not able to identify different contexts, since they are designed using an application-specific design for the network. As the number of applications increases, the application data ranges overlap and it becomes more complex to identify the origin of each data sample to deliver data to the correct application, with the consequence of reducing data accuracy. In order to overcome these limitations, we propose a MDF technique that divides the monitored interval into a set of intervals (non-overlapped intervals and overlapped intervals) and attributes each interval to an abstract sensor. Then we use MDFs to identify (hidden) correlations in abstract sensors and to exploit such knowledge to monitor the behavior of sensors during their working life. Our proposal is validated through simulations and tests on real nodes.

Southern European functional urban areas (FUAs) are increasingly subject to heatwave (HW) events, calling for anticipated climate adaptation measures. In the urban context, such adaptation strategies require a thorough understanding of the built-up response to the incoming solar radiation, i.e., the urban energy balance cycle and its implications for the Urban Heat Island (UHI) effect. Despite readily available, diurnal Land Surface Temperature (LST) data does not provide a meaningful picture of the UHI, in these midlatitudes FUAs. On the contrary, the mid-morning satellite overpass is characterized by the absence of a significant surface UHI (SUHI) signal, corresponding to the period of the day when the urban-rural air temperature difference is typically negative. Conversely, nocturnal high-resolution LST data is rarely available. In this study, an energy balance-based machine learning approach is explored, considering the Local Climate Zones (LCZ), to describe the daily cycle of the heat flux components and predict the nocturnal SUHI, during an HW event. While the urban and rural spatial outlines are not visible in the diurnal thermal image, they become apparent in the latent and storage heat flux maps – built-up infrastructures uptake heat during the day which is released back into the atmosphere, during the night, whereas vegetation land surfaces loose diurnal heat through evapotranspiration. For the LST prediction model, a random forest (RF) approach is implemented. RF results show that the model accurately predicts the LST, ensuring mean square errors inferior to 0.1 K. Both the latent and storage heat flux components, together with LCZ classification, are the most important explanatory variables for the nocturnal LST prediction, supporting the adoption of the energy balance approach. In future research, other locations and time-series data shall be trained and tested, providing an efficient local urban climate monitoring tool, where in-situ air temperature observations are not available.

The ever-growing developments in technology to capture different types of image data [e.g., hyperspectral imaging and light detection and ranging (LiDAR)-derived digital surface model (DSM)], along with new processing techniques, have led to a rising interest in imaging applications for Earth observation. However, analyzing such datasets in parallel, remains a challenging task. In this article, we propose a multisensor deep clustering (MDC) algorithm for the joint processing of multisource imaging data. The architecture of MDC is inspired by autoencoder (AE)-based networks. The MDC paradigm includes three parallel networks, a spectral network using an autoencoder structure, a spatial network using a convolutional autoencoder structure, and lastly, a fusion network that reconstructs the concatenated image information from the concatenated latent features from the spatial and spectral network. The proposed algorithm combines the reconstruction losses obtained by the aforementioned networks to optimize the parameters (i.e., weights and bias) of all three networks simultaneously. To validate the performance of the proposed algorithm, we use two multisensor datasets from different applications (i.e., geological and rural sites) as benchmarks. The experimental results confirm the superiority of our proposed deep clustering algorithm compared to a number of state-of-the-art clustering algorithms. The code will be available at [Online]. Available: https://github.com/Kasra2020/MDC.

During this thesis, some perception approaches for self-driving vehicles were developed using de convolutional neural networks applied to monocular camera images and High-Definition map (HD-ma rasterized images. We focused on camera-only solutions instead of leveraging sensor fusion with rang sensors because cameras are the most cost-effective and discrete sensors. The objective was also to show th camera-based approaches can perform at par with LiDAR-based solutions on certain 3D vision tasks. Rea world data was used for training and evaluation of the developed approaches but simulation was als leveraged when annotated data was lacking or for safety reasons when evaluating driving capabilities. Cameras provide visual information in a projective space where the perspective effect does not preserve th distances homogeneity. Scene understanding tasks such as semantic segmentation are then often operated i the camera-view space and then projected to 3D using a precise depth sensor such as a LiDAR. Having thi scene understanding in the 3D space is useful because the vehicles evolve in the 3D world and the navigatio algorithms reason in this space. Our focus was then to leverage the geometric knowledge about the camer parameters and its position in the 3D world to develop an approach that allows scene understanding in the 3D space using only a monocular image as input. Neural networks have also proven to be useful for more than just perception and are more and more used fo the navigation and planning tasks that build on the perception outputs. Being able to output 3D scen understanding information from a monocular camera has also allowed us to explore the possibility of havin an end-to-end holistic neural network that takes a camera image as input, extracts intermediate semantic information in the 3D space and then lans the vehicle's trajectory., Au cours de cette thèse, des approches de perception pour les véhicules autonomes ont été développées en utilisant des réseaux de neurones convolutifs profonds appliqués à des images de caméra monoculaire et à des images rastérisées de cartes haute définition (HD-map). Nous nous sommes concentrés sur des solutions utilisant uniquement la caméra au lieu de tirer parti de la fusion de capteurs avec des capteurs de distance, car les caméras sont les capteurs les plus rentables et les plus discrets. L'objectif était également de montrer que les approches basées sur des caméras peuvent fonctionner au même niveau que les solutions basées sur LiDAR sur certaines tâches de vision 31). Des données du monde réel ont été utilisées pour l'entraînement et l'évaluation des approches développées, mais la simulation a également été exploitée lorsque les données annotées faisaient défaut ou pour des raisons de sécurité lors de l'évaluation des capacités de conduite. Les caméras fournissent des informations visuelles dans un espace projectif où l'effet de perspective ne préserve pas l'homogénéité des distances. Les tâches de compréhension de scène telles que la segmentation sémantique sont ensuite souvent effectuées dans l'espace de vue de la caméra, puis projetées en 3D à l'aide d'un capteur de profondeur précis tel qu'un LiDAR. Avoir cette compréhension de scène dans l'espace 31) est utile car les véhicules évoluent dans le monde 3D et les algorithmes de navigation raisonnent dans cet espace. Notre objectif était alors d'exploiter les connaissances géométriques sur les paramètres de la caméra et sa position dans le monde 3D pour développer une approche qui perm la compréhension de la scène dans l'espace 31) en utilisant uniquement une image monoculaire comme entrée. Les réseaux de neurones se sont également avérés utiles pour plus que la simple perception et sont de plus en plus utilisés pour les tâches de navigation et de planification qui s'appuient sur les sorties de perception. Etre capable de produire des informations de compréhension de scène 31) à partir d'une caméra monoculaire nous a également permis d'explorer la possibilité d'avoir un réseau neuronal holistique de bout en bout qui prend une image de caméra en entrée, extrait des informations sémantiques intermédiaires dans l'espace 3D, puis planifie la trajectoire du véhicule.

For efficient operation, modern control approaches for biochemical process engineering require information on the states of the process such as temperature, humidity or chemical composition. Those measurement are gathered from a set of sensors which differ with respect to sampling rates and measurement quality. Furthermore, for biochemical processes in particular, analysis of physical samples is necessary, e.g., to infer cellular composition resulting in delayed information. As an alternative for the use of this delayed measurement for control, so-called soft-sensor approaches can be used to fuse delayed multirate measurements with the help of a mathematical process model and provide information on the current state of the process. In this manuscript we present a complete methodology based on cascaded unscented Kalman filters for state estimation from delayed and multi-rate measurements. The approach is demonstrated for two examples, an exothermic chemical reactor and a recently developed model for biopolymer production. The results indicate that the the current state of the systems can be accurately reconstructed and therefore represent a promising tool for further application in advanced model-based control not only of the considered processes but also of related processes.

This article investigates the calibration of low-cost sensors for air pollution. The sensors were deployed on three Internet of Things (IoT) platforms in Spain, Austria, and Italy during the summers of 2017, 2018, and 2019. One of the biggest challenges in the operation of an IoT platform, which has a great impact on the quality of the reported pollution values, is the calibration of the sensors in an uncontrolled environment. This calibration is performed using arrays of sensors that measure cross sensitivities and therefore compensate for both interfering contaminants and environmental conditions. This article investigates how the fusion of data taken by sensor arrays can improve the calibration process. In particular, calibration with sensor arrays, multisensor data fusion calibration with weighted averages, and multisensor data fusion calibration with machine learning models are compared. Calibration is evaluated by combining data from various sensors with linear and nonlinear regression models., National Spanish funding; Regional Project; EU H2020 CAPTOR Project; AGAUR SGR44; 10.13039/501100011033-Agencia Estatal de Investigación; Spanish Ministry of Economy, Industry and Competitiveness

Worker monitoring and protection in collaborative robot (cobots) industrial environments requires advanced sensing capabilities and flexible solutions to monitor the movements of the operator in close proximity of moving robots. Collaborative robotics is an active research area where Internet of Things (IoT) and novel sensing technologies are expected to play a critical role. Considering that no single technology can currently solve the problem of continuous worker monitoring, the paper targets the development of an IoT multisensor data fusion (MDF) platform. It is based on an edge-cloud architecture that supports the combination and transformation of multiple sensing technologies to enable the passive and anonymous detection of workers. Multidimensional data acquisition from different IoT sources, signal pre-processing, feature extraction, data distribution and fusion, along with machine learning (ML) and computing methods are described. The proposed IoT platform also comprises a practical solution for data fusion and analytics. It is able to perform opportunistic and real-time perception of workers by fusing and analyzing radio signals obtained from several interconnected IoT components, namely a multi-antenna WiFi installation (2.4-5 GHz), a sub-THz imaging camera (100 GHz), a network of radars (122 GHz) and infrared sensors (8-13 {\mu}m). The performance of the proposed IoT platform is validated through real use case scenarios inside a pilot industrial plant in which protective human--robot distance must be guaranteed considering latency and detection uncertainties., Comment: This work has been partially funded by MUR Italy and Academy of Finland within the ERA-NET CO-FUND H2020 CHISTERA III project RadioSense. The paper has been published in the IEEE Internet of Things Journal

In this study , we have focused on the automatic segmentation of events in the uterine EMG signal and then on the identification of contractions among these events by referring to the expert's knowledge. Our database includes uterine EMG signals of different weeks of gestation acquired through a matrix of 4x4 electrodes. Therefore, our work has first included an application of the dynamic cumulative sum (DCS) method in a monodimensional approach on monopolar signals in order to obtain a high spatial resolution of the data. Based on the obtained results, our study has then focused on bipolar signals in order to increase the signal-to-noise ratio (SNR) of uterine EMGs. In fact, the DCS method has continued by associating first a series of techniques for the elimination of false detected ruptures either based on Fisher or on the SNR and by developing secondly two fusion methods of these ruptures : the firts one is automatic while the other one is based on the weighted majority voting system, where each channel is weighted by a factor when merging the instants of detected ruptures. In addition, the DCS method is applied in a multidimensional approach, first on the bipolar signals, then on their details after wavelet decomposition. Infact, we were interested in the dynamic selection of these details in both approaches by using a technique based on the Kullback Leibler ditance. Finally, in order to indentify the contractions and reduce the number of other detected events, an assay of parameters extraction of these obtained events has been presented and validated.; Dans cette étude, nous avons mis l’accent sur la segmentation automatique des évènements dans le signal EMG utérin et l’identification ensuite des contractions parmi ces évènements en se référant aux données de l’expert. Notre base de données comprend des signaux EMG utérins de différentes semaines de gestation acquis grâce à une matrice de 4x4 électrodes. Par conséquent, notre travail a compris tout d’abord une application de la méthode de somme cumulé dynamique (DCS) en approche monodimensionnelle sur les signaux monopolaires afin d’obtenir une grande résolution spatiale des données. Suite aux résultats obtenus, notre étude a porté sur les signaux bipolaires afin d'augmenter le rapport signal/bruit (SNR) des EMG utérin. En fait, la méthode DCS a continué en y associant une série des techniques d’éliminations des fausses ruptures détectées, soit basée sur Fisher, soit sur le SNR d’une part, et en développant des méthodes de fusion de ces ruptures d’autre part : l’une automatique tandis que l’autre est basée sur le système de vote à la majorité pondérée où chaque canal est pondéré par un facteur lors de la fusion des instants de ruptures détectés. De plus, la méthode DCS est appliquée en approche multidimensionnelle, tout d’abord sur ces signaux bipolaires, ensuite sur leurs détails après décomposition en ondelettes. En fait, nous sommes intéressés à la sélection dynamique de ces détails dans les deux approches en utilisant une technique basée sur la distance Kullback Leibler. Enfin et dans le but d’identifier les contractions et de réduire le nombre des autres évènements détectés, un essai d’extraction des paramètres de ces évènements obtenus est présenté et validé.

In this study , we have focused on the automatic segmentation of events in the uterine EMG signal and then on the identification of contractions among these events by referring to the expert's knowledge. Our database includes uterine EMG signals of different weeks of gestation acquired through a matrix of 4x4 electrodes. Therefore, our work has first included an application of the dynamic cumulative sum (DCS) method in a monodimensional approach on monopolar signals in order to obtain a high spatial resolution of the data. Based on the obtained results, our study has then focused on bipolar signals in order to increase the signal-to-noise ratio (SNR) of uterine EMGs. In fact, the DCS method has continued by associating first a series of techniques for the elimination of false detected ruptures either based on Fisher or on the SNR and by developing secondly two fusion methods of these ruptures : the firts one is automatic while the other one is based on the weighted majority voting system, where each channel is weighted by a factor when merging the instants of detected ruptures. In addition, the DCS method is applied in a multidimensional approach, first on the bipolar signals, then on their details after wavelet decomposition. Infact, we were interested in the dynamic selection of these details in both approaches by using a technique based on the Kullback Leibler ditance. Finally, in order to indentify the contractions and reduce the number of other detected events, an assay of parameters extraction of these obtained events has been presented and validated., Dans cette étude, nous avons mis l’accent sur la segmentation automatique des évènements dans le signal EMG utérin et l’identification ensuite des contractions parmi ces évènements en se référant aux données de l’expert. Notre base de données comprend des signaux EMG utérins de différentes semaines de gestation acquis grâce à une matrice de 4x4 électrodes. Par conséquent, notre travail a compris tout d’abord une application de la méthode de somme cumulé dynamique (DCS) en approche monodimensionnelle sur les signaux monopolaires afin d’obtenir une grande résolution spatiale des données. Suite aux résultats obtenus, notre étude a porté sur les signaux bipolaires afin d'augmenter le rapport signal/bruit (SNR) des EMG utérin. En fait, la méthode DCS a continué en y associant une série des techniques d’éliminations des fausses ruptures détectées, soit basée sur Fisher, soit sur le SNR d’une part, et en développant des méthodes de fusion de ces ruptures d’autre part : l’une automatique tandis que l’autre est basée sur le système de vote à la majorité pondérée où chaque canal est pondéré par un facteur lors de la fusion des instants de ruptures détectés. De plus, la méthode DCS est appliquée en approche multidimensionnelle, tout d’abord sur ces signaux bipolaires, ensuite sur leurs détails après décomposition en ondelettes. En fait, nous sommes intéressés à la sélection dynamique de ces détails dans les deux approches en utilisant une technique basée sur la distance Kullback Leibler. Enfin et dans le but d’identifier les contractions et de réduire le nombre des autres évènements détectés, un essai d’extraction des paramètres de ces évènements obtenus est présenté et validé.

Road traffic accidents have always been a concern to the driving community which has led to various research developments for improving the way we drive the vehicles. Since human error causes most of the road accidents, introducing automation in the vehicle is an efficient way to address this issue thus making the vehicles intelligent. This approach has led to the development of ADAS (Advanced Driver Assistance Systems) functionalities. The process of introducing automation in the vehicle is continuously evolving. Currently the research in this field has targeted full autonomy of the vehicle with the aim to tackle the road safety to its fullest potential. The gap between ADAS and full autonomy is not narrow. One of the approach to bridge this gap is to introduce collaboration between human driver and autonomous system. There have been different methodologies such as haptic feedback, cooperative driving where the autonomous system adapts according to the human driving inputs/intention for the corrective action each having their own limitations. This work addresses the problem of shared control authority between human driver and autonomous driving system without haptic feedback using the fusion of driving inputs. The development of shared control authority is broadly divided into different stages i.e. shared control framework, driving input assessment, driving behavior prediction, fusion process etc. Conflict resolution is the high level strategy introduced in the framework for achieving the fusion. The driving inputs are assessed with respect to different factors such as collision risk, speed limitation,lane/road departure prevention etc in the form of degree of belief in the driving input admissibility using sensor data. The conflict resolution is targeted for a particular time horizon in the future using a sensor based driving input prediction using neural networks. A two player non-cooperative game (incorporating admissibility and driving intention) is defined to represent the conflict resolution as a bargaining problem. The final driving input is computed using the Nash equilibrium. The shared control strategy is validated using a test rig integrated with the software Simulink and IPG CarMaker. Various aspects of shared control strategy such as human-centered, collision avoidance, absence of any driving input, manual driving refinement etc were included in the validation process.; Ce travail aborde le problème de l’autorité de contrôle partagée entre les conducteurs et système de conduite autonome sans retour haptique utilisant la fusion des entrées de conduite. Le développement d’une autorité de contrôle partagée est divisé en différentes étapes : cadre de contrôle partagé, évaluation des facteurs de conduite, prévision du comportement de conduite, processus de fusion, etc. La résolution des conflits est la stratégie de haut niveau introduite dans le cadre permettant de réaliser la fusion. Les entrées de conduite sont évaluées en fonction de différents facteurs tels que le risque de collision, la limitation de vitesse, la prévention de voie / départ, etc., sous la forme d’un degré de confiance dans l’admissibilité d’une entrée de conduite à l’aide de données de capteur. La résolution de conflit est ciblée pour un horizon temporel particulier dans le futur en utilisant une prédiction d’entrée de conduite basée sur un capteur utilisant des réseaux de neurones. Un jeu non coopératif à deux joueurs (comprenant l’admissibilité et l’intention de conduite) est défini comme représentant la résolution du conflit comme un problème de négociation. L’entrée motrice finale est calculée en utilisant l’équilibre de Nash. La stratégie de contrôle partagé est validée à l’aide d’un banc d’essai intégré aux logiciels Simulink et IPG CarMaker. Divers aspects de la stratégie de contrôle partagé, tels que l’accent mis sur l’homme, la prévention des collisions, l’absence de toute information sur la conduite, l’affinement de la conduite manuelle, etc., ont été inclus dans le processus de validation.

Zukünftige technische Systeme werden in Form von Companion-Systemen realisiert sein. Diese können sich hoch adaptiv auf individuelle Bedürfnisse und Befindlichkeiten ihrer Nutzer anpassen und gehen so in ihrem Bedienkomfort weit über die Fähigkeiten gegenwärtiger smarter Technologien hinaus. Zur Realisierung solcher Funktionen werden Companion-Systeme, neben anderen Komponenten, mit einer Vielzahl von Sensoren ausgestattet sein, welche es erlauben den Nutzer multimodal in Sprache, Gestik, Mimik und Biophysiologie zu erfassen. Ein Companion-System ist so in der Lage den emotionalen Zustand seiner Nutzer zu bewerten. Da emotionales Verhalten sehr komplex ist, werden hierfür Verfahren der Mustererkennung und des maschinellen Lernens eingesetzt um multimodale Sensordaten auf emotionale Kategorien und Intensitäten abzubilden. Diese Verfahren Lernen dabei anhand von umfangreichen Trainingsdaten. Die Arbeit beschreibt zunächst Eigenschaften, welche diese Daten erfüllen müssen, um zum Training von multimodalen Klassifikatoren geeignet zu sein. Mit dem uulmMAC wird ein multimodaler Korpus vorgestellt, welcher die notwendigen Eigenschaften erfüllt. Der uulmMAC verwendet dabei neben klassischen kontinuierlichen Emotionsmodellen exemplarisch Mensch-Maschine-Interaktionstypische Elemente wie mentale Über- und Unterforderung oder Frustration. Da solch umfangreiche Datenbasen bisher nur unter enormem Aufwand manuell annotiert werden können, werden multimodale Ansätze des aktiven Lernens vorgeschlagen, um eine effiziente Annotation unter Verwendung des Annotationswerkzeuges ATLAS zu ermöglichen. ATLAS begleitet den Prozess der Entwicklung emotionserkennender Komponenten der Companion-Technologie bei der Annotation, aber auch bereits beim explorativen Traingsdatenentwurf sowie, dem Klassifikatorentwurf, der Evaluation und dem prototypischen Systementwurf. Aufbauend auf dem annotierten uulmMAC werden multimodale Klassifikationsverfahren vorgestellt und evaluiert, welche es ermöglichen den mentalen Belastungszustand eines Nutzers zu erfassen. Diese werden abschließend in prototypische Companion-Systemen eingesetzt, und auf ihre reale Anwendungsfähigkeit hin bewertet. Dabei kommt die, bereits zur sensorischen Datenerfassung des uulmMAC, vorgeschlagene MAR2S Architektur erneut zum Einsatz, um einen vollständig modularen Aufbau der Prototypen zu erlauben.

After the democratization of industrial robots, the current trend is the development of social robots that create strong interactions with their users. The deployment of such platforms in shops, museums or train stations raises various issues including the autonomous localization of mobile robots. This thesis focuses on the localization of Pepper robots in a non-constrained indoor environment. Pepper robots are daily used in many shops in Japan and must be as autonomous as possible. However, localization solutions in the literature suffer from the limitations of the platform. This thesis is split into two main themes. First, the problem of relocalization in a visually redundant environment is studied. The proposed solution combines vision and Wi-Fi in a probabilistic approach based on the appearance. Then, the question of a consistent metrical mapping is examined. In order to compensate the numerous losses of tracking caused by the low acquisition frequency, odometric constraints are added to a bundle adjustment optimization. These solutions have been tested and validated on several Pepper robots, from data collected in different indoor environments over more than 7 km.; Après la démocratisation des robots industriels, la tendance actuelle est au développement de robots sociaux dont la fonction principale est l'interaction avec ses utilisateurs. Le déploiement de telles plate-formes dans des boutiques, des musées ou des gares relance différentes problématiques dont celle de la localisation pour les robots mobiles. Cette thèse traite ainsi de la localisation du robot Pepper en milieu intérieur non-contraint. Présent dans de nombreuses boutiques au Japon, Pepper est utilisé par des personnes non-expertes et doit donc être le plus autonome possible. Cependant, les solutions de localisation autonome de la littérature souffrent des limitations de la plate-forme. Les travaux de cette thèse s'articulent autour de deux grands axes. D'abord, le problème de la relocalisation dans un environnement visuellement redondant est étudié. La solution proposée consiste à combiner la vision et le Wi-Fi dans une approche probabiliste basée sur l'apparence. Ensuite, la question de la création d'une carte métrique cohérente est approfondie. Pour compenser les nombreuses pertes de suivi d'amers visuels causées par une fréquence d'acquisition basse, des contraintes odométriques sont ajoutées à une optimisation par ajustement de faisceaux. Ces solutions ont été testées et validées sur plusieurs robots Pepper à partir de données collectées dans différents environnements intérieurs sur plus de 7 km.

After the democratization of industrial robots, the current trend is the development of social robots that create strong interactions with their users. The deployment of such platforms in shops, museums or train stations raises various issues including the autonomous localization of mobile robots. This thesis focuses on the localization of Pepper robots in a non-constrained indoor environment. Pepper robots are daily used in many shops in Japan and must be as autonomous as possible. However, localization solutions in the literature suffer from the limitations of the platform. This thesis is split into two main themes. First, the problem of relocalization in a visually redundant environment is studied. The proposed solution combines vision and Wi-Fi in a probabilistic approach based on the appearance. Then, the question of a consistent metrical mapping is examined. In order to compensate the numerous losses of tracking caused by the low acquisition frequency, odometric constraints are added to a bundle adjustment optimization. These solutions have been tested and validated on several Pepper robots, from data collected in different indoor environments over more than 7 km.; Après la démocratisation des robots industriels, la tendance actuelle est au développement de robots sociaux dont la fonction principale est l'interaction avec ses utilisateurs. Le déploiement de telles plate-formes dans des boutiques, des musées ou des gares relance différentes problématiques dont celle de la localisation pour les robots mobiles. Cette thèse traite ainsi de la localisation du robot Pepper en milieu intérieur non-contraint. Présent dans de nombreuses boutiques au Japon, Pepper est utilisé par des personnes non-expertes et doit donc être le plus autonome possible. Cependant, les solutions de localisation autonome de la littérature souffrent des limitations de la plate-forme. Les travaux de cette thèse s'articulent autour de deux grands axes. D'abord, le problème de la relocalisation dans un environnement visuellement redondant est étudié. La solution proposée consiste à combiner la vision et le Wi-Fi dans une approche probabiliste basée sur l'apparence. Ensuite, la question de la création d'une carte métrique cohérente est approfondie. Pour compenser les nombreuses pertes de suivi d'amers visuels causées par une fréquence d'acquisition basse, des contraintes odométriques sont ajoutées à une optimisation par ajustement de faisceaux. Ces solutions ont été testées et validées sur plusieurs robots Pepper à partir de données collectées dans différents environnements intérieurs sur plus de 7 km.

Nowadays, the elderly population represents 13% of the world's population. It is estimated that this rate will increase very rapidly leading to the phenomenon of the population ageing. This phenomenon increases the incidence of various diseases, such as cardiopulmonary pathologies. In this thesis, we are interested in the pathology known as Acute Respiratory Distress Syndrome (ARDS). ARDS is a fatal lung condition occurring in critically ill patients representing a high mortality rate. The aim of this thesis is to develop methods for the prediction of ARDS in real-time using physiological signals, such as heart rate, respiratory rate, peripheral arterial oxygen saturation and mean arterial blood pressure. During the course of this thesis, we developed three original methods for the prediction of ARDS. The first method is based on the novelty detection on each physiological signal, followed by different techniques of decision fusion. The second method consists of the extraction of parameters from the signals, selecting the most relevant ones and fusing them using the theory of belief functions. Finally, we proposed a model that combines both kernel methods and belief function theory for the prediction of ARDS. We also proposed the extraction of new parameters to complete the list used in the previous method., La population âgée représente actuellement 13% de la population mondiale. Ce taux est estimé d’augmenter très rapidement entrainant le phénomène de vieillissement de la population. Ce phénomène augmente la fréquence d’occurrence de diverses maladies, telles que les pathologies cardiopulmonaires. Dans cette thèse, nous nous intéressons à la pathologie connue sous le nom de syndrome de détresse respiratoire aigüe (SDRA). Il s’agit d’une forme très sévère de défaillance pulmonaire aiguë, consécutive à une altération de la perméabilité capillaire. Le SDRA représente un taux de mortalité élevé. Cette thèse a pour objectif de développer des méthodes de prévention du SDRA en temps réel en utilisant des signaux physiologiques, telles que le rythme cardiaque, le rythme respiratoire, le taux d'oxygène dans le sang et la moyenne de la pression artérielle. Au cours de cette thèse, nous avons développé trois méthodes originales pour la prévention du SDRA. La première méthode est basée sur la détection d’anormalité dans chaque signal physiologique, suivie d’une fusion de décisions, selon différentes approches. La deuxième méthode consiste à extraire des caractéristiques des signaux, à sélectionner les paramètres les plus pertinents et à les fusionner à l’aide de la théorie des fonctions de croyance. Finalement, nous avons proposé un modèle combinant à la fois les méthodes à noyaux et la théorie des fonctions de croyance pour la prévention du SDRA. Nous avons également proposé l’extraction de nouveaux paramètres.

Road traffic accidents have always been a concern to the driving community which has led to various research developments for improving the way we drive the vehicles. Since human error causes most of the road accidents, introducing automation in the vehicle is an efficient way to address this issue thus making the vehicles intelligent. This approach has led to the development of ADAS (Advanced Driver Assistance Systems) functionalities. The process of introducing automation in the vehicle is continuously evolving. Currently the research in this field has targeted full autonomy of the vehicle with the aim to tackle the road safety to its fullest potential. The gap between ADAS and full autonomy is not narrow. One of the approach to bridge this gap is to introduce collaboration between human driver and autonomous system. There have been different methodologies such as haptic feedback, cooperative driving where the autonomous system adapts according to the human driving inputs/intention for the corrective action each having their own limitations. This work addresses the problem of shared control authority between human driver and autonomous driving system without haptic feedback using the fusion of driving inputs. The development of shared control authority is broadly divided into different stages i.e. shared control framework, driving input assessment, driving behavior prediction, fusion process etc. Conflict resolution is the high level strategy introduced in the framework for achieving the fusion. The driving inputs are assessed with respect to different factors such as collision risk, speed limitation,lane/road departure prevention etc in the form of degree of belief in the driving input admissibility using sensor data. The conflict resolution is targeted for a particular time horizon in the future using a sensor based driving input prediction using neural networks. A two player non-cooperative game (incorporating admissibility and driving intention) is defined to represent the conflict resolution as a bargaining problem. The final driving input is computed using the Nash equilibrium. The shared control strategy is validated using a test rig integrated with the software Simulink and IPG CarMaker. Various aspects of shared control strategy such as human-centered, collision avoidance, absence of any driving input, manual driving refinement etc were included in the validation process., Ce travail aborde le problème de l’autorité de contrôle partagée entre les conducteurs et système de conduite autonome sans retour haptique utilisant la fusion des entrées de conduite. Le développement d’une autorité de contrôle partagée est divisé en différentes étapes : cadre de contrôle partagé, évaluation des facteurs de conduite, prévision du comportement de conduite, processus de fusion, etc. La résolution des conflits est la stratégie de haut niveau introduite dans le cadre permettant de réaliser la fusion. Les entrées de conduite sont évaluées en fonction de différents facteurs tels que le risque de collision, la limitation de vitesse, la prévention de voie / départ, etc., sous la forme d’un degré de confiance dans l’admissibilité d’une entrée de conduite à l’aide de données de capteur. La résolution de conflit est ciblée pour un horizon temporel particulier dans le futur en utilisant une prédiction d’entrée de conduite basée sur un capteur utilisant des réseaux de neurones. Un jeu non coopératif à deux joueurs (comprenant l’admissibilité et l’intention de conduite) est défini comme représentant la résolution du conflit comme un problème de négociation. L’entrée motrice finale est calculée en utilisant l’équilibre de Nash. La stratégie de contrôle partagé est validée à l’aide d’un banc d’essai intégré aux logiciels Simulink et IPG CarMaker. Divers aspects de la stratégie de contrôle partagé, tels que l’accent mis sur l’homme, la prévention des collisions, l’absence de toute information sur la conduite, l’affinement de la conduite manuelle, etc., ont été inclus dans le processus de validation.

Urban intersections put high demands on fully automated vehicles, in particular, if occlusion occurs. In order to resolve such and support vehicles in unclear situations, a popular approach is the utilization of additional information from infrastructure-based sensing systems. However, a widespread use of such systems is circumvented by their complexity and thus, high costs. Within this paper, a generic interface is proposed, which enables a huge variety of sensors to be connected. The sensors are only required to measure very few features of the objects, if multiple distributed sensors with different viewing directions are available. Furthermore, a Labeled Multi-Bernoulli (LMB) filter is presented, which can not only handle such measurements, but also infers missing object information about the objects��� extents. The approach is evaluated on simulations and demonstrated on a real-world infrastructure setup., acceptedVersion

Environment perception systems for automated vehicles and driver assistance systems commonly use radar and lidar sensors as well as cameras for tracking other vehicles. While these sensors exhibit different strengths and weaknesses, they share the ability to resolve multiple measurements for a single object: lidar sensors typically provide multiple distance measurements for a vehicle, modern high-resolution radar sensors yield multiple detections, and objects cover a set of pixels in camera images. Such dense data is valuable for obtaining a detailed and precise representation of the environment. From a tracking perspective, however, it poses three major challenges: First, tracking algorithms have to process multiple measurements for one object and thus face the extended object problem. This conflicts with the classical assumption of at most one measurement per cycle and object. Secondly, the increased amount of data further complicates multi-object problems which involve measurement-to-object associations in the presence of clutter, occlusion, and misdetections. Thirdly, the data from several heterogeneous sensors needs to be fused into a consistent estimate in order to benefit from complementary sensor strengths or to either achieve increased sensor fields of view or redundancy. This thesis presents a tracking framework that is based on finite set statistics (FISST) and tackles these challenges for the application of vehicle tracking in a Bayesian fashion. The employed multi-object filter uses labeled multi-Bernoulli distributions and allows for combining different sensor modules with sensor-specific update routines through centralized fusion. Furthermore, extended object measurement models which work on the full raw sensor data are developed. In particular, it is demonstrated how separable likelihood approaches can be used to incorporate accurate vehicle models for lidar data and semantically labeled camera images. Additionally, two radar vehicle models are developed. They are able to process multiple radar detections and allow for tracking vehicles in arbitrary maneuvers, including turning vehicles or cross-traffic. While the first radar model is based on expert knowledge, the second uses variational Gaussian mixtures and is learned from actual measurements. The methods are evaluated on experimental data with accurate ground truth. The results demonstrate that the extended object models achieve precise estimates, that the data-driven variational radar model is able to outperform the manually designed model, and how different sensor combinations improve the performance., Umgebungserfassungssysteme für automatisierte Fahrzeuge oder Fahrerassistenzsysteme verwenden üblicherweise Radar- oder Lidarsensoren sowie Kameras, um andere Fahrzeuge zu erfassen und zu verfolgen. Während diese Sensoren verschiedene Vor- und Nachteile aufweisen, so teilen sie die Fähigkeit, mehrere Messungen für ein einzelnes Objekt auflösen zu können: Lidarsensoren liefern üblicherweise mehrere Distanzmessungen für ein Fahrzeug, moderne hochauflösende Radarsensoren erzeugen mehrere Detektionen und Objekte erstrecken sich über mehrere Pixel eines Kamerabilds. Diese Datendichte ist wertvoll, um ein detailliertes und präzises Abbild der Umgebung zu erhalten. Aus Objektverfolgungssicht ergeben sich daraus jedoch drei Herausforderungen: Erstens muss die Objektverfolgung mehrere Messungen für ein einzelnes Objekt verarbeiten und steht daher vor dem Problem, mit sogenannten ausgedehnten Objekten umgehen zu müssen. Dies steht im Widerspruch zur klassischen Annahme von maximal einer Messung pro Objekt und Messzyklus. Zweitens erschwert die zusätzliche Datenmenge Multiobjektproblemstellungen, welche Assoziationen zwischen Messungen und Objekten unter Berücksichtigung von Falschmessungen, Sensorrauschen, Verdeckungen und Fehldetektionen erforderlich machen. Drittens müssen Messungen von verschiedenen, heterogenen Sensoren zu einer einheitlichen Schätzung fusioniert werden, um von sich ergänzenden Sensorstärken zur profitieren oder um vergrößerte Sensorsichtbereiche oder Redundanz zu erreichen. Diese Arbeit stellt ein Objektverfolgungssystem vor, welches auf der Methode der Finite-Set-Statistics (FISST) beruht und diese drei Herausforderungen für die Anwendung der Fahrzeugverfolgung auf Bayes’sche Weise löst. Der eingesetzte Multiobjektfilter verwendet Labeled-Multi-Bernoulli-(LMB)-Verteilungen und ermöglicht die Kombination verschiedener Sensormodule mit sensorspezifischen Update-Routinen durch eine zentrale Fusionsarchitektur. Zusätzlich werden Messmodelle für ausgedehnte Objekte entwickelt, die direkt auf den Sensorrohdaten arbeiten. Im Besonderen wird gezeigt, wie separierbare Wahrscheinlichkeitsfunktionen es ermöglichen, präzise Fahrzeugmodelle für Laserdaten sowie semantisch klassifizierte Kamerabilder zu integrieren. Außerdem werden zwei Radarmodelle entwickelt, die mehrere Radardetektionen verarbeiten können und das Verfolgen von Fahrzeugen in beliebigen Manövern erlauben, einschließlich abbiegender Fahrzeuge und Querverkehr. Während das erste Radarmessmodell auf Expertenwissen basiert, wird das zweite mithilfe variationeller Gaußmixturen auf Basis realer Messungen gelernt. Die Ansätze werden auf experimentellen Sensordaten, für welche akkurate Referenzwerte zur Verfügung stehen, ausgewertet. Die Ergebnisse zeigen, dass die Messmodelle für ausgedehnte Objekte präzise Schätzergebnisse erreichen, dass das datengestützte variationelle Radarmodell bessere Ergebnisse erreicht als das händisch entwickelte Modell und wie die Kombination verschiedener Sensoren die Leistungsfähigkeit erhöht.

Localization is the task of estimating the pose of a vehicle. In the present thesis, this is done using a feature-based map of the environment and different kinds of sensors that are able to detect those environmental features or landmarks. After introducing the notation and mathematical foundation, the random-set Monte-Carlo Localization (MCL) is derived. Landmarks from a map and their respective measurements are modeled as Random Finite Sets (RFSs) with an unknown association between each other. This concept evolved in literature over the past years and is continued in the present work. The main difference of random-set MCL compared to vector-based MCL is the calculation of the likelihood between map landmarks and sensor measurements. By incorporating the multi-object likelihood, which is known from RFS-based tracking algorithms, not only missed detections but also clutter measurements are considered in a mathematically rigorous manner. Hence, random-set MCL is a very robust localization approach as also shown in the evaluation using challenging real-world scenarios. Several assumptions of the multi-object measurement model are typically made: the spatial distribution of landmark measurements is Gaussian, the detection probability follows a Bernoulli distribution, clutter is distributed uniformly in the Field of View (FOV) and its rate follows a Poisson distribution. Those assumptions are also very common for multi-object tracking algorithms that are based on RFSs. However, they have to be validated in order to provide a reliable localization result where not only the position but also its uncertainty estimate is of relevance. Hence, this thesis introduces a new methodology to online estimate and validate the multi-object measurement model. A deviation between assumed and estimated parameters is detected in a stochastic manner by using confidence intervals. By utilizing this redundant information, it is possible to online validate the localization result even without having a reference system. Knowing about the reliability and consistency of a localization estimate is a huge benefit especially for highly automated driving. The localization of a vehicle in a dynamic environment does not only require a theoretic foundation but also a practical implementation. Consequently, methods to extract meaningful landmarks from cameras, laser scanners and radar sensors are also part of this thesis. The extraction process especially focuses on landmarks that are part of the infrastructure and are therefore quite stable over a long time span. In order to localize a vehicle, a map of the environment is necessary. Thus, the second part of this thesis focuses on random-set mapping. First, state-of-the-art techniques are introduced, including RFS-based Simultaneous Localization and Mapping (SLAM) algorithms. Next, a new concept for the feature-based long-term mapping is derived which is able to fuse chronologically ordered transient short-term maps by making use of the Markov assumption. However, at some point in time a map will be outdated. Therefore, the concept of long-term mapping is extended to a continuous mapping approach. The idea is to keep a feature map up-to-date by continuously fusing new information from different vehicles into a common long-term map. Each vehicle thus performs an RFS-based online SLAM by additionally incorporating a probably outdated or sparse map as a prior information. This prior helps circumvent the inevitable error propagation of SLAM. An evaluation of the localization, its consistency as well as the long-term and continuous mapping is provided using real-world data from two test vehicles in challenging scenarios. Those test vehicles were set up at the Institute of Measurement, Control and Microtechnology at Ulm University. They are able to automatically drive in an urban and rural environment by localizing itself using the presented algorithms.

This thesis deals with the problem of localization of dependent elderly people using wireless sensor networks. Each person is equipped with a bracelet or a medallion that consists of a sensor capable of measuring the WiFi signals and communicating with the environment. We tackle the problem of localization by zoning, where the objective is to determine the zone where the person resides, instead of its exact position. It is formulated as a problem of multi-class classification, which we treat by associating the flexibility of statistical learning methods to the theory of Dempster-Shafer for fusion of information with uncertainty. The interest of this theory is in its ability to model at the same time the uncertainty and ambiguity of data, and the reliability and conflict of the sources. First, we propose an observation model exploiting the power of the exchanged signals between the sensors with the fingerprinting technique. Afterwards, we extend the proposed model, through hierarchical clustering, in order to cover larger surface areas with higher number of zones. We then develop a method for selecting the WiFi terminals in favor. This method helps enhancing the overall accuracy and reducing the complexity of the localization method, by selecting the best subset in terms of discriminative capacity and redundancy. We also propose mobility models, which we combine with the observation model, in order to correct the estimations by aggregating all available evidence. In addition, we present a decentralized approach of the localization method, to increase the robustness and reduce the complexity. The performance of the proposed methods is validated through experiments on real data, and evaluated in comparison with other well-known methods in the domain.; Cette thèse porte sur le problème de localisation des personnes âgées dépendantes à l'aide de réseaux de capteurs sans fil. Chaque personne est équipée d'un bracelet ou médaillon, comprenant un capteur intelligent capable de mesurer les signaux WiFi et de communiquer avec l'environnement. Nous abordons le problème de localisation par zonage, où le but est de déterminer la zone où se trouve la personne, au lieu de sa position exacte. Il s'agit d'un problème de classification multi-classes, que nous traitons en associant la flexibilité des méthodes d'apprentissage statistique à la théorie de Dempster-Shafer pour la fusion de l'information avec incertitude. L'intérêt de l'utilisation de cette théorie réside dans sa capacité à modéliser à la fois l'incertitude et l'ambiguïté des données, ainsi que la fiabilité et le conflit des sources. Nous proposons en premier un modèle d'observation exploitant la puissance des signaux échangés entre les capteurs avec la technique de fingerprinting. Nous développons ensuite le modèle proposé, au travers du regroupement hiérarchique, dans le but de couvrir des surfaces plus grandes avec un nombre de zones plus élevé. Nous élaborons ensuite une méthode de sélection de paramètres, qui sont les bornes WiFi à privilégier. Cette méthode permet d'améliorer la précision et de réduire la complexité de la méthode de localisation, en choisissant le meilleur ensemble de paramètres en termes de la capacité de discrimination et de la redondance. Nous proposons également des modèles de mobilité que nous combinons au modèle d'observation, afin de corriger les estimations en fusionnant toutes les preuves disponibles. D'autre part, nous présentons une approche décentralisée de la méthode de localisation, pour accroître la robustesse et diminuer la complexité. Les performances des méthodes proposées sont validées par des expérimentations sur des données réelles, et évaluées en comparaison avec d'autres méthodes très connues dans le domaine.

This thesis deals with the problem of localization of dependent elderly people using wireless sensor networks. Each person is equipped with a bracelet or a medallion that consists of a sensor capable of measuring the WiFi signals and communicating with the environment. We tackle the problem of localization by zoning, where the objective is to determine the zone where the person resides, instead of its exact position. It is formulated as a problem of multi-class classification, which we treat by associating the flexibility of statistical learning methods to the theory of Dempster-Shafer for fusion of information with uncertainty. The interest of this theory is in its ability to model at the same time the uncertainty and ambiguity of data, and the reliability and conflict of the sources. First, we propose an observation model exploiting the power of the exchanged signals between the sensors with the fingerprinting technique. Afterwards, we extend the proposed model, through hierarchical clustering, in order to cover larger surface areas with higher number of zones. We then develop a method for selecting the WiFi terminals in favor. This method helps enhancing the overall accuracy and reducing the complexity of the localization method, by selecting the best subset in terms of discriminative capacity and redundancy. We also propose mobility models, which we combine with the observation model, in order to correct the estimations by aggregating all available evidence. In addition, we present a decentralized approach of the localization method, to increase the robustness and reduce the complexity. The performance of the proposed methods is validated through experiments on real data, and evaluated in comparison with other well-known methods in the domain.; Cette thèse porte sur le problème de localisation des personnes âgées dépendantes à l'aide de réseaux de capteurs sans fil. Chaque personne est équipée d'un bracelet ou médaillon, comprenant un capteur intelligent capable de mesurer les signaux WiFi et de communiquer avec l'environnement. Nous abordons le problème de localisation par zonage, où le but est de déterminer la zone où se trouve la personne, au lieu de sa position exacte. Il s'agit d'un problème de classification multi-classes, que nous traitons en associant la flexibilité des méthodes d'apprentissage statistique à la théorie de Dempster-Shafer pour la fusion de l'information avec incertitude. L'intérêt de l'utilisation de cette théorie réside dans sa capacité à modéliser à la fois l'incertitude et l'ambiguïté des données, ainsi que la fiabilité et le conflit des sources. Nous proposons en premier un modèle d'observation exploitant la puissance des signaux échangés entre les capteurs avec la technique de fingerprinting. Nous développons ensuite le modèle proposé, au travers du regroupement hiérarchique, dans le but de couvrir des surfaces plus grandes avec un nombre de zones plus élevé. Nous élaborons ensuite une méthode de sélection de paramètres, qui sont les bornes WiFi à privilégier. Cette méthode permet d'améliorer la précision et de réduire la complexité de la méthode de localisation, en choisissant le meilleur ensemble de paramètres en termes de la capacité de discrimination et de la redondance. Nous proposons également des modèles de mobilité que nous combinons au modèle d'observation, afin de corriger les estimations en fusionnant toutes les preuves disponibles. D'autre part, nous présentons une approche décentralisée de la méthode de localisation, pour accroître la robustesse et diminuer la complexité. Les performances des méthodes proposées sont validées par des expérimentations sur des données réelles, et évaluées en comparaison avec d'autres méthodes très connues dans le domaine.

A pulse-width multiplexing method for reducing the nuisance-alarm rate of a phase-sensitive optical time-domain reflectometer ( ϕ -OTDR) is described. In this method, light pulses of different pulse-widths are injected into the sensing fiber, the data acquired at different pulse-widths are regarded as the outputs of different sensors, and these data are then processed by a multisensor data fusion algorithm. In laboratory tests with a sensing fiber on a vibrating table, the effects of pulse-width on the signal-to-noise ratio (SNR) of the ϕ -OTDR data are observed. Furthermore, by utilizing the SNR as the feature in a feature-layer algorithm based on Dempster&ndash, Shafer evidential theory, a four-pulse-width multiplexing ϕ -OTDR system is constructed, and the nuisance-alarm rate is reduced by about 70%. These experimental results show that the proposed method has great potential for perimeter protection, since the nuisance-alarm rate is significantly reduced by using a simple configuration.

A method for automatic determination of sensors positions and orientations in multisensor laser triangulation coordinate measuring systems is presented. Based on a series of measurements on a conical artifact without the use of external light sources, the approach allows for both optimized adjustment of the physical sensors alignment and improved 3D data registration by software. Simulations were conducted to quantify the sensitivity of the method. Experimental results based on measurements of steep details of complex shaped parts show a 5-times reduction of deviations between measurements of sensors with respect to an already optimized adjustment by manual methods.

Multitarget tracking algorithms based on sonar usually run into detection uncertainty, complex channel and more clutters, which cause lower detection probability, single sonar sensors failing to measure when the target is in an acoustic shadow zone, and computational bottlenecks. This paper proposes a novel tracking algorithm based on multisensor data fusion to solve the above problems. Firstly, under more clutters and lower detection probability condition, a Gaussian Mixture Probability Hypothesis Density (GMPHD) filter with computational advantages was used to get local estimations. Secondly, this paper provided a maximum-detection capability multitarget track fusion algorithm to deal with the problems caused by low detection probability and the target being in acoustic shadow zones. Lastly, a novel feedback algorithm was proposed to improve the GMPHD filter tracking performance, which fed the global estimations as a random finite set (RFS). In the end, the statistical characteristics of OSPA were used as evaluation criteria in Monte Carlo simulations, which showed this algorithm&rsquo, s performance against those sonar tracking problems. When the detection probability is 0.7, compared with the GMPHD filter, the OSPA mean of two sensor and three sensor fusion was decrease almost by 40% and 55%, respectively. Moreover, this algorithm successfully tracks targets in acoustic shadow zones.

Algunas aplicaciones de carretera actuales, tales como los servicios de información al viajero, llamadas de emergencia automáticas, control de flotas o telepeaje eletrónico, requieren una solución de calidad al problema del posicionamiento de un vehículo terrestre, que funcione en cualquier entorno y a un coste razonable. Esta tesis presenta una solución a este problema, fusionando para ello la información procedente principalmente de sensores de navegación por satélite y sensores inerciales. Para ello emplea un nuevo filtro de fusion multisensorial IMM-EKF. El comportamiento del sistema ha sido analizado en entornos reales y controlados, y comparado con otras soluciones propuestas. Finalmente, su aplicabilidad al problema planteado ha sido verificada., Road applications such as traveller information, automatic emergency calls, freight management or electronic fee, collection require a onboard equipment (OBE) capable to offer a high available accurate position, even in unfriendly environments with low satellite visibility at low cost. Specifically in life critical applications, users demand from the OBEs accurate continuous positioning and information of the reliability of this position. This thesis presents a solution based on the fusion of Global Navigation Satellite Systems (GNSS) and inertial sensors (GNSS/INS), running an Extended Kalman Filter combined with an Interactive Multi-Model method (IMM-EKF). The solution developed in this work supplies continuous positioning in marketable conditions, and a meaningful trust level of the given solution. A set of tests performed in controlled and real scenarios proves the suitability of the proposed IMM-EKF implementation, as compared with low cost GNSS based solutions, dead reckoning systems and single model extended Kalman filter (SM-EKF) solutions.

Web applications are typically developed with hard time constraints and are often deployed with security vulnerabilities. Automatic web vulnerability scanners can help to locate these vulnerabilities and are popular tools among developers of web applications. Their purpose is to stress the application from the attackers point of view by issuing a huge amount of interaction within it. Two of the most widely spread and dangerous vulnerabilities in web applications are SQL injection and cross site scripting XSS , because of the damage they may cause to the victim business. The most common types of software faults are injected in the web application code which is then checked by the scanners. The results are compared by analyzing coverage of vulnerability detection and false positives. Added to this it checks whether the domain or URL gives is present in any of the domain blacklist sites. It also provides I Category the category classification for the given domain. So this gives the analyst an advantage of having the Domain rating for the given domain with a lookup on top blacklist providing sites and also the Category Classification for the given domain. Three leading commercial scanning tools are evaluated and the results show that in general the coverage is low and the percentage of false positives is very high. Sarumathy. B | Indra. E "Data Injection using Kalman-Like Particle Filter Based Smoother is Diagonalized into Subsystem" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-2 , February 2018, URL: https://www.ijtsrd.com/papers/ijtsrd9608.pdf

Sensor data fusion is the key to a comprehensive environment perception by today’s and future systems basing on object tracking. Early and rather simple advanced driver assistance systems (ADAS) are still using a sensor setup with a single sensor where the problem of fusing different data sources does not arise. With the intent on fusing data from multiple heterogeneous sensors into one common object tracking system, various fusion methods are conceivable. Probabilistic data association (PDA) is one of these methods and it has been shown to be feasible and effective in former publications [Mäh09; Mun11]. In very complex scenarios, algorithms based on finite set statistics [Mah03] have become popular over the last few years. These allow the modeling of interactions between the tracked objects to resolve ambiguities comparable to the multi-object Bayes filter. Independent of the fusion method, all procedures share one major drawback: the interchangeability of the sensors is not possible. In many systems it is necessary to transmit knowledge about the sensors to the fusion system. Thus, a change of the sensor setup can entail comprehensive consequences and can be expensive and costly in development time. Besides increasing the sensor interchangeability, the anonymization of the sensor is desirable. This allows sensors to be used without knowledge of the sensors’ theoretical principles and enables the sensor manufacturer to maintain secrecy about the details. The generic linkage of sensors to object tracking systems as well as the anonymization of sensors is the purpose of this work. Therefore, a mathematically equivalent alternative to the Kalman filter, the information filter, is used. The focus is set on probabilistic data association and the successful adaption of it to use the information filter is evaluated in simulation and real-data scenarios. Additionally, it is shown how to use the information filter approach in many other fusion systems. In the further course of this work a novel approach to preprocessing high density data from distance measuring sensors is presented. This new approach meets the requirements on generically linked sensors. It allows the use of the information space and simultaneously increases the perception performance markedly in comparison to former attempts. This is achieved by filtering the raw sensor data over time and generating reliable object hypotheses using the filtered data. The performance of the achieved sensor model is demonstrated in various real-data scenarios., Sensordatenfusion ist der Schlüssel zu heutigen und zukünftigen Systemen, die auf einer Objektverfolgung zur umfassenden Umgebungserfassung beruhen. Frühe und einfache Systeme basieren noch heute auf lediglich einem Sensor, wodurch sich das Problem der Fusion verschiedener Datenquellen nicht ergibt. Sollen jedoch Daten mehrerer heterogener Sensoren in eine gemeinsame Objektverfolgung eingebracht werden, kommen diverse Fusionsmethoden in Frage. Die probabilistische Datenassoziation (PDA) ist eine dieser Methoden und hat sich in Untersuchungen früherer Publikationen [Mäh09; Mun11] als einfach und wirkungsvoll erwiesen. In sehr komplexen Szenarien kommen in den letzten Jahren auch vermehrt Methoden der Statistik endlicher Mengen (engl. finite set statistics (FISST)) [Mah03] zum Einsatz. Diese erlauben zusätzlich die Modellierung von Abhängigkeiten zwischen den Objekten, wodurch Mehrdeutigkeiten aufgelöst werden können. Unabhängig von der Fusionsmethode teilen sich alle Verfahren jedoch ein Problem: die Austauschbarkeit der Sensorik ist nicht vollständig gewährleistet. Es muss stets Wissen über die verwendeten Sensoren in das System eingebracht werden. Eine Veränderung des Sensoraufbaus kann weitreichende Konsequenzen nach sich ziehen und verursacht sowohl Zeit- als auch Kostenaufwand. Des Weiteren ist neben dem Vereinfachen des Sensoraustauschs auch eine Anonymisierung des Sensors wünschenswert. Dies würde es erlauben, Sensoren ohne Wissen über ihr Messprinzip zu verwenden. Auf der anderen Seite müsste der Hersteller dann keine detaillierten Informationen über den Sensor preisgeben. Die generische Anbindung von Sensoren an fusionierende Systeme zur Objektverfolgung sowie die Anonymisierung der Sensoren sind Ziel dieser Arbeit. Um dies zu realisieren, wird der Informationsfilter als Alternative zum mathematisch äquivalenten Kalman Filter verwendet. Besonderes Augenmerk wird dabei auf die PDA gelegt. Deren erfolgreiche Anpassung wird anhand von Simulation und realer Daten ausgewertet. Darüber hinaus wird gezeigt, wie der Informationsfilter-Ansatz auch in vielen anderen Fusionsmethoden eingesetzt werden kann. Im weiteren Verlauf dieser Arbeit wird ein neuartiger Ansatz zur Vorverarbeitung für hochauflösende und Distanz messende Sensoren präsentiert, die den Anforderungen an generisch angebundenen Sensoren entspricht. Dies ermöglicht den Einsatz des Informationsraumes und verbssert gleichzeitig die Detektionsleistung deutlich gegenüber bisherigen Ansätzen. Dabei werden direkt die Sensorrohdaten zeitlich verfolgt, um zuverlässige Hypothesen erzeugen zu können. Die Leistungsfähigkeit des neuen Ansatzes wird in verschiedenen realen Szenarien nachgewiesen.

Image fusion deals with the ability to integrate data from image sensors at different instants when the source information is uncertain. Although there exist many techniques on the subject, in this paper, we develop two originative techniques based on principal component analysis and slicing image transformation to efficiently fuse a small set of noisy images. For instance, in neural data fusion, this approach requires a considerable number of corrupted images to efficiently produce the desired outcome and also requiring a considerable computing time because of the dynamics involved in the fusion data process. In our approaches, the computation time is considerably smaller. This results appealing to increasing feasibility, for instance, in remote sensing or wireless sensor network. Moreover, and according to our numerical experiments, when our methods are compared against the neural data fusion algorithm, they present better performance.

Thesis (PhD(Computer and Information Science))--University of South Australia, 2018. Includes bibliographical references (pages [235]-257) This dissertation introduces ‘tangible spatiotemporal visualisation’ as a comprehensive tool of big data visualisation, which improves user understanding and control through a display included with tangible user interfaces (TUIs). National security in the dissertation is the research topic that is the focus of Data to Decisions Cooperative Research Centre (D2D CRC), a company that supports research by providing funds. As the research scenario, there is a new type of surveillance environment using a simulated closed-circuit television (CCTV) system and maritime ship movement visualisation using multiple datasets, such as ship trajectory data from global positioning systems (GPS) and light data from satellite images. Further, the visualisations are expressed using a tabletop screen, augmented reality (AR)/hologram digital information, and the TUIs. The TUIs in the system help users better understand and control the visualised data as a part of visualisation.

Yer belirleme, hareket eden insan/nesne konumu için belirli koordinatları belirtir. En yaygın yer belirleme sistemi örneği, bir mobil Küresel Konumlandırma Sistemi (GPS) alıcısı ve GPS uyduları kullanmaktadır. GPS konumu, doğruluğu 5-20 m’ye kadar belirlenebilmekte ve GPS uyduları sayesinde tüm dünyada erişilebilmektedir. Ne yazık ki, iç mekan yer tespiti uygulamaları için uygun değildir. İç mekan yer tespiti sistemleri farklı türden yer tespiti uygulamaları gerektirmektedir. Çünkü, GPS sinyalleri kapalı alanlarda sönümlenmeye uğramakta ve engellerden etkilenmektedir. Doğruluk oranı daha yüksek iç mekan lokalizasyon sistemleri, alışveriş merkezleri, fabrikalar ve havalimanları gibi mekanlarda reklam, bina navigasyonu gibi çeşitli uygulamaların daha verimli geliştirilmesini sağlayabilir. Günümüzde iç mekan yer belirleme sistemleri için Wi-Fi, Bluetooth, Görünür Işık İletişimi (VLC) gibi çeşitli teknikler kullanılabilmektedir.Bu tez çalışması, 3 boyutlu Kartezyen Koordinatlarda 1-10 cm arası doğruluk oranına sahip Techilus iç mekan yer tespiti sisteminin tasarımı ve uygulanmasını tanıtmaktadır. Techilus Ultrasonik ile Kapalı Alan Yer Tespiti Teknolojisinin kısaltmasıdır. Techilus sistemi Ultrasonik ve WiFi teknolojilerinn hibrit bir yöntem olarak kullanıldığı yeni bir kapalı alan yer tespiti uygulamasını temsil eder. Techilus sistemi donanım ve mobil uygulama kısımlarından oluşmaktadır. Sistemin donanım kısmı verici olarak, ve yaygın olarak kullanılan WiFi destekli akıllı telefonlar alıcı olarak konumlandırılmıştır. Donanım bölümünde, 3 boyutta yer tespiti yapabilmek için dört ünite sistemi tasarlanmıştır. Bu ünite sistemi, ultrasonik ve WiFi bilenşelerine sahip bir ana kontrol ünitesi ve ultrasonik vericilere sahip üç adet sinyal üretim ünitesinden oluşmaktadır. Sinyal üretim birimleri ve ana ünite, maksimum 150Vrms voltaj ile 20 KHz frekansında sinyal üretir. MATLAB ve Android platformlarında sinyal algılama, WiFi zaman senkronizasyonu ve 3 boyutta yer tespiti ölçümleri için çeşitli algoritmalar geliştrilmiştir.--------------------Localization refers to determine specific coordinates for traveler location. The most common localization system example is using a mobile Global Positioning System (GPS) receiver and GPS satellites. . GPS signal accuracy is 5-20m and the range is available in worldwide. Unfortunately, it is not appropriate for indoor localization systems. Indoor localization systems require different kind of localization systems because GPS signals cannot receive in indoor areas due to the signal dampening and obstructions. More accurate indoor localization systems can improve the efficiency of various applications like advertising, building navigation in places such as malls, factories and airports. Nowadays, indoor positioning systems include various techniques such as GPS, Wi-Fi, Bluetooth, Visible Light Communication (VLC) for indoor positioning. This thesis describes the design and implementation of the Techilus indoor localization system, which is an abbreviation of Technology for Indoor Localization with Ultrasound, that provides 1-10cm accuracy in 3D Cartesian Coordinates. Techilus system defines a new way of the indoor localization applications which uses Ultrasound and WiFi technologies as a hybrid method. Techilus system consists of hardware and mobile application parts. The hardware part of the system positioned as a transmitter and widely used smartphones with WiFi support are positioned as receivers. In the hardware part, a four units system has been designed to measure the location in 3D. It consists of a main control unit with ultrasound and WiFi components, and three signal generation units with ultrasonic transmitters. Signal generation units and main unit produce 20 KHz ultrasound with a maximum voltage of 150Vrms. Several algorithms developed for signal detection, WiFi time synchronization, and 3D measurement, in MATLAB and Android Platform.