Search

Highly accurate reference vehicle trajectories are required in the automotive domain e. g. for testing mobile GNSS devices. Common methods used to determine reference trajectories are based on the same working principles as the device under test and suffer from the same underlying error problems. In this paper, a new method to generate reference vehicle trajectories in real-world situations using simultaneously acquired aerial imagery from a helicopter is presented. This method requires independent height information which is coming from a LIDAR DTM and the relative height of the GNSS device. The reference trajectory is then derived by forward intersection of the vehicle position in each image with the DTM. In this context, the influence of all relevant error sources were analysed, like the error from the LIDAR DTM, from the sensor latency, from the semi-automatic matching of the vehicle marking, and from the image orientation. Results show that the presented method provides a tool for creating reference trajectories that is independent of the GNSS reception at the vehicle. Moreover, it can be demonstrated that the proposed method reaches an accuracy level of 10 cm, which is defined as necessary for certification and validation of automotive GNSS devices.

In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method AerialMPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of AerialMPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the L1 and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Tracking datasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, L1 yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research.

The EU-funded ECOPOTENTIAL project shall demonstrate, among others, the application potential of satellite images for detailed ecological studies of environmental and ecological protected areas within pre-defined bio-geographical regions of Europe. These studies aim at the corroboration of essential variables for the monitoring of protected areas in Europe. The innovative character of our project is a systematic assessment of how to exploit SAR satellite images in combination with multispectral optical satellite images and/or local in-situ measurements. The results of the project shall demonstrate the scientific gain when combining SAR image data with optical instrument data. While standard products of Sentinel-2 and WorldView-2 provide a sound basis for multispectral analysis and interpretation of vegetated areas, any additional information contained in SAR images (of TerraSAR-X and Sentinel-1) can improve the classification results and the analysis of time series data. We estimate that about 10 essential protected area variables (out of 93 envisaged variables) can be extracted from these images.

The volume of civil high resolution Earth Observation (EO) images has steeply increased during the past decade due to numerous advances in airborne and spaceborne imaging technologies and has already leveraged a number of new applications. On the other hand, the large quantity of available images has extremely increased the challenge of exploring and understanding the full content of the images (i.e., their semantics). Therefore, the development of new image mining systems providing satisfactory results with reasonable computational effort became highly demanded. The existing EO image mining systems are usually based on extracted image features provided by various feature descriptors which can represent either pixel level patterns or the higher level semantics of images. Thus, developing feature descriptors which are able to represent the content of images relevant to the users' requirements helps to improve the accuracy and efficiency of image mining systems. As a consequence, this dissertation introduces new approaches based on Latent Dirichlet Allocation (LDA), a topic model for low and high level image feature descriptions. Moreover, the dissertation proposes novel methods based on LDA and information theory for evaluating various image feature descriptors independent of their application case. Since users usually evaluate image mining results based on their semantics, we conducted user studies for assessing the issues such as the sensory and the semantic gaps which affect the user acceptance of the results. Furthermore, this dissertation shows the importance of prior knowledge about the semantic structure of images in shortening the semantic gap between users and computers. All corresponding experiments are conducted on multispectral and SAR (airborne and space-borne) images; the results are validated by employing standard classification and clustering methods (e.g., SVM and k-means) in order to be comparable to previously obtained results in our discipline. The results demonstrate that by using higher level feature descriptors, the user acceptance of image mining results increases because the images are described by their semantic content. Furthermore, the results show that an evaluation of the feature descriptors regardless of their application allows us to generalize the evaluation outcomes to various applications. In addition, our studies and experiments indicate that the sensory and the semantic gaps should not be overlooked due to their high impact upon the user acceptance of image mining results. Finally, our analyses show that exploring the space of image features leverages an understanding of the image semantics. Die Menge an hochaufgelösten zivilen Erdbeobachtungsbildern hat sich in den letzten 10 Jahren aufgrund zahlreicher Fortschritte bei den Bildaufnahmetechnologien in der Luft- und Raumfahrt stark erhöht und hat bereits zu einer Reihe von neuen Anwendungen geführt. Andererseits hat die große Menge an verfügbaren Bildern auch die Herausforderungen beim Durchsuchen und Verstehen ihres gesamten Inhalts (d.h. ihrer Semantik) extrem gesteigert. Daraus entstand der Bedarf nach neu zu entwickelnden Image-Mining-Systemen mit zufriedenstellender Güte und vertretbaren Rechenzeiten. Die momentan vorhandenen Image-Mining-Systeme der Erdbeobachtung basieren in der Regel auf extrahierten Bildmerkmalen, die durch verschiedene Merkmalsdeskriptoren erzeugt werden, die entweder pixelbezogene Muster oder die Semantik von Bildern auf höherer Ebene darstellen. Daher hilft die Entwicklung von neuen Merkmalsdeskriptoren, die den Inhalt von Bildern, wie von den Nutzern erwartet, beschreiben können bei der Verbesserung der Genauigkeit und der Effizienz von Image-Mining-Systemen. Daher stellt diese Dissertation neue Ansätze mit Latent Dirichlet Allocation (LDA) vor, einer Themenmodellierung für Merkmalsdeskriptoren auf niedriger und hoher Ebene. Weiterhin werden in der Dissertation neuartige auf LDA und auf der Informationstheorie basierende Methoden vorgeschlagen, um verschiedene Bildmerkmalsdeskriptoren unabhängig vom jeweiligen Anwendungsfall zu beurteilen. Da Nutzer ihre erhaltenen Image-Mining-Ergebnisse normalerweise aufgrund ihrer Semantik beurteilen, haben wir Nutzerstudien zur Beurteilung von Fragen wie der sensorischen oder semantischen Lücke durchgeführt, die die Nutzerakzeptanz der Ergebnisse beeinflussen. Weiterhin zeigt diese Dissertation, wie wichtig Vorwissen über die semantische Struktur von Bildern ist, um die semantische Lücke zwischen Nutzern und Rechnern zu verkleinern. Alle zugehörigen Experimente wurden mithilfe von multispektralen und SAR-Bildern aus der Luft- und Raumfahrt durchgeführt; die Ergebnisse wurden mit Standardverfahren zur Klassifizierung und zum Clustering (z.B. mit SVM und k-Means) validiert, um kompatibel mit bereits früher erhaltenen Resultaten in unserem Fachgebiet zu sein. Die Ergebnisse demonstrieren, dass die Nutzerakzeptanz von Image-Mining-Resultaten durch die Verwendung von höheren Merkmalsdeskriptoren steigt, da die Bilder dann durch ihren semantischen Inhalt beschrieben werden. Weiterhin zeigen die Ergebnisse, dass eine vom Anwendungsfall unabhängige Bewertung der Merkmalsdeskriptoren es erlaubt, die Bewertungsergebnisse für unterschiedliche Anwendungsfälle zu verallgemeinern. Darüber hinaus weisen unsere Untersuchungen und Experimente nach, dass die sensorische als auch die semantische Lücke wegen ihrer hohen Auswirkungen auf die Nutzerakzeptanz von Image-Mining-Resultaten nicht übersehen werden sollten. Schließlich zeigen unsere Untersuchungen auch, dass eine Untersuchung des Bildmerkmalsraums das Verstehen der Bildsemantik unterstützt.

This Show & Tell session will begin by introducing the sensory and semantic gap. The sensory gap will then be the focus, and its causes (e.g., resolution, perspective, scale, field of view) will be highlighted with a visual demonstration. This will be followed by a quick experiment involving the audience, and results will be processed in real time, with the purpose of visually quantifying and demonstrating the sensory gap. Concluding the Show & Tell session will be a section concentrating on why the sensory gap is important to consider for practical applications, such as annotation tools, and image learning and mining systems.

The increasing amount of high resolution Earth Observation (EO) data during recent years, has brought the content analysis of the provided data into the spotlight. Most of the current content analysis is based on unsupervised methods (e.g., clustering). However, the structure discovered by these methods is not necessarily human understandable. Moreover, they require some prior knowledge of the structure of the data for initialization. In this paper, we propose an interactive method to discover the semantic structure behind SAR image collections. Thus, we use a modified version of k-means, namely weight-balanced k-means, to perform clustering on the given images. The interaction mechanism allows users to provide the clustering method with relevant knowledge about the structure of the data. Experimental results demonstrate that the structure discovered by the proposed interactive method is closer to human understanding of the data.

Large volume of detailed features of land covers, provided by High-Resolution Earth Observation (EO) images, has attracted the interests to assess the discovery of these features by Content-Based Image Retrieval systems. In this paper, we perform Latent Dirichlet Allocation (LDA) on the Bag-of-Words (BoW) representation of collections of EO images to discover their high-level features, so-called topics. To assess the discovered topics, the images are represented based on the occurrence of different topics, we name it Bag-of-Topics (BoT). Then, the BoT model is compared to the BoW model of images based on the given human-annotations of the data. In our experiments, we compare the classification accuracy resulted by BoT and BoW representations of two different EO datasets, a Synthetic Aperture Radar (SAR) dataset and a multi-spectral satellite dataset. Moreover, we provide isualizations of feature space for better perceiving the changes in the discovered information by BoT and BoW models. Experimental results demonstrate that the dimensionality of the data can be reduced by BoT representation of images; while it either causes no significant reduction in the classification accuracy or even increase the accuracy by sufficient number of topics.

The amount of collected Earth Observation (EO) data is increasing intensively in order of several Terabytes a day. Consequently, new trends to explore and retrieve information from the available data is highly demanded. Recent proposed methods for exploring EO data are mainly based on Image Information Mining (IIM). Because the main steps of this approach are image features extraction, data reduction, and labeling, developing a new process chain, mainly based on human interaction, might be a promising solution. More precisely, this chain provides an active learning system by interacting human users with features descriptors in a virtual environment. The focus of this article is an Immersive Visual Information Mining in which feature descriptors/images are visualized in a 3-D virtual environment, so called CAVE. This environment also allows the users to manipulate the feature space to increase the performance of learning process in an interactive manner. In our experiments, we use a dataset of Synthetic Aperture Radar (SAR) images. To process the data, the contents of images are represented by three different feature descriptors comprising Gabor, adapted WLD, and Bag-of-Words.

The present article presents a method for assessing the radial stiffness of nitinol stents. An idealized stent model was created, and its radial stiffness was calculated by means of finite element modeling. The calculations were validated against experimental measurements. The variation of radial stiffness with geometrical dimensions was calculated, and the effect of increasing radial stiffness on endovascular deployment was analyzed. Peak tensile and compressive stresses as well as stent penetration were calculated in the case of an idealized pulmonary artery model having realistic dimensions as well as stiffness. The results of stress calculations were compared with a second set of simulations, where an idealized behavior of the stent (uniform expansion to a theoretical contact diameter) was modeled. The results show how in reality nitinol stents behave in a non-ideal way, having a non-uniform expansion and exerting non-uniform pressure on the contact areas with the artery. Such non-ideality decreases though with the increase in radial stiffness. The radial force alone may be insufficient in describing the stent-artery interaction, and numerical modeling proves to be necessary for capturing such complexity.

Haze contains floating particles in the air which can result in image quality degradation and visibility reduction in airborne data. Haze removal task has several applications in image enhancement and can improve the performance of automatic image analysis systems, namely object detection and segmentation. Unlike rich haze removal literature in ground imagery, there is a lack of methods specifically designed for aerial imagery, considering the fact that there is a characteristic difference between the aerial imagery domain and ground one. In this paper, we propose a method to dehaze aerial images using Convolutional Neural Networks~(CNNs). Currently, there is no available data for dehazing methods in aerial imagery. To address this issue, we have created a synthetically-hazed aerial image dataset to train the neural network on aerial hazy image dataset. We train All-in-One dehazing network (AOD-Net) as the base approach on hazy aerial images and compare the performance of our proposed approach against the classical model. We have tested our model on natural as well as the synthetically-hazed aerial images. Both qualitative and quantitative results of the adapted network show an improvement in dehazing results. We show that the adapted AOD-Net on our aerial image test set increases PSNR and SSim by 2.2% and 9%, respectively.