Search

The expansion of off-/onshore wind farms plays a key role in the transformation of energy production from burning of fossil fuels and nuclear energy to sustainable and safe power generation. However, the wind energy sector is permanently under strong cost pressure and the maintenance of the turbines is currently still carried out quite expensively with human industrial climbers. In this article, we present the results of an interdisciplinary research project on the automation of various image-based inspection steps. Since the use of unmanned aerial vehicles (UAV) is a problem especially offshore, we present here a simple, cost-effective method to obtain a three-dimensional model of a wind energy plant using solely a digital camera equipped with a sensor array to use it for the detection and management of damages and abnormalities. A first approach to detect abnormalities on the surface with deep learning methods achieved an F1-score of about 95%.

The application of protective coating systems is the major measure against the corrosion of steel structures for onshore wind turbines. The organic coatings are, however, susceptible to atmospheric exposure and tend to deteriorate during the operation. At the same time, onshore turbines become more powerful and require taller and more resistant tower structures. The inspection and condition monitoring of protective coating systems on large onshore turbines (in excess of 120 m height) is a demanding and time-consuming procedure and requires high human effort. The rapid developments in digitization and data analysis offer opportunities to notably increase the efficiency of monitoring processes and to develop (semi-)automated standardized procedures. The paper describes a data-oriented approach to utilize digital data for the monitoring and maintenance planning of surface protection systems of large onshore wind turbines. The proposed approach includes the following steps: the segmentation of an existing wind power structure into a number of reference areas based on an In-situ Virtual Twin; the definition of a local deterioration degree for each individual reference area; the annotation of image data; the use of heterogenous multi-modal data (image data, geodetical data, meteorological data, profile scanning data) as the sources for condition assessment and monitoring. An example procedure is exercised for a tower structure of an onshore wind power turbine in order to illustrate the practical relevance of the approach.

The paper describes the concept of a fully data-based, mainly automated, procedure for the condition monitoring of surface protection systems on steel structures. Acquisition, processing and evaluation of relevant data as well as their implementation into a mathematical-empirical condition monitoring model are discussed. The monitoring model is based on the definition and assessment of a local deterioration degree, which is characteristic for pre-defined reference areas at the structure. The creation of the reference areas by means of In-situ Virtual Twin Computation is introduced. The data-based assessment procedure is practically exemplified on a steel tower of an onshore wind power structure in a rural corrosive environment. The procedure can be applied to steel structures with similar geometries (storage tanks, silos, chimneys) as well as to offshore conditions.

The application of protective coating systems is the major measure against the corrosion of steel for tower sections of wind turbines. The inspection, condition monitoring and maintenance of protective coating system is a demanding and time-consuming procedure and requires high human effort. The paper introduces for the first time a Digital Twin concept for the condition monitoring and prescriptive maintenance planning for surface protection systems on wind turbine towers. The initial point of the concept is an in-situ Virtual Twin for the generation of reference areas for condition monitoring. The paper describes the integration of an online image annotation and processing tool, a maintenance model, corrosive resistance parameters, structural load parameters, and sensor data into the Digital Twin concept. The prospects of the concept and its practical relevance are shown for tower structures of large onshore wind turbines.

The number of offshore windmills keeps growing around the world since they are an essential part of any strategy to produce energy without using nuclear power plants or burning fossil resources. Due to the growing number of installations new monitoring and inspection strategies and routines need to be developed that are not only effective but efficient. In this work we will present first results from a joint project of industry partners and academia that aims at the development of new approaches making use of modern imaging technologies and new methods from artificial intelligence and computer vision research. We investigate three different strategies for collecting digital photos or video and posterior computational analysis using Convolutional Neural Networks (CNN). Our results indicate, that patterns of interest like rust or coating damage can be detected and classified with F 1 scores between 0.80 and 0.94 in photos collected by inspectors. In videos collected with unmanned aerial vehicles (UAV) rust patterns, oils spills and coating damage were detected with F 1 = 0.91. To support users in the time-consuming task of visual data exploration, we present a new visualization method referred to as “virtual twin”. Using hand - recorded image collections, a data-driven 3D model of a windmill is generated, that can be used to investigate damage patterns in the full context of the windmill and in full detail. Altogether our results indicate, that although each single approach has its limitations, a combination of different imaging methods, deep learning computer vision algorithms and sophisticated visual data exploration by experienced users appears to have potential to overcome the bottleneck in data analysis, interpretation and decision making in the context of the future inspections of offshore windmills, platforms or other constructions.

Wind energy is a critical part of overcoming the use of fossil or nuclear energy usage. The price pressure on the renewable industry sector demands to cut the costs for costly regular inspections carried out by industrial climbers. Drone-based video-inspection reduces costs as well as increases the safety of inspection personal. To further increase the throughput, automatic or semi-automatic solutions to analyze these videos are needed. However, modern machine learning architectures need a lot of data to work reliably. This is by design a problem, as structural damage is rather rare in industrial infrastructure. Our proposed approach uses Generative Adversarial Networks to generate synthetic unmanned aerial vehicle imagery. This allows us to create a large enough training dataset (> 103) from a dataset, which is at least an order of magnitude smaller (approx. 102). We show that we can increase the classification accuracy of up to 6 percentage points.

Deep convolutional neural networks are emerging as the state of the art method for supervised classification of images also in the context of taxonomic identification. Different morphologies and imaging technologies applied across organismal groups lead to highly specific image domains, which need customization of deep learning solutions. Here we provide an example using deep convolutional neural networks (CNNs) for taxonomic identification of the morphologically diverse microalgal group of diatoms. Using a combination of high-resolution slide scanning microscopy, web-based collaborative image annotation and diatom-tailored image analysis, we assembled a diatom image database from two Southern Ocean expeditions. We use these data to investigate the effect of CNN architecture, background masking, data set size and possible concept drift upon image classification performance. Surprisingly, VGG16, a relatively old network architecture, showed the best performance and generalizing ability on our images. Different from a previous study, we found that background masking slightly improved performance. In general, training only a classifier on top of convolutional layers pre-trained on extensive, but not domain-specific image data showed surprisingly high performance (F1 scores around 97%) with already relatively few (100–300) examples per class, indicating that domain adaptation to a novel taxonomic group can be feasible with a limited investment of effort.

With the mining of polymetallic nodules from the deep sea seafloor again approaching commercial viability, decisions must be taken on how to most efficiently regulate and monitor physical and community disturbance in these remote ecosystems. Image based approaches allow non-destructive assessment of larger fauna abundances to be derived from survey data, with repeat surveys of areas possible to allow time series data collection. At time of writing key underwater imaging platforms commonly used to map seafloor fauna abundances are Automated Underwater Vehicles (AUVs), Remotely Operated Vehicles (ROVs) and towed camera Ocean Floor Observation Systems (OFOSs). These systems are highly customisable, with mounted cameras, illumination systems and deployment protocols rapidly changing over time, and even within survey cruises. In this study 8 image datasets were collected from a discrete area of polymetallic nodule rich seafloor by an AUV and several OFOSs deployed at various altitudes above the seafloor. A fauna identification catalogue was used by 5 annotators to estimate the abundances of 20 fauna categories from the different data sets. Results show that for many categories of megafauna differences in image resolution greatly influenced the estimations of fauna abundance determined by the annotators. This is an important finding for the development of future monitoring legislation for these areas. When and if commercial exploitation of these marine resources commences, to ensure best monitoring practice, unambiguous rules on how camera-based monitoring surveys should be conducted, and with what equipment, must be put in place.

Digital imaging is gaining more and more attention in marine environmental monitoring and exploration. Nowadays, mobile platforms such as autonomous underwater vehicles (AUVs) are equipped with high-resolution cameras that can collect gigabytes of digital images in a single dive. To extract quantitative and qualitative information from the accumulating image collections, annotation tools such as BIIGLE 2.0 have been proposed recently and have been established in the data analysis workflow. These tools run as web applications on a central server and can be accessed worldwide via the Internet. However, marine science and engineering are naturally associated with a high degree of mobility and, in some cases, limited resources or Internet access. Here we present a new application architecture for BIIGLE 2.0, which is particularly suitable for offshore deployment on a variety of platforms such as a server, workstation, laptop or even a small single-board computer such as a Raspberry Pi. We refer to the application architecture in combination with a mobile hardware platform as "BIIGLE2Go", which addresses the need for more flexibility and mobility in image annotation. We present and evaluate a first prototype for BIIGLE2Go, which runs as a mobile annotation system on a low-cost Raspberry Pi 3B.

Within the recent years clock rates of modern processors stagnated while the demand for computing power continued to grow. This applied particularly for the fields of life sciences and bioinformatics, where new technologies keep on creating rapidly growing piles of raw data with increasing speed. The number of cores per processor increased in an attempt to compensate for slight increments of clock rates. This technological shift demands changes in software development, especially in the field of high performance computing where parallelization techniques are gaining in importance due to the pressing issue of large sized datasets generated by e.g., modern genomics. This paper presents an overview of state-of-the-art manual and automatic acceleration techniques and lists some applications employing these in different areas of sequence informatics. Furthermore, we provide examples for automatic acceleration of two use cases to show typical problems and gains of transforming a serial application to a parallel one. The paper should aid the reader in deciding for a certain techniques for the problem at hand. We compare four different state-of-the-art automatic acceleration approaches (OpenMP, PluTo-SICA, PPCG, and OpenACC). Their performance as well as their applicability for selected use cases is discussed. While optimizations targeting the CPU worked better in the complex k-mer use case, optimizers for Graphics Processing Units (GPUs) performed better in the matrix multiplication example. But performance is only superior at a certain problem size due to data migration overhead. We show that automatic code parallelization is feasible with current compiler software and yields significant increases in execution speed. Automatic optimizers for CPU are mature and usually no additional manual adjustment is required. In contrast, some automatic parallelizers targeting GPUs still lack maturity and are limited to simple statements and structures.

Motivation: Bioimaging techniques rapidly develop toward higher resolution and dimension. The increase in dimension is achieved by different techniques such as multitag fluorescence imaging, Matrix Assisted Laser Desorption / Ionization (MALDI) imaging or Raman imaging, which record for each pixel an N-dimensional intensity array, representing local abundances of molecules, residues or interaction patterns. The analysis of such multivariate bioimages (MBIs) calls for new approaches to support users in the analysis of both feature domains: space (i.e. sample morphology) and molecular colocation or interaction. In this article, we present our approach WHIDE (Web-based Hyperbolic Image Data Explorer) that combines principles from computational learning, dimension reduction and visualization in a free web application. Results: We applied WHIDE to a set of MBI recorded using the multitag fluorescence imaging Toponome Imaging System. The MBI show field of view in tissue sections from a colon cancer study and we compare tissue from normal/healthy colon with tissue classified as tumor. Our results show, that WHIDE efficiently reduces the complexity of the data by mapping each of the pixels to a cluster, referred to as Molecular Co-Expression Phenotypes and provides a structural basis for a sophisticated multimodal visualization, which combines topology preserving pseudocoloring with information visualization. The wide range of WHIDE's applicability is demonstrated with examples from toponome imaging, high content screens and MALDI imaging (shown in the Supplementary Material). Availability and implementation: The WHIDE tool can be accessed via the BioIMAX website http://ani.cebitec.uni-bielefeld.de/BioIMAX/; Login: whidetestuser; Password: whidetest. Supplementary information: Supplementary data are available at Bioinformatics online. Contact: tim.nattkemper@uni-bielefeld.de

study of mapping and interaction of co-localized proteins at a sub-cellular level is important for understanding complex biological phenomena. One of the recent techniques to map co-localized proteins is to use the standard immuno-fluorescence microscopy in a cyclic manner (Nat Biotechnol 24:1270–8, 2006; Proc Natl Acad Sci 110:11982–7, 2013). Unfortunately, these techniques suffer from variability in intensity and positioning of signals from protein markers within a run and across different runs. Therefore, it is necessary to standardize protocols for preprocessing of the multiplexed bioimaging (MBI) data from multiple runs to a comparable scale before any further analysis can be performed on the data. In this paper, we compare various normalization protocols and propose on the basis of the obtained results, a robust normalization technique that produces consistent results on the MBI data collected from different runs using the Toponome Imaging System (TIS). Normalization results produced by the proposed method on a sample TIS data set for colorectal cancer patients were ranked favorably by two pathologists and two biologists. We show that the proposed method produces higher between class Kullback-Leibler (KL) divergence and lower within class KL divergence on a distribution of cell phenotypes from colorectal cancer and histologically normal samples. Electronic supplementary material The online version of this article (doi:10.1186/s13040-016-0088-2) contains supplementary material, which is available to authorized users.

Computational underwater image analysis is developing into a mature field of research, with an increasing number of companies, academic groups and researchers showing interest in it. While on the one hand, the basic question is addressed by many groups, how algorithms can be applied to automatically detect and classify objects of interest (OOI) in underwater image footage, on the other hand the questions for efficiency and performance, i.e. the time a computer (or a compute cluster) needs to perform this task, has received much attention yet. In this paper we will show, how nowadays methods for high performance computing like parallelization and GPU computing via CUDA (Compute Unified Device Architecture) can be used to achieve both, image enhancement and segmentation in less than 0.2 sec per image (4224 × 2376 pixel) on average, which paves the way to real time online applications.

We propose a machine learning based approach to real-time detection and classification assistance for images from unknown environments. While systems for detecting and classifying regular structures like faces in still images are well established, the task of e. g. detecting new morphotypes/objects in an environment is much more complex. The morphotypes/objects are not guaranteed to have apriori known characteristics such as facial landmarks, thus adaptive and data-driven approaches are needed that are trained online with a growing number of available labels provided by a human observer. Our approach combines an effective points of interest computation with an online learning network for real-time proposition of classes. In an example application using images from an arctic deep-sea underwater observatory we report a maximum accuracy of 0.87 and a top-3 accuracy of 0.97 in the range from 10 to 1499 available labels. The average accuracy in that range is 0.74, with the top-3 accuracy being 0.93. Additionally, a convolutional neural network achieves an accuracy of 0.965 at 1500 labels. Points of interest are computed in approximately 3.7 seconds, right after loading a new image. Proposing a class for a computationally determined point of interest or a manually selected one takes about 0.2 seconds. If the image is already in cache the time is reduced to 0.1 seconds. The suggestion of points of interest combined with a proposition of putative classes allows for significantly faster labeling times in new environments.