Search

Nowadays monitoring health conditions of machines is necessary to reduce costs and repairing time and to secure the quality of the products. Therefore, the potential of acoustic measurements in combination with machine learning techniques for non-invasive diagnostics of machine performance has been investigated. The idea is to establish relations between the acoustic images produced by a sound camera and the machine conditions and then create a strategy for processing the images using Convolutional Neural Networks. Several working conditions of the machine have been considered and experiments have been performed both under nominal and abnormal conditions of the machine, obtained by mimicking the presence of a disturbance. The use of the algorithms for image classification allows isolation of the faults in the machine behaviour by the definition of the primary sound sources. The procedure shows promising results with a short computational time, easy application and high accuracy.

Hierarchical neural networks with large numbers of layers are the state of the art for most computer vision problems including image classification, multi-object detection and semantic segmentation. While the computational demands of training such deep networks can be addressed using specialized hardware, the availability of training data in sufficient quantity and quality remains a limiting factor. Main reasons are that measurement or manual labelling are prohibitively expensive, ethical considerations can limit generating data, or a phenomenon in questions has been predicted, but not yet observed. In this position paper, we present the Digital Reality concept are a structured approach to generate training data synthetically. The central idea is to simulate measurements based on scenes that are generated by parametric models of the real world. By investigating the parameter space defined of such models, training data can be generated in a controlled way compared to data that was captured from real world situations. We propose the Digital Reality concept and demonstrate its potential in different application domains, including industrial inspection, autonomous driving, smart grid, and microscopy research in material science and engineering.

Three-dimensional (3D) characterization of nanomaterials is traditionally performed by either cross-sectional milling with a focused ion beam (FIB), or transmission electron microscope tomography. While these techniques can produce high quality reconstructions, they are destructive, or require thin samples, often suspended on support membranes. Here, we demonstrate a complementary technique allowing non-destructive investigation of the 3D structure of samples on bulk substrates. This is performed by imaging backscattered electron (BSE) emission at multiple primary beam energies - as the penetration depth of primary electrons is proportional to the beam energy, depth information can be obtained through variations in the beam acceleration. The detected signal however consists of a mixture of the penetrated layers, meaning the structure's three-dimensional geometry can only be retrieved after deconvolving the BSE emission profile from the observed BSE images. This work demonstrates this novel approach by applying a blind source separation deconvolution algorithm to multi-energy acquired BSE images. The deconvolution can thereby allow a 3D reconstruction to be produced from the acquired images of an arbitrary sample, showing qualitative agreement with the true depth structure, as verified through FIB cross-sectional imaging.

This paper describes an imaging system that was developed to aid industrial bin picking tasks. The purpose of this system was to provide accurate 3D models of parts and objects in the bin, so that precise grasping operations could be performed. The technology described here is based on two types of sensors: range mapping scanners and video cameras. The geometry of bin contents was reconstructed from range maps and modeled using superquadric representations, providing location and parts surface information that can be employed to guide the robotic arm. Texture was also provided by the video streams and applied to the recovered models. The system is expected to improve the accuracy and efficiency of bin sorting and represents a step toward full automation.

In this paper we present two novel techniques developed in the context of the stereo to multi-view conversion research at Philips in support of the introduction of stereoscopic and auto-stereoscopic displays. First, we show that we can use a relatively simple filtering approach, based on the recently popular bilateral filters, to address the correspondence problem, which is at the heart of depth and motion estimation. The proposed recursive filter uses Gaussian kernels to filter best matches and to incorporate image-based constraints. It iteratively refines the depth values starting from a random initialization and converges in a limited number of iterations to a time-stable high-quality depth map. The second contribution of the paper is an occlusion detection method that uses robust filtering for the detection of occlusion that is primarily based on the analysis of the variation of the matching metric used in the disparity estimation process. The basic underlying ideas behind the occlusion detection method are (1) that occluded areas are highly likely to be located near image boundaries (where luminance or color changes abruptly), and (2) occluded regions are characterized by a large decrease in the quality of the matching metric across these boundaries. The two algorithms were tested on real-world stereoscopic video content showing promising results.

Recently 3D face reconstruction and recognition has gained an important role in computer vision and biometrics research. Depth information of a 3D face can aid solving the uncertainties in illumination and pose variation associated with face recognition. The registration of data that is usually acquired from different views is a fundamental element of any reconstruction process. This chapter focuses on the problem of automatic registration of 3D face point sets through a criterion based on Gaussian fields. The method defines a straightforward energy function, which is always differentiable and convex in a large neighborhood of the alignment parameters; allowing for the use of powerful standard optimization techniques. The introduced technique overcomes the necessity of close initialization, which is a requirement when applying the Iterative Closest Point algorithm. Moreover, the use of the Fast Gauss Transform reduces the computational complexity of the registration algorithm.

This research focuses on the conversion of stereoscopic video material into an image + depth format which is suitable for rendering on the multiview auto-stereoscopic displays of Philips. The recent interest shown in the movie industry for 3D significantly increased the availability of stereo material. In this context the conversion from stereo to the input formats of 3D displays becomes an important task. In this paper we present a stereo algorithm that uses multiple footprints generating several depth candidates for each image pixel. We characterize the various matching windows and we devise a robust strategy for extracting high quality estimates from the resulting depth candidates. The proposed algorithm is based on a surface filtering method that employs simultaneously the available depth estimates in a small local neighborhood while ensuring correct depth discontinuities by the inclusion of image constraints. The resulting highquality image-aligned depth maps proved an excellent match with our 3D displays.

The focus of this paper is on the reconstruction of 3D representations of real world scenes and objects using multiple sensors with, as one of its main applications, the enhancement of the autonomy and mobility of unmanned vehicles. The sensors considered are primarily range acquisition devices (such as laser scanners and stereo systems) that allow the recovery of 3D geometry. One of the most important technical challenges that we are addressing is the registration task in both its multi-modal and single modality aspects. Our work is based on a unified approach that formulates the correspondence problem as an optimization task. In this context we developed a criterion that can be used for 3D free-form shape registration. The new criterion is derived from simple Boolean matching principles by approximation and relaxation. Technically, one of the main advantages of the proposed approach is convexity in the neighborhood of the alignment parameters and continuous differentiability, allowing for the use of standard gradient-based optimization techniques. The proposed algorithm allows also for a significant automation of the scene modeling task by reducing the intervention of human operators in the tedious image registration task. Furthermore, we show that the criterion can be computed in linear time complexity which permits the fast implementation critical in many applications of autonomous mobile platforms.

Extended abstract of a paper presented at Microscopy and Microanalysis 2013 in Indianapolis, Indiana, USA, August 4 – August 8, 2013.

In this paper we present a new method for the registration of multiple sensors applied to a mobile robotic inspection platform. Our main technical challenge is automating the integration process for various multimodal inputs, such as depth maps, and multi-spectral images. This task is approached through a unified framework based on a new registration criterion that can be employed for both 3D and 2D datasets. The system embedding this technology reconstructs 3D models of scenes and objects that are inspected by an au tonomous platform in high security areas. The models are processed and rendered with corresponding multi-spectral textures, which greatly enhances both human and machine identification of threat objects.

Three dimensional reconstruction of bulk biological samples in SEM is currently achieved by Serial Sectioning or Serial Block Face (SBF) techniques. To image deeper layers of a sample these approaches differ mainly in the physical cutting method employed: diamond-knife microtome slicing or Focused Ion Beam (FIB) milling. During stack acquisition slicing steps are alternated with imaging steps to build the 3D volume. While these methods are proving very useful for a variety of Cell Biology, and in particular Neuroanatomy studies, they are still limited to a z-resolution of around 5nm for FIB techniques, 15-30nm for Serial Block Face (microtome) Slicing and around 2540nm for serial section techniques. Furthermore most of these techniques are destructive to the imaged samples. What TEM/STEM tomography can achieve for thin sections, being in principle non-destructive and providing high levels of resolution, is clearly missing in 3D SEM on bulk. Nonetheless, recent work [1] on computational modeling of beam-sample interactions in epoxyembedded osmium-stained samples shows remarkable signal linearity, offering well-proven deconvolution techniques as a tool for 3D reconstruction. Such models, significantly improved and extended with our own experimental work, indicate that the Point Spread Function (PSF) governing backscatter (BS) image formation remains well confined in the lateral directions for over 100 nm below the sample surface for commonly used landing energies (around 2keV).

The Scanning Electron Microscope (SEM) as a three dimensional imaging tool is increasingly popular for studies in neurobiology. Serial slicing methods based on diamond-knife cutting are, however, reaching practical limitations in terms of achievable z-resolution and voxel isotropy. While Focused-Ion Beam SEM serial block face imaging can improve the z-resolution to 5nm, this technology is restricted in terms of the total volume of material that can be processed. In this work we describe the Multi Energy Confocal Scanning Electron Microscope or “MECSEM” method that achieves high isotropic resolution by a combination of backscatter image sequence acquisition and deconvolution (DC).The method is built on improved understanding of beam-sample interaction for classically prepared samples. Furthermore the point spread function (PSF) of backscatter electrons (BSE) in these materials appears to be well confined laterally. As the range of penetration in the sample is dependent on the energy of the primary beam, acquiring an image sequence with increasing landing energies leads to the acquisition of images from increasing depth.The lateral confinement of BSE PSFs allows for restricting the DC to a layer separation task (z direction). As the structure of the PSF is difficult to obtain experimentally it will be considered unknown leading to a blind DC problem requiring the recovery of both depth layers and PSFs.To verify the reconstruction results we combine this technique with classical FIB-SEM serial block imaging using the through the lens detector in BSE mode on an FEI Helios 650 DualBeam. The z-resolution was controlled by varying the primary beam energy. The comparison with finer FIB reconstruction shows identical structures on the studied samples proving the reliability of the 3D technique, while the achieved depth using DC can reach up to 180nm.