Search

Under seismic actions, stone masonry buildings are prone to damage. To assess the severity of damaged masonry buildings and their failure modes, engineers connect these problems to surface crack features, such as the crack width and the extent of cracking. We aim to further these assessments in this study, wherein we propose using simple machine learning models to predict: 1) three ratios encoding the degradation of stiffness, strength, and displacement capacity of damaged rubble stone masonry piers as a function of the observed crack features and the applied axial load and shear span ratio; and 2) the pre-peak vs. post-peak regime, based on the crack features. When predicting the stiffness, force, and drift ratios, the prediction error is significantly reduced when the axial load and shear span ratio are included in the feature vector. Furthermore, when predicting the pre-peak vs. post-peak regime, simple machine learning models such as the k-nearest neighbor and the logistic regression result in remarkable accuracy. The obtained results have significant implications on the automated post-earthquake assessment of masonry buildings using image data. It is shown based on documented laboratory test data, that, by selecting proper crack features and incorporating information about the kinematic and static boundary conditions, even simple machine learning models can predict accurately the damage level caused to a rubble masonry pier. The three crack features used in this study are the maximum crack width, length density, and complexity dimension. The pipeline developed in this paper is general enough and is applicable to other masonry typologies and elements upon new evaluation of crack features and image data., Automation in Construction, 140, ISSN:0926-5805

An excessive deposition of collagen matrix in the myocardium has been clearly identified as an indication of the progression towards heart failure. Nevertheless, few studies have been performed for its quantification and most of them use 2D histological images, thus losing valuable encoded 3D information. In this study, several biopsies of areas of the left ventricle from age-matched spontaneously hypertensive rats and Wistar Kyoto rats were imaged using synchrotron radiation-based X-ray phase contrast imaging. Then, an optimized deep neural network was used for automatic image segmentation in order to assess collagen fraction differences between models as well as its age dependency. The results show a general increase in the collagen percentage in the hypertensive model and for older rats. Such tendency is comparable with the reports found in the literature. Therefore, this proof of concept shows that synchrotron imaging in combination with deep neural networks is a powerful tool for the investigation and quantification of cardiac microstructures.

In digital photography, two image restoration tasks have been studied extensively and resolved independently: demosaicing and super-resolution. Both these tasks are related to resolution limitations of the camera. Performing super-resolution on a demosaiced images simply exacerbates the artifacts introduced by demosaicing. In this paper, we show that such accumulation of errors can be easily averted by jointly performing demosaicing and super-resolution. To this end, we propose a deep residual network for learning an end-to-end mapping between Bayer images and high-resolution images. By training on high-quality samples, our deep residual demosaicing and super-resolution network is able to recover high-quality super-resolved images from low-resolution Bayer mosaics in a single step without producing the artifacts common to such processing when the two operations are done separately. We perform extensive experiments to show that our deep residual network achieves demosaiced and super-resolved images that are superior to the state-of-the-art both qualitatively and in terms of PSNR and SSIM metrics.

We present an improved version of the Simple Linear Iterative Clustering (SLIC) superpixel segmentation. Unlike SLIC, our algorithm is non-iterative, enforces connectivity from the start, requires lesser memory, and is faster. Relying on the superpixel boundaries obtained using our algorithm, we also present a polygonal partitioning algorithm. We demonstrate that our superpixels as well as the polygonal partitioning are superior to the respective state-of-the-art algorithms on quantitative benchmarks.

Reflections are a common artifact in images taken through glass windows. Automatically removing the reflection artifacts after the picture is taken is an ill-posed problem. Attempts to solve this problem using optimization schemes therefore rely on various prior assumptions from the physical world. Instead of removing reflections from a single image, which has met with limited success so far, we propose a novel approach to suppress reflections. It is based on a Laplacian data fidelity term and an l-zero gradient sparsity term imposed on the output. With experiments on artificial and real-world images we show that our reflection suppression method performs better than the state-of-the-art reflection removal techniques.

Amateur home videos rarely convey intent effectively, primarily because of the limitations of conventional consumer-quality video cameras and the difficulties of video postprocessing. The authors describe a general approach for video-intent delivery based on offline cinematography and automated continuity editing concepts and demonstrate its use with four basic emotions: cheer, serenity, gloom, and excitement.

It is becoming increasingly clear that mitochondria play an important role in neural function. Recent studies show mitochondrial morphology to be crucial to cellular physiology and synaptic function and a link between mitochondrial defects and neuro-degenerative diseases is strongly suspected. Electron microscopy (EM), with its very high resolution in all three directions, is one of the key tools to look more closely into these issues but the huge amounts of data it produces make automated analysis necessary. State-of-the-art computer vision algorithms designed to operate on natural 2-D images tend to perform poorly when applied to EM data for a number of reasons. First, the sheer size of a typical EM volume renders most modern segmentation schemes intractable. Furthermore, most approaches ignore important shape cues, relying only on local statistics that easily become confused when confronted with noise and textures inherent in the data. Finally, the conventional assumption that strong image gradients always correspond to object boundaries is violated by the clutter of distracting membranes. In this work, we propose an automated graph partitioning scheme that addresses these issues. It reduces the computational complexity by operating on supervoxels instead of voxels, incorporates shape features capable of describing the 3-D shape of the target objects, and learns to recognize the distinctive appearance of true boundaries. Our experiments demonstrate that our approach is able to segment mitochondria at a performance level close to that of a human annotator, and outperforms a state-of-the-art 3-D segmentation technique.

While there has been substantial progress in segmenting natural images, state-of-the-art methods that perform well in such tasks unfortunately tend to underperform when confronted with the different challenges posed by electron microscope (EM) data. For example, in EM imagery of neural tissue, numerous cells and subcellular structures appear within a single image, they exhibit irregular shapes that cannot be easily modeled by standard techniques, and confusing textures clutter the background. We propose a fully automated approach that handles these challenges by using sophisticated cues that capture global shape and texture information, and by learning the specific appearance of object boundaries. We demonstrate that our approach significantly outperforms state-of-the-art techniques and closely matches the performance of human annotators.

Detection of salient image regions is useful for applications like image segmentation, adaptive compression, and region-based image retrieval. In this paper we present a novel method to determine salient regions in images using low-level features of luminance and color. The method is fast, easy to implement and generates high quality saliency maps of the same size and resolution as the input image. We demonstrate the use of the algorithm in the segmentation of semantically meaningful whole objects from digital images.

Faces usually are the most interesting objects in certain categories of video, like home videos and news clips. A novel sensor fusion based face tracking system is presented that tracks faces in compressed video, and aids automatic video indexing. Tracking is done by fusing the measurements from three independent sensors - motion and colour based trackers (Achanta, R. et al., IEEE Int. Conf. on Multimedia and Expo, 2002) and a face detector (Wang, J. et al., Proc. Int. Workshop on Advanced Image Technology, 2002) using a novel hierarchical framework based on Kalman filter state vector fusion. The tracking results show that the fused results are better than those of any individual sensors or their mean.

Object tracking is of utmost importance for automatic indexing of video content. This work presents an object tracker that operates directly on MPEG compressed data. Motion vectors and discrete cosine transform (DCT) coefficients directly available from the compressed video stream are exploited for the purpose of tracking. Tracking proceeds in two steps: motion vector based tracking in P and B frames within the groups of pictures (GOPs), and object identification in I frames. Colour, which is one of the strongest cues for tracking is used for the identification step. Such a system offers speed, simplicity and robustness against occlusion and camera motion, with good intra-shot tracking for shots in excess of 500 frames, as shown in the experimental results.

Face recognition systems are designed to handle well-aligned images captured under controlled situations. However real-world images present varying orientations, expressions, and illumination conditions. Traditional face recognition algorithms perform poorly on such images. In this paper we present a method for face recognition adapted to real-world conditions that can be trained using very few training examples and is computationally efficient. Our method consists of performing a novel alignment process followed by classification using sparse representation techniques. We present our recognition rates on a difficult dataset that represents real-world faces where we significantly outperform state-of-the-art methods.

Reliable methods for detecting pixels that represent cracks from laboratory images taken for digital image correlation (DIC) are required for two main reasons. Firstly, the segmented crack maps are used as an input for some DIC methods that are based on discontinuous fields. Secondly, detected crack patterns can serve as inputs for predictive empirical models to obtain the level of damage to a body. The aim of this paper is to compare the performance of two approaches for crack segmentation on grayscale images acquired from two experimental campaigns on stone masonry walls. In the first approach, a threshold is applied to the maximum principal strain map calculated using post-processed DIC results. In the second approach, a deep convolutional neural network is used. The two methods are compared in terms of standard segmentation criteria, namely precision, dice coefficient and sensitivity. It is shown that the precision and dice coefficient obtained from the deep learning approach are much higher than those obtained from the threshold method (by almost 47% and 34%, respectively). However, the sensitivity computed from the deep learning method is slightly (~4%) lower than the threshold method. These results show that the deep learning method can better preserve the geometry of detected crack patterns, and the prediction in terms of pixels belonging to a crack is finally more accurate than the threshold method.

While there has been substantial progress in segmenting natural im-ages, state-of-the-art methods that perform well in such tasks unfortunately tend to underperform when confronted with the different challenges posed by electron microscope (EM) data. For example, in EM imagery of neural tissue, numerous cells and subcellular structures appear within a single image, they exhibit irregular shapes that cannot be easily modeled by standard techniques, and confusing textures clutter the background. We propose a fully automated approach that handles these challenges by using sophisticated cues that capture global shape and texture information, and by learning the specific appearance of object boundaries. We demonstrate that our approach significantly outperforms state-of-the-art techniques and closely matches the performance of human annotators.

It is useful to have a small set of representative images from a database of thousands of images to summarize its content. There are two key aspects of such image summaries: how to generate them and how to present them. We address both issues. We extend the idea of image saliency to databases and introduce the notion of database saliency. We argue that in image databases, there are certain images that are more uncommon or salient than others and therefore are more interesting. We compute the database saliency value of an image as its total distance from all the pre-defined cluster centers of the database. We demonstrate the use of database saliency in two visualization applications: creating image collages and mosaics using automatically chosen salient images.

Computer vision applications have come to rely increasingly on superpixels in recent years, but it is not always clear what constitutes a good superpixel algorithm. In an effort to understand the benefits and drawbacks of existing methods, we empirically compare five state-of-the-art superpixel algorithms for their ability to adhere to image boundaries, speed, memory efficiency, and their impact on segmentation performance. We then introduce a new superpixel algorithm, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels. Despite its simplicity, SLIC adheres to boundaries as well as or better than previous methods. At the same time, it is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.

We address the difficult problem of estimating the attributes of weight and height of individuals from pictures taken in completely unconstrained settings. We present a deep learning scheme that relies on simultaneous prediction of human silhouettes and skeletal joints as strong regularizers that improve the prediction of attributes such as height and weight. Apart from imparting robustness to the prediction of attributes, our regularization also allows for better visual interpretability of the attribute prediction. For height estimation, our method shows lower mean average error compared to the state of the art despite using a simpler approach. For weight estimation, which has hardly been addressed in the literature, we set a new benchmark.

It is challenging to complete an image whose 99 percent pixels are randomly missing. We present a solution to this extreme image completion problem. As opposed to existing techniques, our solution has a computational complexity that is linear in the number of pixels of the full image and is real-time in practice. For comparable quality of reconstruction, our algorithm is thus almost 2 to 5 orders of magnitude faster than existing techniques.

A near-infrared (NIR) image of a scene contains different information than the corresponding visible spectrum (RGB) image, due to physical phenomena related to scattering, absorption, and reflectance of the different radiation bands. It can be desirable to combine the most relevant image content from both NIR and RGB to improve image quality. In this paper, we present two schemes of combining NIR and RGB information to obtain visually better quality images. The first is an automatic approach that performs segment-based NIR-blending by local and global entropy maximization. The second uses a GUI-aided approach where the user chooses the segments and manually con- trols blending. To minimize artifacts at the segment boundaries, three different boundary smoothing methods are introduced and compared in terms of speed and quality of output.

We propose Deep Feature Factorization (DFF), a method capable of localizing similar semantic concepts within an image or a set of images. We use DFF to gain insight into a deep convolutional neural network's learned features, where we detect hierarchical cluster structures in feature space. This is visualized as heat maps, which highlight semantically matching regions across a set of images, revealing what the network `perceives' as similar. DFF can also be used to perform co-segmentation and co-localization, and we report state-of-the-art results on these tasks., The European Conference on Computer Vision (ECCV), 2018

Detection of visually salient image regions is useful for applications like object segmentation, adaptive compression, and object recognition. Recently, full-resolution salient maps that retain well-defined boundaries have attracted attention. In these maps, boundaries are preserved by retaining substantially more frequency content from the original image than older techniques. However, if the salient regions comprise more than half the pixels of the image, or if the background is complex, the background gets highlighted instead of the salient object. In this paper, we introduce a method for salient region detection that retains the advantages of such saliency maps while overcoming their shortcomings. Our method exploits features of color and luminance, is simple to implement and is computationally efficient. We compare our algorithm to six state-of-the-art salient region detection methods using publicly available ground truth. Our method outperforms the six algorithms by achieving both higher precision and better recall. We also show application of our saliency maps in an automatic salient object segmentation scheme using graph-cuts.