Search

This paper proposes a novel approach to detect road intersections from GNSS traces. Different from the existing methods of detecting intersections directly from the road users' turning behaviors, the proposed method detects intersections indirectly from common sub-tracks shared by different traces. We first compute the local distance matrix for each pair of traces. Second, we apply image processing techniques to find all "sub-paths" in the matrix, which represents good alignment between local common sub-tracks. Lastly, we identify the intersections from the endpoints of the common sub-tracks through Kernel Density Estimation (KDE). Experimental results show that the proposed method outperforms the traditional turning point-based methods in terms of the F-score, and our previous connecting point-based method in terms of computational efficiency. [ABSTRACT FROM AUTHOR]

Over the last two decades, UAVs have become an indispensable acquisition platform in the remote sensing community. Meanwhile, advanced lightweight sensors have been introduced in the market, including LiDAR scanners with multiple beams and hyperspectral cameras measuring reflectance using many different narrow-banded filters. To date, however, few fully fledged drone systems exist that combine different sensing modalities in a way that complements the strengths and weaknesses of each. In this paper, we present our multimodal drone payload and sensor fusion pipeline, which allows multispectral point clouds to be generated at subcentimeter accuracy. To that end, we combine high-frequency navigation outputs from a professional-grade GNSS with photogrammetric bundle adjustment and a dedicated point cloud registration algorithm that takes full advantage of LiDAR's specifications. We demonstrate that the latter significantly improves the quality of the reconstructed point cloud in terms of fewer ghosting effects and less noise. Finally, we thoroughly discuss the impact of the quality of the GNSS/INS system on the structure from the motion and LiDAR SLAM reconstruction process. [ABSTRACT FROM AUTHOR]

Multi-exposure image fusion (MEF) methods for high dynamic range (HDR) imaging suffer from ghosting artifacts when dealing with moving objects in dynamic scenes. The state-of-the-art methods use optical flow to align low dynamic range (LDR) images before merging, introducing distortion into the aligned LDR images from inaccurate motion estimation due to large motion and occlusion. In place of pre-alignment, attention-based methods calculate the correlation between the reference LDR image and non-reference LDR images, thus excluding misaligned regions in LDR images. Nevertheless, they also exclude the saturated details at the same time. Taking advantage of both the alignment and attention-based methods, we propose an efficient Deep HDR Deghosting Fusion Network (DDFNet) guided by optical flow and image correlation attentions. Specifically, the DDFNet estimates the optical flow of the LDR images by a motion estimation module and encodes that optical flow as a flow feature. Additionally, it extracts correlation features between the reference LDR and other non-reference LDR images. The optical flow and correlation features are employed to adaptably combine information from LDR inputs in an attention-based fusion module. Following the merging of features, a decoder composed of Dense Networks reconstructs the HDR image without ghosting. Experimental results indicate that the proposed DDFNet achieves state-of-the-art image fusion performance on different public datasets. [ABSTRACT FROM AUTHOR]

Abstract: Digital image correlation (DIC) has seen widespread acceptance and usage as a non-contact method for the determination of full-field displacements and strains in experimental mechanics. The advances of imaging hardware in the last decades led to high resolution and speed cameras being more affordable than in the past making large amounts of data image available for typical DIC experimental scenarios. The work presented in this paper is aimed at maximizing both the accuracy and speed of DIC methods when employed with such images. A low-level framework for speckle image partitioning which replaces regularly shaped blocks with image-adaptive cells in the displacement calculation is introduced. The Newton–Raphson DIC method is modified to use the image pixels of the cells and to perform adaptive regularization to increase the spatial consistency of the displacements. Furthermore, a novel robust framework for strain calculation based also on the Newton–Raphson algorithm is introduced. The proposed methods are evaluated in five experimental scenarios, out of which four use numerically deformed images and one uses real experimental data. Results indicate that, as the desired strain density increases, significant computational gains can be obtained while maintaining or improving accuracy and rigid-body rotation sensitivity. [Copyright &y& Elsevier]

In this paper we present a more efficient approach to the problem of finding the connected components in binary images. In conventional connected components algorithms, the main data structure to compute and store the connected components is the region label image. We replace the region label image with a singly-linked list of line segments (or runs) for each region. This enables us to design a very fast and memory efficient connected components algorithm. Most conventional algorithms require (at least) two raster scans. Those that only need one raster scan, require irregular and unbounded image access. The proposed algorithm is a single pass regular access algorithm and only requires access to the three most recently processed image lines at any given time. Experimental results demonstrate that our algorithm is considerably faster than the fastest conventional algorithm. Additionally, our novel region coding data structure uses much less memory in typical cases than the traditional region label image. Even in worst case situations the processing time of our algorithm is linear with the number of pixels in an image. [ABSTRACT FROM AUTHOR]

Depth sensing has improved rapidly in recent years, which allows for structural information to be utilized in various applications, such as virtual reality, scene and object recognition, view synthesis, and 3D reconstruction. Due to the limitations of the current generation of depth sensors, the resolution of depth maps is often still much lower than the resolution of color images. This hinders applications, such as view synthesis or 3D reconstruction, from providing high-quality results. Therefore, super-resolution, which allows for the upscaling of depth maps while still retaining sharpness, has recently drawn much attention in the deep learning community. However, state-of-the-art deep learning methods are typically designed and trained to handle a fixed set of integer-scale factors. Moreover, the raw depth map collected by the depth sensor usually has many depth data missing or misestimated values along the edges and corners of observed objects. In this work, we propose a novel deep learning network for both depth completion and depth super-resolution with arbitrary scale factors. The experimental results on the Middlebury stereo, NYUv2, and Matterport3D datasets demonstrate that the proposed method can outperform state-of-the-art methods. [ABSTRACT FROM AUTHOR]

We develop three novel wavelet domain denoising methods for subband-adaptive, spatially-adaptive and multivalued image denoising. The core of our approach is the estimation of the probability that a given coefficient contains a significant noise-free component, which we call "signal of interest." In this respect, we analyze cases where the probability of signal presence is 1) fixed per subband, 2) conditioned on a local spatial context, and 3) conditioned on information from multiple image bands. All the probabilities are estimated assuming a generalized Laplacian prior for noise-free subband data and additive white Gaussian noise. The results demonstrate that the new subband-adaptive shrinkage function outperforms Bayesian thresholding approaches in terms of mean-squared error. The spatially adaptive version of the proposed method yields better results than the existing spatially adaptive ones of similar and higher complexity. The performance on color and on multispectral images is superior with respect to recent multiband wavelet thresholding. [ABSTRACT FROM AUTHOR]

Abstract: The interpretation of ultrasound images remains a difficult task and the opinion of different doctors is generally not unequivocal. Therefore, there is a growing interest in the field of computer-aided diagnosis. In the field of medical image processing, computer-aided diagnosis includes image enhancement to facilitate visual interpretation, automatic indication of affected areas, organs and other regions of medical interest, the performance of automatic measurements and image registration. In this article, we introduce a new algorithm for ultrasound image enhancement that employs a multivariate texture classifier based on the co-occurrence matrix, which, in combination with an adaptive texture smoothing filter, is used to enhance the visual difference between and improve boundary detection between healthy neonatal brain tissue and tissue affected by periventricular leukomalacia. For a quantitative comparison, we delineate the periventricular leukomalacia-affected regions with two different active contours before and after processing 10 images with the proposed technique and several speckle filters from the literature. The semi-automatic delineations thus obtained are compared with the manual delineations of a neonatologist. In all cases, the average delineation achieved with the proposed technique is closer to that of the manual expert delineation than when the images are processed with the other techniques. [Copyright &y& Elsevier]

Image-scrambling schemes are designed to render the image content unintelligible. Wyner has proposed an elegant one-dimensional (1-D) scrambling scheme without bandwidth expansion, making use of the discrete prolate spheroidal sequences (LWSS). The DPSS are optimal regarding their energy concentration in a given frequency subband. In this paper, we propose the two-dimensional (2-D) extension and application of this algorithm. We discuss new possibilities introduced by the 2-D approach. We also include experimental results. [ABSTRACT FROM AUTHOR]

Many image processing systems, including those for printing applications, need sampling conversions for the representation of an image from one lattice to another. For example in the case of printing, classical halftoning requires new sample values on the halftone lattice. Although often considered as a straightforward procedure, resampling can cause so-called sampling moire due to aliasing. These artifacts are often very noticeable and as such undesirable, in particular for high-quality printing. In color printing, each color separation uses its own halftone lattice. Therefore, moire patterns will not only display an unexpected new frequency and orientation, but also influence the color appearance itself. These artifacts are frequently encountered in commercial (even high-quality) printing since the interpolation algorithms used in RIPs are simple (e.g., bilinear interpolation) and do not take into account the nature of the target lattice. Approaches such as simple low-pass filtering unacceptably blur the edges, while manual selective smoothing by an operator is very time-consuming.This paper proposes an optimal prefilter which is based on the least-squares linear resampling paradigm. Our approach requires proper discrete/continuous models, i.e., for both the source and the target lattices, and computes the associated reconstruction function which minimizes the error between the representations in the continuous domain. The reconstruction function jointly takes into account the Nyquist areas of every color separation using a novel hexagonal spline model resulting into an optimal prefilter before halftoning. Experimental results show that after prefiltering, the images are much less prone to moire while not suffering from noticeable edge blurring. [Copyright &y& Elsevier]

In this paper, we propose a robust wavelet domain method for noise filtering in medical images. The proposed method adapts itself to various types of image noise as well as to the preference of the medical expert; a single parameter can be used to balance the preservation of (expert-dependent) relevant details against the degree of noise reduction. The algorithm exploits generally valid knowledge about the correlation of significant image features across the resolution scales to perform a preliminary coefficient classification. This preliminary coefficient classification is used to empirically estimate the statistical distributions of the coefficients that represent useful image features on the one hand and mainly noise on the other. The adaptation to the spatial context in the image is achieved by using a wavelet domain indicator of the local spatial activity. The proposed method is of low complexity, both in its implementation and execution time. The results demonstrate its usefulness for noise suppression in medical ultrasound and magnetic resonance imaging. In these applications, the proposed method clearly outperforms single-resolution spatially adaptive algorithms, in terms of quantitative performance measures as well as in terms of visual quality of the images. [ABSTRACT FROM AUTHOR]

Presents a wavelet-based image denoising method which extends a geometrical Bayesian framework. Compromise between noise reduction and preserving significant image details; Simulation method for characterizing statistical properties of different significance measures for wavelet coefficients; Discussion of the isotropic Markov random field model.

Resampling is a common operation in digital image processing systems. The standard procedure involves the (conceptual) reconstruction of a continuous image succeeded by sampling on the new lattice sites. When the reconstruction is done by classical interpolation functions, results might be sub-optimal because the information loss is not minimized. In the particular case of subsampling (i.e., resampling to a coarser lattice), aliasing artifacts might arise and produce disturbing moire patterns. This paper first introduces a spline model for different orders, both for orthogonal and hexagonal lattices. Next, an expression for a least-squares approximation is derived which can be applied to convolution-based resampling. Experimental results for a printing application demonstrate the feasibility of the proposed method and are compared against the standard approach. Our technique can be applied to general least-squares resampling between regular lattices. [Copyright &y& Elsevier]

Presents a study which showed that an n x 1 integer vector can be exactly recovered from its Hadamard transform coefficients. Information on lossy image coding schemes; Theory of lossless quantization-dequantization; Lossless compression ratios for different images; Experimental results; Conclusion.

Compares techniques for intra-frame coding of arbitrarily shaped video object boundary blocks. Polynomial orthonormal transform outperforming shape adaptive discrete cosine transform (SADCT) and the delta SADCT with at the expense of a nonprohibitive increase in the number of computations; Description of the SADCT, delta DC-SADCT and the WSPOL methods.

With the rapid development in sensing technology, data mining, and machine learning fields for human health monitoring, it became possible to enable monitoring of personal motion and vital signs in a manner that minimizes the disruption of an individual's daily routine and assist individuals with difficulties to live independently at home. A primary difficulty that researchers confront is acquiring an adequate amount of labeled data for model training and validation purposes. Therefore, activity discovery handles the problem that activity labels are not available using approaches based on sequence mining and clustering. In this paper, we introduce an unsupervised method for discovering activities from a network of motion detectors in a smart home setting. First, we present an intra-day clustering algorithm to find frequent sequential patterns within a day. As a second step, we present an inter-day clustering algorithm to find the common frequent patterns between days. Furthermore, we refine the patterns to have more compressed and defined cluster characterizations. Finally, we track the occurrences of various regular routines to monitor the functional health in an individual's patterns and lifestyle. We evaluate our methods on two public data sets captured in real-life settings from two apartments during seven-month and three-month periods. [ABSTRACT FROM AUTHOR]

Extrinsic camera calibration is essential for any computer vision task in a camera network. Typically, researchers place a calibration object in the scene to calibrate all the cameras in a camera network. However, when installing cameras in the field, this approach can be costly and impractical, especially when recalibration is needed. This paper proposes a novel, accurate and fully automatic extrinsic calibration framework for camera networks with partially overlapping views. The proposed method considers the pedestrians in the observed scene as the calibration objects and analyzes the pedestrian tracks to obtain extrinsic parameters. Compared to the state of the art, the new method is fully automatic and robust in various environments. Our method detect human poses in the camera images and then models walking persons as vertical sticks. We apply a brute-force method to determines the correspondence between persons in multiple camera images. This information along with 3D estimated locations of the top and the bottom of the pedestrians are then used to compute the extrinsic calibration matrices. We also propose a novel method to calibrate the camera network by only using the top and centerline of the person when the bottom of the person is not available in heavily occluded scenes. We verified the robustness of the method in different camera setups and for both single and multiple walking people. The results show that the triangulation error of a few centimeters can be obtained. Typically, it requires less than one minute of observing the walking people to reach this accuracy in controlled environments. It also just takes a few minutes to collect enough data for the calibration in uncontrolled environments. Our proposed method can perform well in various situations such as multi-person, occlusions, or even at real intersections on the street. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new cooperative method that improves the accuracy of Turn Movement Count (TMC) under challenging conditions by introducing contextual observations from the surrounding areas. The proposed method focuses on the correct identification of the movements in conditions where current methods have difficulties. Existing vision-based TMC systems are limited under heavy traffic conditions. The main problems for most existing methods are occlusions between vehicles that prevent the correct detection and tracking of the vehicles through the entire intersection and the assessment of the vehicle's entry and exit points, incorrectly assigning the movement. The proposed method intends to overcome this incapability by sharing information with other observation systems located at neighboring intersections. Shared information is used in a cooperative scheme to infer the missing data, thereby improving the assessment that would otherwise not be counted or miscounted. Experimental evaluation of the system shows a clear improvement over related reference methods. [ABSTRACT FROM AUTHOR]

High dynamic range (HDR) imaging technology is increasingly being used in automated driving systems (ADS) for improving the safety of traffic participants in scenes with strong differences in illumination. Therefore, a combination of HDR video, that is video with details in all illumination regimes, and (HDR) object perception techniques that can deal with this variety in illumination is highly desirable. Although progress has been made in both HDR imaging solutions and object detection algorithms in the recent years, they have progressed independently of each other. This has led to a situation in which object detection algorithms are typically designed and constantly improved to operate on 8 bit per channel content. This makes these algorithms not ideally suited for use in HDR data processing, which natively encodes to a higher bit-depth (12 bits/16 bits per channel). In this paper, we present and evaluate two novel convolutional neural network (CNN) architectures that intelligently convert high bit depth HDR images into 8-bit images. We attempt to optimize reconstruction quality by focusing on ADS object detection quality. The first research novelty is to jointly perform tone-mapping with demosaicing by additionally successfully suppressing noise and demosaicing artifacts. The first CNN performs tone-mapping with noise suppression on a full-color HDR input, while the second performs joint demosaicing and tone-mapping with noise suppression on a raw HDR input. The focus is to increase the detectability of traffic-related objects in the reconstructed 8-bit content, while ensuring that the realism of the standard dynamic range (SDR) content in diverse conditions is preserved. The second research novelty is that for the first time, to the best of our knowledge, a thorough comparative analysis against the state-of-the-art tone-mapping and demosaicing methods is performed with respect to ADS object detection accuracy on traffic-related content that abounds with diverse challenging (i.e., boundary cases) scenes. The evaluation results show that the two proposed networks have better performance in object detection accuracy and image quality, than both SDR content and content obtained with the state-of-the-art tone-mapping and demosaicing algorithms. [ABSTRACT FROM AUTHOR]

Intelligent driver assistance systems are becoming increasingly popular in modern passenger vehicles. A crucial component of intelligent vehicles is the ability to detect vulnerable road users (VRUs) for an early and safe response. However, standard imaging sensors perform poorly in conditions of strong illumination contrast, such as approaching a tunnel or at night, due to their dynamic range limitations. In this paper, we focus on the use of high-dynamic-range (HDR) imaging sensors in vehicle perception systems and the subsequent need for tone mapping of the acquired data into a standard 8-bit representation. To our knowledge, no previous studies have evaluated the impact of tone mapping on object detection performance. We investigate the potential for optimizing HDR tone mapping to achieve a natural image appearance while facilitating object detection of state-of-the-art detectors designed for standard dynamic range (SDR) images. Our proposed approach relies on a lightweight convolutional neural network (CNN) that tone maps HDR video frames into a standard 8-bit representation. We introduce a novel training approach called detection-informed tone mapping (DI-TM) and evaluate its performance with respect to its effectiveness and robustness in various scene conditions, as well as its performance relative to an existing state-of-the-art tone mapping method. The results show that the proposed DI-TM method achieves the best results in terms of detection performance metrics in challenging dynamic range conditions, while both methods perform well in typical, non-challenging conditions. In challenging conditions, our method improves the detection F2 score by 13%. Compared to SDR images, the increase in F 2 score is 49%. [ABSTRACT FROM AUTHOR]

In order to develop smart laser powder bed fusion (LPBF) devices that autonomously identify a defect and remove it during the process for first-time-right and zero-defect parts, it is important to develop reliable on-machine defect measuring capabilities. As defects in LPBF parts often occur below the layer that is being processed, capturing the information of a printing layer may not give information about physical phenomena that are occurring below this layer. Therefore, to predict volumetric features such as porosity only by looking at the layer being processed, the correlation between process signatures identified in-process and defects measured via post-process inspection methods (for example X-ray computed tomography) needs to be conducted. Hence, in situ monitoring and post-process metrology form a basis to better understand the fundamental physics involved in an LPBF process and ultimately to determine its stability. By utilizing high-speed imaging, various process signatures are produced during single-track formation of 316L stainless steel with various combinations of laser power and scan speed. In this study, we evaluate whether these signatures can be used to detect the onset of potential defects. To identify process signatures, image segmentation and feature detection are applied to the monitoring data along the line scans. The process signatures determined in the current study are mainly related to the features like the process zone length-to-width ratio, process zone area, process zone mean intensity, spatter speed and number of spatters. It is shown that the scan speed has a significant impact on the process stability and spatter formation during single-track fusion. Simulations with similar processing conditions were also performed to predict melt pool geometric features. Post-process characterization techniques such as X-ray computed tomography and 2.5-D surface topography measurement were carried out for a quality check of the line track. An attempt was made to correlate physics-based features with process-related defects and a correlation between the number of keyhole porosities, and the number of spatters was observed for the line tracks. [ABSTRACT FROM AUTHOR]

Presents information on a study which considered the computation of the two-dimensional fast Fourier transform of images represented by quadtrees. Information on the fast Fourier transform algorithm.

Stereomatching is an effective way of acquiring dense depth information from a scene when active measurements are not possible. So-called lightfield methods take a snapshot from many camera locations along a defined trajectory (usually uniformly linear or on a regular grid--we will assume a linear trajectory) and use this information to compute accurate depth estimates. However, they require the locations for each of the snapshots to be known: the disparity of an object between images is related to both the distance of the camera to the object and the distance between the camera positions for both images. Existing solutions use sparse feature matching for camera location estimation. In this paper, we propose a novel method that uses dense correspondences to do the same, leveraging an existing depth estimation framework to also yield the camera locations along the line. We illustrate the effectiveness of the proposed technique for camera location estimation both visually for the rectification of epipolar plane images and quantitatively with its effect on the resulting depth estimation. Our proposed approach yields a valid alternative for sparse techniques, while still being executed in a reasonable time on a graphics card due to its highly parallelizable nature. [ABSTRACT FROM AUTHOR]

Recent advancements in visual sensor technologies have made behavior analysis practical for in-home monitoring systems. The current in-home monitoring systems face several challenges: (1) visual sensor calibration is a difficult task and not practical in real-life because of the need for recalibration when the visual sensors are moved accidentally by a caregiver or the senior citizen, (2) privacy concerns, and (3) the high hardware installation cost. We propose to use a network of cheap low-resolution visual sensors (30 × 30 pixels) for long-term behavior analysis. The behavior analysis starts by visual feature selection based on foreground/background detection to track the motion level in each visual sensor. Then a hidden Markov model (HMM) is used to estimate the user's locations without calibration. Finally, an activity discovery approach is proposed using spatial and temporal contexts. We performed experiments on 10 months of real-life data. We show that the HMM approach outperforms the k-nearest neighbor classifier against ground truth for 30 days. Our framework is able to discover 13 activities of daily livings (ADL parameters). More specifically, we analyze mobility patterns and some of the key ADL parameters to detect increasing or decreasing health conditions. ©The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI. [DOI: 10.1117/1.JEI.25.4.041003] [ABSTRACT FROM AUTHOR]

Pedestrian detection is an important research domain due to its relevance for autonomous and assisted driving, as well as its applications in security and industrial automation. Often, more than one type of sensor is used to cover a broader range of operating conditions than a single-sensor system would allow. However, it remains difficult to make pedestrian detection systems perform well in highly dynamic environments, often requiring extensive retraining of the algorithms for specific conditions to reach satisfactory accuracy, which, in turn, requires large, annotated datasets captured in these conditions. In this paper, we propose a probabilistic decision-level sensor fusion method based on naive Bayes to improve the efficiency of the system by combining the output of available pedestrian detectors for colour and thermal images without retraining. The results in this paper, obtained through long-term experiments, demonstrate the efficacy of our technique, its ability to work with non-registered images, and its adaptability to cope with situations when one of the sensors fails. The results also show that our proposed technique improves the overall accuracy of the system and could be very useful in several applications. [ABSTRACT FROM AUTHOR]

Pansharpening is an image fusion procedure, which aims to produce a high spatial resolution multispectral (MS) image by combining a low spatial resolution MS image and a high spatial resolution panchromatic image. The most popular and successful paradigm for pansharpening is the framework known as detail injection, while it cannot fully exploit complex and nonlinear complementary features of both images. In this article, we propose a detail-injection-model-inspired deep fusion network for pansharpening (DIM-FuNet). First, by treating pansharpening as a complicated and nonlinear detail learning and injection problem, we establish a unified optimizing detail-injection model with triple detail fidelity terms: 1) a band-dependent spatial detail fidelity term; 2) a local detail fidelity term; and 3) a complicated detail synthesis term. Second, the model is optimized via the iterative gradient descent and unfolded into a deep convolutional neural network. Subsequently, the unrolling network has triple branches, in which a point-wise convolutional subnetwork and a depth-wise convolutional subnetwork are corresponding to the former two detail-constrained terms, and an adaptive weighted reconstruction module with a fusion subnetwork to aggregate details of two branches and to synthesize the final complicated details. Finally, the deep unrolling network is trained in an end-to-end manner. Different from traditional deep fusion networks, the architecture design of DIM-FuNet is guided by the optimizing model and thus promotes better interpretability. Experimental results on reduced and full-resolution demonstrate the effectiveness of the proposed DIM-FuNet which achieves the best performance compared with the state-of-the-art pansharpening method. [ABSTRACT FROM AUTHOR]

Realistic visualization is crucial for a more intuitive representation of complex data, medical imaging, simulation, and entertainment systems. In this respect, multiview autostereoscopic displays are a great step toward achieving the complete immersive user experience, although providing high-quality content for these types of displays is still a great challenge. Due to the different characteristics/settings of the cameras in the multiview setup and varying photometric characteristics of the objects in the scene, the same object may have a different appearance in the sequences acquired by the different cameras. Images representing views recorded using different cameras, in practice, have different local noise, color, and sharpness characteristics. View synthesis algorithms introduce artifacts due to errors in disparity estimation/bad occlusion handling or due to an erroneous warping function estimation. If the input multiview images are not of sufficient quality and have mismatching color and sharpness characteristics, these artifacts may become even more disturbing. Accordingly, the main goal of our method is to simultaneously perform multiview image sequence denoising, color correction, and the improvement of sharpness in slightly defocused regions. Results show that the proposed method significantly reduces the amount of the artifacts in multiview video sequences, resulting in a better visual experience. [ABSTRACT FROM AUTHOR]

Pixel binning-based Quad sensors (mega-pixel resolution camera sensor) offer a promising solution to address the hardware limitations of compact cameras for low-light imaging. However, the binning process leads to reduced spatial resolution and introduces non-Bayer CFA artifacts. In this paper, we propose a Quad CFA-driven remosaicing model that effectively converts noisy Quad Bayer and standard Bayer patterns compatible to existing Image Signal Processor (ISP) without any loss in resolution. To enhance the practicality of the remosaicing model for real-world images affected by mixed noise, we introduce a novel dual-head joint remosaicing and denoising network (DJRD), which addresses the order of denoising and remosaicing by performing them in parallel. In DJRD, we customize two denoising branches for Quad Bayer and Bayer inputs. These branches model non-local and local dependencies, CFA location, and frequency information using residual convolutional layers, Swin Transformer, and wavelet transform-based CNN. Furthermore, to improve the model’s performance on challenging cases, we fine-tune DJRD to handle difficult scenarios by identifying problematic patches through Moire and zipper detection metrics. This post-training phase allows the model to focus on resolving complex image regions. Extensive experiments conducted on simulated and real images in both Bayer and sRGB domains demonstrate that DJRD outperforms competing models by approximately 3 dB, while maintaining the simplicity of implementation without adding any hardware. [ABSTRACT FROM AUTHOR]

Presents a possible explanation for the reduction of the block effects in the image decompression method proposed by Prost, et al. Details on the error in the theoretical derivations.

Dual cameras with visible-thermal multispectral pairs provide both visual and thermal appearance, thereby enabling detecting pedestrians around the clock in various conditions and applications, including autonomous driving and intelligent transportation systems. However, due to the greatly varying real-world scenarios, the performance of a detector trained on a source dataset might change dramatically when evaluated on another dataset. A large amount of training data is often necessary to guarantee the detection performance in a new scenario. Typically, human annotators need to conduct the data labeling work, which is time-consuming, labor-intensive and unscalable. To overcome the problem, we propose a novel unsupervised transfer learning framework for multispectral pedestrian detection, which adapts a multispectral pedestrian detector to the target domain based on pseudo training labels. In particular, auxiliary detectors are utilized and different label fusion strategies are introduced according to the estimated environmental illumination level. Intermediate domain images are generated by translating the source images to mimic the target ones, acting as a better starting point for the parameter update of the pedestrian detector. The experimental results on the KAIST and FLIR ADAS datasets demonstrate that the proposed method achieves new state-of-the-art performance without any manual training annotations on the target data. [ABSTRACT FROM AUTHOR]

Urban air quality mapping has been widely applied in urban planning, air pollution control and personal air pollution exposure assessment. Urban air quality maps are traditionally derived using measurements from fixed monitoring stations. Due to high cost, these stations are generally sparsely deployed in a few representative locations, leading to a highly generalized air quality map. In addition, urban air quality varies rapidly over short distances (<1 km) and is influenced by meteorological conditions, road network and traffic flow. These variations are not well represented in coarse-grained air quality maps generated by conventional fixed-site monitoring methods but have important implications for characterizing heterogeneous personal air pollution exposures and identifying localized air pollution hotspots. Therefore, fine-grained urban air quality mapping is indispensable. In this context, supplementary low-cost mobile sensors make mobile air quality monitoring a promising alternative. Using sparse air quality measurements collected by mobile sensors and various contextual factors, especially traffic flow, we propose a context-aware locally adapted deep forest (CLADF) model to infer the distribution of NO2 by 100 m and 1 h resolution for fine-grained air quality mapping. The CLADF model exploits deep forest to construct a local model for each cluster consisting of nearest neighbor measurements in contextual feature space, and considers traffic flow as an important contextual feature. Extensive validation experiments were conducted using mobile NO2 measurements collected by 17 postal vans equipped with low-cost sensors operating in Antwerp, Belgium. The experimental results demonstrate that the CLADF model achieves the lowest RMSE as well as advances in accuracy and correlation, compared with various benchmark models, including random forest, deep forest, extreme gradient boosting and support vector regression. [ABSTRACT FROM AUTHOR]

In laser powder bed fusion (LPBF), melt pool instability can lead to the development of pores in printed parts, reducing the part's structural strength. While camera-based monitoring systems have been introduced to improve melt pool stability, these systems only measure melt pool stability in limited, indirect ways. We propose that melt pool stability can be improved by explicitly encoding stability into LPBF monitoring systems through the use of temporal features and pore density modelling. We introduce the temporal features, in the form of temporal variances of common LPBF monitoring features (e.g., melt pool area, intensity), to explicitly quantify printing stability. Furthermore, we introduce a neural network model trained to link these video features directly to pore densities estimated from the CT scans of previously printed parts. This model aims to reduce the number of online printer interventions to only those that are required to avoid porosity. These contributions are then implemented in a full LPBF monitoring system and tested on prints using 316L stainless steel. Results showed that our explicit stability quantification improved the correlation between our predicted pore densities and true pore densities by up to 42%. [ABSTRACT FROM AUTHOR]

In this paper, we propose a unified and flexible framework for general image fusion tasks, including multi-exposure image fusion, multi-focus image fusion, infrared/visible image fusion, and multi-modality medical image fusion. Unlike other deep learning-based image fusion methods applied to a fixed number of input sources (normally two inputs), the proposed framework can simultaneously handle an arbitrary number of inputs. Specifically, we use the symmetrical function (e.g., Max-pooling) to extract the most significant features from all the input images, which are then fused with the respective features from each input source. This symmetry function enables permutation-invariance of the network, which means the network can successfully extract and fuse the saliency features of each image without needing to remember the input order of the inputs. The property of permutation-invariance also brings convenience for the network during inference with unfixed inputs. To handle multiple image fusion tasks with one unified framework, we adopt continual learning based on Elastic Weight Consolidation (EWC) for different fusion tasks. Subjective and objective experiments on several public datasets demonstrate that the proposed method outperforms state-of-the-art methods on multiple image fusion tasks. [ABSTRACT FROM AUTHOR]

Image segmentation is one of the most important tasks in medical image analysis and is often the first and the most critical step in many clinical applications. In brain MRI analysis, image segmentation is commonly used for measuring and visualizing the brain’s anatomical structures, for analyzing brain changes, for delineating pathological regions, and for surgical planning and image-guided interventions. In the last few decades, various segmentation techniques of different accuracy and degree of complexity have been developed and reported in the literature. In this paper we review the most popular methods commonly used for brain MRI segmentation. We highlight differences between them and discuss their capabilities, advantages, and limitations. To address the complexity and challenges of the brain MRI segmentation problem, we first introduce the basic concepts of image segmentation. Then, we explain different MRI preprocessing steps including image registration, bias field correction, and removal of nonbrain tissue. Finally, after reviewing different brain MRI segmentation methods, we discuss the validation problem in brain MRI segmentation. [ABSTRACT FROM AUTHOR]

Hand tracking in video is an increasingly popular research field due to the rise of novel human-computer interaction methods. However, robust and real-time hand tracking in unconstrained environments remains a challenging task due to the high number of degrees of freedom and the non-rigid character of the human hand. In this paper, we propose an unsupervised method to automatically learn the context in which a hand is embedded. This context includes the arm and any other object that coherently moves along with the hand. We introduce two novel methods to incorporate this context information into a probabilistic tracking framework, and introduce a simple yet effective solution to estimate the position of the arm. Finally, we show that our method greatly increases robustness against occlusion and cluttered background, without degrading tracking performance if no contextual information is available. The proposed real-time algorithm is shown to outperform the current state-of-the-art by evaluating it on three publicly available video datasets. Furthermore, a novel dataset is created and made publicly available for the research community. [ABSTRACT FROM AUTHOR]

Today, solar energy is taking an increasing share of the total energy mix. Unfortunately, many operational photovoltaic plants suffer from a plenitude of defects resulting in non-negligible power loss. The latter highly impacts the overall performance of the PV site; therefore, operators need to regularly inspect their solar parks for anomalies in order to prevent severe performance drops. As this operation is naturally labor-intensive and costly, we present in this paper a novel system for improved PV diagnostics using drone-based imagery. Our solution consists of three main steps. The first step locates the solar panels within the image. The second step detects the anomalies within the solar panels. The final step identifies the root cause of the anomaly. In this paper, we mainly focus on the second step comprising the detection of anomalies within solar panels, which is done using a region-based convolutional neural network (CNN). Experiments on six different PV sites with different specifications and a variety of defects demonstrate that our anomaly detector achieves a true positive rate or recall of more than 90% for a false positive rate of around 2% to 3% tested on a dataset containing nearly 9000 solar panels. Compared to the best state-of-the-art methods, the experiments revealed that we achieve a slightly higher true positive rate for a substantially lower false positive rate, while tested on a more realistic dataset. [ABSTRACT FROM AUTHOR]

Remote sensing using multisensor platforms has been systematically applied for monitoring and optimizing human activities. Several advanced techniques have been developed to enhance and extract the spatially and spectrally semantic information in the hyperspectral image (HSI) and light detection and ranging (LiDAR) data processing and analysis. However, an abundance of redundant information and sometimes a lack of discriminative features reduce the efficiency and effectiveness of multisource classification methods. This article proposes a fractional Gabor convolutional network (FGCN), focusing on efficient feature fusion and comprehensive feature extraction. First, the proposed FGCN uses Octave convolution layers to perform multisource information fusion and preserve discriminative information. Second, fractional Gabor convolutional (FGC) layers are proposed to extract multiscale, multidirectional, and semantic change features. The completeness and discrimination of the multisource features using different FGC kernels are improved, which yield robust feature extraction against semantic changes. Finally, the fractional Gabor feature and spectral feature are combined with two weighting factors which can be learned during the network training. Experimental results and comparisons with state-of-the-art multisource classification methods indicate the effectiveness of the proposed FGCN. With the FGCN, we can obtain an 89.90% overall accuracy on the challenging Muufl Gulfport (MUUFL) data set, with an improvement of 3% over state-of-the-art methods. [ABSTRACT FROM AUTHOR]

A new method for segmenting intensity images into smooth surface segments is presented. The main idea is to divide the image into flat, planar, convex, concave, and saddle patches that coincide as well as possible with meaningful object features in the image. Therefore, we propose an adaptive region growing algorithm based on low-degree polynomial fitting. The algorithm uses a new adaptive thresholding technique with the L8 fitting cost as a segmentation criterion. The polynomial degree and the fitting error are automatically adapted during the region growing process. The main contribution is that the algorithm detects outliers and edges, distinguishes between strong and smooth intensity transitions and finds surface segments that are bent in a certain way. As a result, the surface segments corresponding to meaningful object features and the contours separating the surface segments coincide with real-image object edges. Moreover, the curvature-based surface shape information facilitates many tasks in image analysis, such as object recognition performed on the polynomial representation. The polynomial representation provides good image approximation while preserving all the necessary details of the objects in the reconstructed images. The method outperforms existing techniques when segmenting images of objects with diffuse reflecting surfaces. [ABSTRACT FROM AUTHOR]

Abstract: Blur estimation is required in image processing techniques such as auto-focussing, quality assessment for compressed images and image fusion. In this paper a multi-scale local blur estimation method is proposed. We use the energy of a pair of quadrature filters with first and second derivatives of a Gaussian at several scales as its constituents. A new strategy for analyzing the extrema of energy across scale is proposed. Comparing to the methods which use just a Gaussian first derivative kernel, a smaller number of scales needs to be processed. Also our method yields only one response at the centroid of line-shape features at a position independent of the scale. This is in contrast to other methods which yield multiple responses at scale dependent positions. We evaluated the method for synthetic and real images from the LIVE database. Depending on details of the image, the proposed method is several to tens of times faster in comparison with using just a Gaussian first derivative. The accuracy of the blur estimation is found to be the best or second best while comparing with 16 different methods for Gaussian blur. In addition, the performance is still good for motion blurred images. [Copyright &y& Elsevier]

Abstract: In compressed sensing theory, decomposing a signal based upon redundant dictionaries is of considerable interest for data representation in signal processing. The signal is approximated by an over-complete dictionary instead of an orthonormal basis for adaptive sparse image decompositions. Existing sparsity-based super-resolution methods commonly train all atoms to construct only a single dictionary for super-resolution. However, this approach results in low precision of reconstruction. Furthermore, the process of generating such dictionary usually involves a huge computational cost. This paper proposes a sparse representation and position prior based face hallucination method for single face image super-resolution. The high- and low-resolution atoms for the first time are classified to form local dictionaries according to the different regions of human face, instead of generating a single global dictionary. Different local dictionaries are used to hallucinate the corresponding regions of face. The patches of the low-resolution face inputs are approximated respectively by a sparse linear combination of the atoms in the corresponding over-complete dictionaries. The sparse coefficients are then obtained to generate high-resolution data under the constraint of the position prior of face. Experimental results illustrate that the proposed method can hallucinate face images of higher quality with a lower computational cost compared to other existing methods. [Copyright &y& Elsevier]

Abstract: In this paper we propose a novel inference method for maximum a posteriori estimation with Markov random field prior. The central idea is to integrate a kind of joint “voting” of neighboring labels into a message passing scheme similar to loopy belief propagation (LBP). While the LBP operates with many pairwise interactions, we formulate “messages” sent from a neighborhood as a whole. Hence the name neighborhood-consensus message passing (NCMP). The practical algorithm is much simpler than LBP and combines the flexibility of iterated conditional modes (ICM) with some ideas of more general message passing. The proposed method is also a generalization of the iterated conditional expectations algorithm (ICE): we revisit ICE and redefine it in a message passing framework in a more general form. We also develop a simplified version of NCMP, called weighted iterated conditional modes (WICM), that is suitable for large neighborhoods. We verify the potentials of our methods on four different benchmarks, showing the improvement in quality and/or speed over related inference techniques. [Copyright &y& Elsevier]

We propose a new statistical model for image restoration in which neighborhoods of wavelet subbands are modeled a discrete mixture of linear projected Gaussian Scale Mixtures (MPGSM). In each projection, a lower dimensional approximation of the local neighborhood is obtained, thereby modeling the strongest correlations in that neighborhood. The model is generalization of the recently developed Mixture of GSM (MGSM) model, that offers a significant improvement both in PSNR and visually compared to the current state-of-the-art wavelet techniques. However, the computation cost is very high which hampers its use for practical purposes. We present a fast EM algorithm that takes advantage of the projection bases to speed up the algorithm. The results show that, when projecting on a fixed data-independent basis, even computational advantages with a limited loss PSNR can be obtained with respect to the BLS-GSM denoising method, while data-dependent bases of Principle Components offer a higher denoising performance, both visually and in PSNR compared to the current wavelet-based state-of-the-art denoising methods. [ABSTRACT FROM AUTHOR]

Images, captured with digital imaging devices, often contain noise. In literature, many algorithms exist for the removal of white uncorrelated noise, but they usually fail when applied to images with correlated noise. In this paper, we design a new de-noising method for the removal of correlated noise, by modeling the significance of the noise-free wavelet coefficients in a local window using a new significance measure that defines the "signal of interest" and that is applicable to correlated noise. We combine the intrascale model with a hidden Markov tree model to capture the interscale dependencies between the wavelet coefficients. We propose a denoising method based on the combined model and a less redundant wavelet transform. We present results that show that the new method performs as well as the state-of-the-art wavelet-based methods, while having a lower computational complexity. [ABSTRACT FROM AUTHOR]

In video communication over lossy packet networks (e.g., the Internet), packet loss errors can severely damage the transmitted video. The damaged video can largely be repaired with passive error concealment, where neighboring information is used to estimate missing information. We address the problem of passive error concealment for wavelet coded data with dispersive packetization. The reported techniques of this kind have many problems and usually fail in the reconstruction of high-frequency content. This paper presents a novel locally adaptive error concealment method for subband coded I-frames based on an inhomogeneous Gaussian Markov random field model. We estimate the parameters of this model from a local context of each lost coefficient, and we interpolate the lost coefficients accordingly. The results demonstrate a significant improvement over the reported related methods both in terms of objective performance measures and visually. The biggest improvement of the proposed method compared to the state-of-the-art in the field is the correct reconstruction of high-frequency information such as textures and edges. [ABSTRACT FROM AUTHOR]

In image communication over lossy packet networks (e.g., cell phone communication), packet loss errors lead to damaged images. Damaged images can be repaired with passive error concealment methods, which use neighboring coefficient or pixel values to estimate the missing ones. Neighboring image data should, thus, be spread over different packets. This paper presents a novel robust packetization method for the transmission of image content in lossy packet networks. We first define novel criteria for a good packetization. Based on these properties, we propose a cost function for packetization masks. We then use stochastic optimization to calculate optimal packetization masks. We test our packetization technique on both wavelet coding and DCT coding. Compared to other packetization techniques, we are able to achieve the same or better mean quality of the reconstructed images but with less fluctuation in quality, which is important for the viewer experience. In this way, we significantly increase the worst case quality, especially for high packet loss rates. This leads to visually more pleasing images in case of a passive reconstruction. [ABSTRACT FROM AUTHOR]

This paper proposes a novel video denoising method based on nondecimated wavelet band filtering. In the proposed method, motion estimation and adaptive recursive temporal filtering are performed in a closed loop, followed by an intra-frame spatially adaptive filter. All processing occurs in the wavelet domain. The paper introduces new wavelet-based motion reliability measures. We make a difference between motion reliability per orientation and reliability per wavelet band. These two reliability measures are employed in different stages of the proposed denoising scheme. The reliability per orientation (horizontal and vertical) measure is used in the proposed motion estimation scheme while the reliability of the estimated motion vectors (MVs) per wavelet band is utilized for subsequent adaptive temporal and spatial filtering. We propose a novel cost function for motion estimation which takes into account the spatial orientation of image structures and their motion matching values. Our motion estimation approach is a novel wavelet-domain three-step scheme, where the refinement of MVs in each step is determined based on the proposed motion reliabilities per orientation. The temporal filtering is performed separately in each wavelet band along the estimated motion trajectory and the parameters of the temporal filter depend on the motion reliabilities per wavelet band. The final spatial filtering step employs an adaptive smoothing of wavelet coefficients that yields a stronger filtering at the positions where the temporal filter was less effective. The results on various grayscale sequences demonstrate that the proposed filter outperforms several state-of-the-art filters visually (as judged by a small test panel) as well as in terms of peak signal-to-noise ratio. [ABSTRACT FROM AUTHOR]

Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named “structured sparse low-rank representation” (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation. [ABSTRACT FROM AUTHOR]

This paper presents an integrated method of the adaptive enhancement for an unsupervised global-to-local segmentation of brain tissues in three-dimensional (3-D) MRI (Magnetic Resonance Imaging) images. Three brain tissues are of interest: CSF (CerebroSpinal Fluid), GM (Gray Matter), WM (White Matter). Firstly, we de-noise the images using a newly proposed versatile wavelet-based filter, and segment the images with minimum error global thresholding. Subsequently, we combine a spatial-feature-based FCM (Fuzzy C-Means) clustering with 3-D clustering-result-weighted median and average filters, so as to further achieve a locally adaptive enhancement and segmentation. This integrated strategy yields a robust and accurate segmentation, particularly in noisy images. The performance of the proposed method is validated by four indices on MRI brain phantom images and on real MRI images. [Copyright &y& Elsevier]

Earth observation using multisensor data is drawing increasing attention. Fusing remotely sensed hyperspectral imagery and light detection and ranging (LiDAR) data helps to increase application performance. In this article, joint classification of hyperspectral imagery and LiDAR data is investigated using an effective hierarchical random walk network (HRWN). In the proposed HRWN, a dual-tunnel convolutional neural network (CNN) architecture is first developed to capture spectral and spatial features. A pixelwise affinity branch is proposed to capture the relationships between classes with different elevation information from LiDAR data and confirm the spatial contrast of classification. Then in the designed hierarchical random walk layer, the predicted distribution of dual-tunnel CNN serves as global prior while pixelwise affinity reflects the local similarity of pixel pairs, which enforce spatial consistency in the deeper layers of networks. Finally, a classification map is obtained by calculating the probability distribution. Experimental results validated with three real multisensor remote sensing data demonstrate that the proposed HRWN significantly outperforms other state-of-the-art methods. For example, the two branches CNN classifier achieves an accuracy of 88.91% on the University of Houston campus data set, while the proposed HRWN classifier obtains an accuracy of 93.61%, resulting in an improvement of approximately 5%. [ABSTRACT FROM AUTHOR]

The simultaneous acquisition of multi-spectral images on a single sensor can be efficiently performed by single shot capture using a mutli-spectral filter array. This paper focused on the demosaicing of color and near-infrared bands and relied on a convolutional neural network (CNN). To train the deep learning model robustly and accurately, it is necessary to provide enough training data, with sufficient variability. We focused on the design of an efficient training procedure by discovering an optimal training dataset. We propose two data selection strategies, motivated by slightly different concepts. The general term that will be used for the proposed models trained using data selection is data selection-based multi-spectral demosaicing (DSMD). The first idea is clustering-based data selection (DSMD-C), with the goal to discover a representative subset with a high variance so as to train a robust model. The second is an adaptive-based data selection (DSMD-A), a self-guided approach that selects new data based on the current model accuracy. We performed a controlled experimental evaluation of the proposed training strategies and the results show that a careful selection of data does benefit the speed and accuracy of training. We are still able to achieve high reconstruction accuracy with a lightweight model. [ABSTRACT FROM AUTHOR]