Search

In recent years, most video segmentation methods use deep CNN to process the input image, but they did not fully mine the rich intermediate predictions in spatio‐temporal space. And, the segmentation challenges such as occlusion, severe deformation and illumination have not been well solved so far. To alleviate these problems, this paper focuses on constructing multi module network structures that represent multi semantics and proposes a video object segmentation network via coupled‐stream architecture with feature memory mechanism. This network first extracts high‐level semantic features, edge features, long‐term and short‐term stable depth features of the target, and then decode them into the segmentation mask of target. In addition, negative skeleton inhibition and frame interpolation are used to prevent the interference of similar objects and motion blur, respectively. The method has a low GPU memory usage, regardless of the number of object in video. And performs 86.5%and 62.4% in J&F measure on DAVIS 2016 and DAVIS 2017 validation set, without fine‐tuning and online training. [ABSTRACT FROM AUTHOR]

In the field of scientific visualization, the upscaling of time‐varying volume is meaningful. It can be used in in situ visualization to help scientists overcome the limitations of I/O speed and storage capacity when analysing and visualizing large‐scale, time‐varying simulation data. This paper proposes self‐attention residual network‐based spatial super‐resolution (SARN‐SSR), a spatial super‐resolution model based on self‐attention residual networks that can generate time‐varying data with temporal coherence. SARN‐SSR consists of two components: a generator and a discriminator. The generator takes the low‐resolution volume sequences as the input and gives the corresponding high‐resolution volume sequences as the output. The discriminator takes both synthesized and real high‐resolution volume sequence as the input and gives a matrix to predict the realness as the output. To verify the validity of SARN‐SSR, four sets of time‐varying volume datasets are applied from scientific simulation. In addition, SARN‐SSR is compared on these datasets, both qualitatively and quantitatively, with two deep learning‐based techniques and one traditional technique. The experimental results show that by using this method, the closest time‐varying data to the ground truth can be obtained. [ABSTRACT FROM AUTHOR]

Most existing multi‐view stereo (MVS) methods fail to consider global context information in the stage of feature extraction and cost aggregation. As transformers have shown remarkable performance on various vision tasks due to their ability to perceive global contextual information, this paper proposes a transformer‐based feature enhancement network (TF‐MVSNet) to facilitate feature representation learning by combining local features (both 2D and 3D) with long‐range contextual information. To reduce memory consumption of feature matching, the cross‐attention mechanism is leveraged to efficiently construct 3D cost volumes under the epipolar constraint. Additionally, a colour‐guided network is designed to refine depth maps at a coarse stage, hence reducing incorrect depth predictions at a fine stage. Extensive experiments were performed on the DTU dataset and Tanks and Temples (T&T) benchmark and results are reported. [ABSTRACT FROM AUTHOR]

In this paper, a method of stroke‐based rendering is proposed for image representation and reconstruction. The proposed method involves compositing a set of ellipses that greatly vary in location, size, rotation angle, color, and opacity. This study proves that alpha compositing is differentiable if a two‐dimensional Gaussian function is used to draw a solid ellipse. The gradient method can then be employed for automatically identifying the parameters of each ellipse such that the difference between the input image and the composited image is minimized. Experimental results indicate that the proposed method can represent various types of images including portraits, landscapes, buildings, street scenes, artificial objects, medical images etc. The proposed method can particularly render the most details and significant features of an input image with fewer strokes compared to other stroke‐based rendering algorithms. This study also demonstrates that the proposed method can be applied in painting style simulation and sparse‐view computed tomography imaging. [ABSTRACT FROM AUTHOR]

The limited of texture details information in low‐resolution facial or eye images presents a challenge for gaze estimation. To address this, FSKT‐GE (feature maps similarity knowledge transfer for low‐resolution gaze estimation) is proposed, a gaze estimation framework consisting of both a high resolution (HR) network and low resolution (LR) network with the identical structure. Rather than mere feature imitation, this issue is addressed by assessing the cosine similarity of feature layers, emphasizing the distribution similarity between the HR and LR networks. This enables the LR network to acquire richer knowledge. This framework utilizes a combination loss function, incorporating cosine similarity measurement, soft loss based on probability distribution difference and gaze direction output, along with a hard loss from the LR network output layer. This approach on low‐resolution datasets derived from Gaze360 and RT‐Gene datasets is validated, demonstrating excellent performance in low‐resolution gaze estimation. Evaluations on low‐resolution images obtained through 2×, 4×, and 8× down‐sampling are conducted on two datasets. On the Gaze360 dataset, the lowest mean angular errors of 10.97°, 11.22°, and 13.61° were achieved, while on the RT‐Gene dataset, the lowest mean angular errors of 6.73°, 6.83°, and 7.75° were obtained. [ABSTRACT FROM AUTHOR]

The denoising of images is an important research direction in computer vision. We consider the image denoising task as an estimation problem of the filtering policy related to image features, which is different from end‐to‐end image mapping. Commonly used simple filters such as gaussian filtering and bilateral filtering have fixed global denoising policies. However, the denoising policies of different filters can only adapt to limited image features. To solve this problem, we propose a method that applies different filters to different spatial ranges and adjusts the parameters of these filters simultaneously. Since not all filters can be easily transformed into differentiable forms and it is difficult to obtain paired datasets of filter action areas, we use reinforcement learning (RL) methods to estimate the spatial domain action range and adjustable parameters of filters, respectively. Furthermore, for removing higher intensity noise, simple filters can iteratively approximate higher‐order denoising policies and obtain more accurate and stable denoising results with the increase of iteration steps. Experimental results show that our proposed method can not only generate intuitive and interpretable denoising policies but also achieve comparable or better visual effects and computational efficiency than baseline methods. [ABSTRACT FROM AUTHOR]

Screen content video (SCV) consists primarily of text areas, computer graphics and other computer‐generated content and possesses unique perceptual characteristics. To compress SCVs more effectively with less reduction in subjective quality, perceptual characteristics of SCVs are analyzed and a perceptual redundancy (PR) model for SCV compression is proposed, including spatial PR (SPR), temporal PR (TPR) and foveated PR (FPR) model. In SPR modeling, the SCV is divided into sharp edge (SE) areas and non‐SE areas, then SPR is estimated separately. In TPR modeling, both inter‐frame luminance adaptation effect and motion masking effect are taken into account. In FPR modeling, each frame of SCV is classified into abrupt frames, relative motion frames or static frames. Then fixation points of different kinds of frames are predicted using different methods, and FPR is modeled considering foveated masking effect and visual attention. Finally, the perceptual redundancy of SCV is estimated based on the product of SPR, TPR and FPR. It is experimentally demonstrated that compared to the state‐of‐the‐art models, the authors' model could obtain more accurate estimates of PR. Moreover, the model is incorporated into SCV compression with an adaptive perceptual quantizer. An average of 7.42% bits could be saved with less decline in subjective quality. [ABSTRACT FROM AUTHOR]

This study investigates local orientation‐based approaches to the complex problem of pattern segmentation in three‐dimensional (3D) texture image. The current problem focuses on the extraction of so‐called lamellar colonies in titanium alloy, which, from the materials science and engineering point of view, are microstructural features that play a fundamental role on crack propagation and bifurcation during mechanical loading. Methods based on local orientation estimation extend the notion of using local gradient to reveal variation of semi‐planar pattern orientation in the 3D image. The study introduces a computational approach that accelerates the calculation of the eigenvectors from the local matrices of inertia of all voxels composing the 3D image. Then different paths are proposed to segment colonies or inter‐colony boundaries, i.e. polar orientation map and minimum scalar product map, in order to delimitate regions of similar orientations. The investigated segmentation methods have been compared with other methods that are mainly based on the popular solution of filter banks. Tests, which have been performed on both synthetic and real 3D images, show that the proposed local orientation‐based methods better delineate object boundaries than the counterparts. [ABSTRACT FROM AUTHOR]

Preliminary diagnosis based on ultrasound scanning is the first step in the treatment of many abdominal diseases. The noisy nature of the ultrasound image coupled with minimal contrasting features complicates the task of automatic classification if not impossible. This study presents a segmentation‐based approach to automatic classification of ten types of diffused and focal liver diseases from ultrasound images. A novel approach using Isocontour Segmentation based on Marching Squares, a computer graphics algorithm is presented. GLCM and fractal features are extracted from the segmented ultrasound images and classified using support vector machines and artificial neural networks (ANN) and the results are analysed. An overall classification accuracy of 92% is achieved using fractal features and ANN. [ABSTRACT FROM AUTHOR]

In this study, the authors propose an adaptive scheme to improve motion estimation of a variational model based on image features and flow quality measurements. Using image features, the authors introduce adaptive functions and inject them into the energy function to fine‐tune the estimation process. They propose a hybrid scheme to deal with large motions and improve the accuracy of the flow field. They introduce a trusted‐map based on constraints to measure flow quality. They use this map as a reference for the proposed occlusion‐aware filter. The proposed filter and hybrid scheme are integrated to correct the flow field iteratively, thus significantly improving the estimation results. The filter also enhances the flow field in occlusion areas. The authors experimental results demonstrate that their method provides sharp flow fields and significantly improved estimation accuracy. [ABSTRACT FROM AUTHOR]