Search

Copyright of Chinese Journal of Intelligent Science & Technology is the property of Beijing Xintong Media Co., Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This paper proposes a cascade deep convolutional neural network to address the loosening detection problem of bolts on axlebox covers. Firstly, an SSD network based on ResNet50 and CBAM module by improving bolt image features is proposed for locating bolts on axlebox covers. And then, theA2-PFN is proposed according to the slender features of the marker lines for extracting more accurate marker lines regions of the bolts. Finally, a rectangular approximation method is proposed to regularize the marker line regions as away to calculate the angle of the marker line and plot all the angle values into an angle table, according to which the criteria of the angle table can determine whether the bolt with the marker line is in danger of loosening. Meanwhile, our improved algorithm is compared with the pre-improved algorithmin the object localization stage. The results show that our proposed method has a significant improvement in both detection accuracy and detection speed, where our mAP (IoU=0.75) reaches 0.77 and fps reaches 16.6. And in the saliency detection stage, after qualitative comparison and quantitative comparison, our method significantly outperforms other state-of-the-art methods, where our MAE reaches 0.092, F-measure reaches 0.948 and AUC reaches 0.943. Ultimately, according to the angle table, out of 676 bolt samples, a total of 60 bolts are loose, 69 bolts are at risk of loosening, and 547 bolts are tightened. [ABSTRACT FROM AUTHOR]

Due to ignoring the extraction of multi-scale features and the differences between features at different levels, salient object detection still has the problems of incomplete prediction structure and loss of details. To solve these problems, this paper proposes a new saliency detection model M3 Net. The network mainly consists of Multi-scale Features Adaptive Fusion Module (MAF) and Recurrent Feedback Aggregation Module (RFA). MAF can adaptively capture and aggregate multi-scale features at different levels. RFA can effectively prevent feature dilution while aggregating features at different levels in the iterative process. The experimental results on 5 benchmark datasets show that the network outperforms 10 existing networks in three evaluation metrics: Fβ, Em, and MAE. On the DUT-OMRON dataset, the Fβ is 0.4% higher than the second-ranked saliency detection model, and the Em is 0.3% higher. In ECSSD dataset, the Fβ is 0.2% higher than the second-ranked saliency detection model, and the Em is 0.3% higher, and the network also has excellent speed performance [ABSTRACT FROM AUTHOR]

"Picking out the impurities" is a typical scenario in production line which is both time consuming and laborious. In this article, we propose a target-oriented robotic push-grasping system which is able to actively discover and pick the impurities in dense environments with the synergies between pushing and grasping actions. First, we propose an attention module, which includes target saliency detection and density-based occluded-region inference. Without the necessity of expensive labeling of semantic segmentation, our attention module can quickly locate the targets in the view or predict the candidate regions where the targets are most likely to be occluded. Second, we propose a push–grasp synergy framework to sequentially select proper actions in different situations until all targets are picked out. Moreover, we introduce an active pushing mechanism based on a novel metric, namely Target-Centric Dispersion Degree (TCDD) for better grasping. TCDD describes whether the targets are isolated from the surrounding objects. With this metric, the robot becomes more focused on the actions around the targets and push irrelevant objects away. Experimental results on both simulated environment and real-world environment show that our proposed system outperforms several baseline approaches,which also has the capability to be generalized to new scenarios. [ABSTRACT FROM AUTHOR]

Given one specific image, it would be quite significant if humanity could simply retrieve all those pictures that fall into a similar category of images. However, traditional methods are inclined to achieve high-quality retrieval by utilizing adequate learning instances, ignoring the extraction of the image's essential information which leads to difficulty in the retrieval of similar category images just using one reference image. Aiming to solve this problem above, we proposed in this paper one refined sparse representation based similar category image retrieval model. On the one hand, saliency detection and multi-level decomposition could contribute to taking salient and spatial information into consideration more fully in the future. On the other hand, the cross mutual sparse coding model aims to extract the image's essential feature to the maximumextent possible. At last, we set up a database concluding a large number of multi-source images. Adequate groups of comparative experiments show that our method could contribute to retrieving similar category images effectively. Moreover, adequate groups of ablation experiments show that nearly all procedures play their roles, respectively. [ABSTRACT FROM AUTHOR]

Saliency detection as an active research direction in image understanding and analysis has been studied extensively. In this paper, to improve the accuracy of saliency detection, we propose an efficient unsupervised salient object detection method. The first step of our method is that we extract local low-level features of each superpixel after segmenting the image into different scale parts, which helps to locate the approximate locations of salient objects. Then, we use convolutional neural networks to extract high-level, semantically rich features as complementary features of each superpixel, and low-level features, as well as high-level features of each superpixel, are incorporated into a new feature vector to measure the distance between different superpixels. The last step is that we use a manifold space-ranking method to calculate the saliency of each superpixel. Extensive experiments over four challenging datasets indicate that the proposed method surpasses state-of-the-art methods and is closer to the ground truth. [ABSTRACT FROM AUTHOR]

Visual perception modelling of saliency detection has received widespread concerns recently from both the cybernetics and computational intelligence domains. In particular, those distinct background and foreground-oriented models are capable of engendering competitive results. The implicitly vital issue of the above computing approaches is how to reliably choose seeds of the foreground and background cues for kicking off the subsequent saliency-detection procedure. To address this barrier, this paper explores the local geometry and statistical attribute of the detected orientational blocks via an improved discrete wavelet frame transform algorithm to estimate the center position of individual salient object in the original input. Specially, the calculated centroid can be regarded as the prominent focus of visual perception in the initial image, which is beneficial to choose the credible seed during the computing of the superpixel-based foreground and background cues. Then, both sides of the complementary and visually oriented cues are integrated concurrently into a dependable and robust saliency map with reliability. Substantial experimental evaluations in term of freely open-access databases testify the effectiveness of the designed framework, and have prove that the designed algorithm is accurate and outperforms the other distinct representative saliency detection algorithms. [ABSTRACT FROM AUTHOR]

Current haze removal methods for unmanned aerial vehicle (UAV) images are mostly based on natural image dehazing methods, which ignore the particular imaging mechanism of UAVs. They often fail to restore regions with rich spectral and textural information. In this letter, we propose a saliency-guided parallel learning mechanism for UAV image haze removal. First, we design a saliency-guided parallel dehazing module with two parallel paths. The residual feature extraction path obtains deep-level features to realize global dehazing effectively. The key feature enhancement path, which comprises saliency dense blocks, realizes local textural preservation and spectral restoration. Second, a sporadic foreground saliency detection method is proposed for UAV images with sporadic objects. The saliency map guides the learning of significant spectral and textural information in hazy images. Finally, a multiscale reconstruction module is introduced to more accurately estimate small-scale textural details and large-scale spectral information. Experimental results show that the proposed method has better detail performance and visual effects than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Field of view (FoV) prediction is critical in 360-degree video multicast, which is a key component of the emerging virtual reality and augmented reality applications. Most of the current prediction methods combining saliency detection and FoV information neither take into account that the distortion of projected 360-degree videos can invalidate the weight sharing of traditional convolutional networks nor do they adequately consider the difficulty of obtaining complete multi-user FoV information, which degrades the prediction performance. This article proposes a spherical convolution-empowered FoV prediction method,which is a multi-source prediction framework combining salient features extracted from 360-degree video with limited FoV feedback information. A spherical convolutional neural network is used instead of a traditional two-dimensional convolutional neural network to eliminate the problem of weight sharing failure caused by video projection distortion. Specifically, salient spatial-temporal features are extracted through a spherical convolution-based saliency detection model, after which the limited feedback FoV information is represented as a time-series model based on a spherical convolution-empowered gated recurrent unit network. Finally, the extracted salient video features are combined to predict future user FoVs. The experimental results show that the performance of the proposed method is better than other prediction methods. [ABSTRACT FROM AUTHOR]

Accurate recognition of circular marks is crucial for calibration, object tracking, and three-dimensional reconstruction in videogrammetry. However, most existing studies were designed under single or relatively simple scenes. When the existing algorithms are applied to more complex scenarios, it will result in higher false detection and miss-detection rate. In this article, we present a high-precision recognition method based on a novel deep learning model, circular-MarkNet (CMNet) to solve this problem. The proposed network consists of three main steps: first, circular marks are detected using the improved YOLOv4 model to narrow the search region of the circular contour; the contour of the circular marks is then extracted based on the saliency object detection model BASNet; and finally, least square fitting is used to calculate the central pixel coordinate of the identified contour on the saliency map. The proposed method was tested under three complex scenarios with different characteristics and disturbances. The experimental results demonstrated that: the proposed CMNet can effectively recognize of circular marks within complex scenes, which reveals the superiority and generalization ability of the proposed method; the improved YOLOv4 can significantly enhance the detection accuracy of circular marks, which is crucial to the subsequent saliency courter detection and circle center identification; and CMNet achieved the best performance, with an RMSE of 0.0713 pixel, compared to the state-of-the-art methods.

Extracting salient regions plays an important role in computer vision tasks, e.g., object detection, recognition and video compression. Previous saliency detection study is mostly conducted on individual frames and tends to extract saliency with spatial cues. The development of various motion feature further extends the saliency concept to the motion saliency from videos. In contrast to image-based saliency extraction, video-based saliency extraction is more challenging due to the complicated distractors, e.g., the background dynamics and shadows. In this paper, we propose a novel saliency extraction method by fusing temporal and spatial cues. In specific, the long-term and short-term variations are comprehensively fused to extract the temporal cue, which is then utilized to establish the background guidance for generating the spatial cue. Herein, the long-term variations and spatial cues jointly highlight the contrast between objects and the background, which can solve the problem caused by shadows. The short-term variations contribute to the removal of background dynamics. Spatiotemporal cues are fully exploited to constrain the saliency extraction across frames. The saliency extraction performance of our method is demonstrated by comparing it to both unsupervised and supervised methods. Moreover, this novel saliency extraction model is applied in the video compression tasks, helping to accelerate the video compression task and achieve a larger PSNR value for the region of interest (ROI).

Saliency detection is a key research topic in the field of computer vision. Humans can be accurately and quickly mesmerized by an area of interest in complex and changing scenes through the visual perception area of the brain. Although existing saliency-detection methods can achieve competent performance, they have deficiencies such as unclear margins of salient objects and the interference of background information on the saliency map. In this study, to improve the defects during saliency detection, a multiscale cascaded attention network was designed based on ResNet34. Different from the typical U-shaped encoding–decoding architecture, we devised a contextual feature extraction module to enhance the advanced semantic feature extraction. Specifically, a multiscale cascade block (MCB) and a lightweight channel attention (CA) module were added between the encoding and decoding networks for optimization. To address the blur edge issue, which is neglected by many previous approaches, we adopted the edge thinning module to carry out a deeper edge-thinning process on the output layer image. The experimental results illustrate that this method can achieve competitive saliency-detection performance, and the accuracy and recall rate are improved compared with those of other representative methods. [ABSTRACT FROM AUTHOR]

Although deep learning-based techniques for salient object detection have considerably improved over recent years, estimated saliency maps still exhibit imprecise predictions owing to the internal complexity and indefinite boundaries of salient objects of varying sizes. Existing methods emphasize the design of an exemplary structure to integrate multi-level features by employing multi-scale features and attention modules to filter salient regions from cluttered scenarios. We propose a saliency detection network based on three novel contributions. First, we use a dense feature extraction unit (DFEU) by introducing large kernels of asymmetric and grouped-wise convolutions with channel reshuffling. The DFEU extracts semantically enriched features with large receptive fields and reduces the gridding problem and parameter sizes for subsequent operations. Second, we suggest a cross-feature integration unit (CFIU) that extracts semantically enriched features from their high resolutions using dense short connections and sub-samples the integrated information into different attentional branches based on the inputs received for each stage of the backbone. The embedded independent attentional branches can observe the importance of the sub-regions for a salient object. With the constraint-wise growth of the sub-attentional branches at various stages, the CFIU can efficiently avoid global and local feature dilution effects by extracting semantically enriched features via dense short-connections from high and low levels. Finally, a contour-aware saliency refinement unit (CSRU) was devised by blending the contour and contextual features in a progressive dense connected fashion to assist the model toward obtaining more accurate saliency maps with precise boundaries in complex and perplexing scenarios. Our proposed model was analyzed with ResNet-50 and VGG-16 and outperforms most contemporary techniques with fewer parameters. [ABSTRACT FROM AUTHOR]

In recent years, Convolutional Neural Network (CNN) has been gradually applied to Image Quality Assessment (IQA). Most CNNs segment the image into patches for training, which lead to increase of data and affect calculation speed of the model. Meanwhile, the parameters of CNN usually reach millions, which is the root cause of overfitting. In this paper, a multiscale CNN for NR-IQA is established to solve these problems. Since IQA simulates the perception of Human Visual System (HVS) on image quality, salient areas are more valuable for reference. Therefore a patch sampling method was designed based on saliency detection. Firstly, patches with salient values between given thresholds are retained as training data. Secondly, the sampled patches are fed into multiscale CNN. The network consists of three branches with multiscale convolutional kernels. Finally, the weighted average of the quality scores from the salient patches is the final score. The CNN was trained on LIVE dataset and cross-validated on CSIQ dataset. The experimental results show that the proposed method can achieve better performance with fewer parameters compared with state-of-the-art NR-IQA algorithms. [ABSTRACT FROM AUTHOR]

With the prevalence of thermal cameras, RGB-T multi-modal data have become more available for salient object detection (SOD) in complex scenes. Most RGB-T SOD works first individually extract RGB and thermal features from two separate encoders and directly integrate them, which pay less attention to the issue of defective modalities. However, such an indiscriminate feature extraction strategy may produce contaminated features and thus lead to poor SOD performance. To address this issue, we propose a novel CCFENet for a perspective to perform robust and accurate multi-modal expression encoding. First, we propose an essential cross-collaboration enhancement strategy (CCE), which concentrates on facilitating the interactions across the encoders and encouraging different modalities to complement each other during encoding. Such a cross-collaborative-encoder paradigm induces our network to collaboratively suppress the negative feature responses of defective modality data and effectively exploit modality-informative features. Moreover, as the network goes deeper, we embed several CCEs into the encoder, further enabling more representative and robust feature generation. Second, benefiting from the proposed robust encoding paradigm, a simple yet effective cross-scale cross-modal decoder (CCD) is designed to aggregate multi-level complementary multi-modal features, and thus encourages efficient and accurate RGB-T SOD. Extensive experiments reveal that our CCFENet outperforms the state-of-the-art models on three RGB-T datasets with a fast inference speed of 62 FPS. In addition, the advantages of our approach in complex scenarios (e.g., bad weather, motion blur, etc.) and RGB-D SOD further verify its robustness and generality. The source code will be publicly available via our project page: https://git.openi.org.cn/OpenVision/CCFENet. [ABSTRACT FROM AUTHOR]

RGB-D Salient Object Detection (RGB-D SOD) aims at detecting remarkable objects by complementary information from RGB images and depth cues. Although many outstanding prior arts have been proposed for RGB-D SOD, most of them focus on performance enhancement, while lacking concern about practical deployment on mobile devices. In this paper, we propose mobile asymmetric dual-stream networks (MoADNet) for real-time and lightweight RGB-D SOD. First, inspired by the intrinsic discrepancy between RGB and depth modalities, we observe that depth maps can be represented by fewer channels than RGB images. Thus, we design asymmetric dual-stream encoders based on MobileNetV3. Second, we develop an inverted bottleneck cross-modality fusion (IBCMF) module to fuse multimodality features, which adopts an inverted bottleneck structure to compensate for the information loss in the lightweight backbones. Third, we present an adaptive atrous spatial pyramid (A2SP) module to speed up the inference, while maintaining the performance by appropriately selecting multiscale features in the decoder. Extensive experiments are conducted to compare our method with 15 state-of-the-art approaches. Our MoADNet obtains competitive results on five benchmark datasets under four evaluation metrics. For efficiency analysis, the proposed method significantly outperforms other baselines by a large margin. The MoADNet only contains 5.03 M parameters and runs 80 FPS when testing a $256\times 256$ image on a single NVIDIA 2080Ti GPU. [ABSTRACT FROM AUTHOR]

Taking advantage of the functional complementarity between infrared and visible light sensors imaging, pixel-level real-time image fusion based on infrared and visible light images of different resolutions is a promising strategy for visual enhancement, which has demonstrated tremendous potential for autonomous driving, military reconnaissance, video surveillance, etc. Great progress has been made in this field in recent years, but the fusion speed and quality of visual enhancement are still not satisfactory. Herein, we propose a multi-scale FPGA-based image fusion technology with substantially enhanced visual enhancement capability and fusion speed. Specifically, the source images are first decomposed into three distinct layers using guided filter and saliency detection, which are the detail layer, saliency layer and background layer. Fusion weight map of the saliency layer is subsequently constructed using attention mechanism. Afterwards weight fusion strategy is used for saliency layer fusion and detail layer fusion, while weight average fusion strategy is used for the background layer fusion, followed by the incorporation of image enhancement technology to improve the fused image contrast. Finally, high-level synthesis tool is used to design the hardware circuit. The method in the present study is thoroughly tested on XCZU15EG board, which could not only effectively improve the image enhancement capability in glare and smoke environments, but also achieve fast real-time image fusion with 55FPS for infrared and visible images with a resolution of 640 × 470. [ABSTRACT FROM AUTHOR]

Conventional salient object detection models cannot differentiate the importance of different salient objects. Recently, two works have been proposed to detect saliency ranking by assigning different degrees of saliency to different objects. However, one of these models cannot differentiate object instances and the other focuses more on sequential attention shift order inference. In this paper, we investigate a practical problem setting that requires simultaneously segment salient instances and infer their relative saliency rank order. We present a novel unified model as the first end-to-end solution, where an improved Mask R-CNN is first used to segment salient instances and a saliency ranking branch is then added to infer the relative saliency. For relative saliency ranking, we build a new graph reasoning module by combining four graphs to incorporate the instance interaction relation, local contrast, global contrast, and a high-level semantic prior, respectively. A novel loss function is also proposed to effectively train the saliency ranking branch. Besides, a new dataset and an evaluation metric are proposed for this task, aiming at pushing forward this field of research. Finally, experimental results demonstrate that our proposed model is more effective than previous methods. We also show an example of its practical usage on adaptive image retargeting. [ABSTRACT FROM AUTHOR]

Saliency detection is one of the challenging problems still tackled by image processing and computer vision research communities. Although not very numerous, recent studies reveal that optimization-based methods provide relatively accurate and fast solutions for such problems. This paper presents a novel unsupervised hybrid optimization method that aims to propose reasonable solution to saliency detection problem by combining the familiar artificial bee colony and firefly algorithms. The proposed method, HABCFA, is based on creating hybrid-personality individuals behaving like both bees and fireflies. A superpixel-based method is used to obtain better background intensity values in the saliency detection process, providing a better precision in extracting the salient regions. HABCFA algorithm is capable of achieving an optimum saliency map without requiring any extra mask or training step. HABCFA has produced superior performance against its basis algorithms, artificial bee colony, and firefly on four known benchmark problems regarding convergence rate and iteration count. On the other hand, the experimental results on four commonly used datasets, including MSRA-1000, ECSSD, ICOSEG, and DUTOMRON, demonstrate that HABCFA is adequately robust and effective in terms of accuracy, precision, and speed in comparison with the eleven state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Copyright of Aero Weaponry is the property of Aero Weaponry Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

• Saliency-Augmented Module is proposed for clean data, reflecting latent categories. • A new self-supervised learning network SiSL-Net is developed. • 4.35% performance improvement on classification in self-supervised learning. The emergent studies and success on contrastive self-supervised learning have been well verified in the pretext task of instance discrimination, which learns visual representations by maximizing agreement between different augmented views of the same image sample (positive pairs). However, randomly cropping on original images may cause that the augmented view contains interference from a large proportion of the backgrounds, referred to as noisy data. Aiming to optimize the data augmentation and improve positive pairs, a Saliency-Augmented Module is proposed to obtain the augmented views, which only contain the "latent" object area, referred to as clean data. Furthermore, a Saliency-Guided Self-Supervised Learning Network (SiSL-Net) is constructed as a new pattern of contrastive learning. A symmetric structure of trunk net and branch net is trained to learn a feature mapping from the clean data space and the noisy data space. Besides, a novel loss function is designed, including the embedding contrastive loss and distribution consistency loss, to optimize the feature representations during network training. The linear classification performance of our SiSL-Net is evaluated on the mini ImageNet dataset with ResNet-50. Experiments show that our method achieves the top-1 accuracy from 64.67% to 69.02%, outperforming the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Modeling 3D visual saliency has received great attention due to the development of emerging 3D display technologies. Traditional methods relying on low-level features may not be efficient in interpreting 3D visual content from high-level semantic perspective. Despite numerous efforts dedicated to this area, existing 3D visual saliency detection methods do not necessarily excel in exploring the stereoscopic image saliency driven by the intra-view and inter-view dependencies among left and right views. In this paper, we propose a visual saliency detection method for stereoscopic images grounded on adaptive viewpoint feature enhancement via binocular vision. More specifically, the correlation among left and right views is investigated through a delicately designed binocular stereoscopic saliency feature aggregation module, enabling the generation of more representative saliency features towards binocular vision. Subsequently, to further aggregate the saliency features in multiple scales, we design a progressive attention-based saliency feature pyramid extraction module to effectively integrate the features from top-level to down-level based on the network hierarchy mechanism. The saliency maps are ultimately produced for stereoscopic images by evaluating the obtained saliency features. In addition, we create a stereoscopic image saliency dataset (SIS-3D) that includes 1086 stereoscopic image pairs with various content and their corresponding human eye fixation annotations, aiming to further facilitate the research on visual saliency detection for stereoscopic images. Extensive experiments demonstrate that our proposed method improves CC by an average of 4.02% compared to representative counterparts on the newly built saliency dataset and another publicly available dataset. [ABSTRACT FROM AUTHOR]

• Boosting few-shot learning (FSL) via saliency guidance provided only in training. • A unified framework for improved feature learning and classification head building. • Generating more accurate spatial attention maps compared to saliency predictions. • SGCA outperforms other saliency-based FSL works by a significant margin. • Achieving SOTA performance on multiple few-shot recognition datasets and scenarios. Despite significant progress in recent deep neural networks, most deep learning algorithms rely heavily on abundant training samples. To address the issue, few-shot learning (FSL) methods are designed to learn models that can generalize to novel classes with limited training data. In this work, we propose an effective and interpretable FSL approach termed Saliency-Guided Complementary Attention (SGCA). Concretely, SGCA aims to boost few-shot visual recognition from two perspectives: learning generalizable feature representations and building a robust classification module in a unified framework. For generalizable representation learning, we propose to explore the intrinsic structure of natural images by training the feature extractor with an auxiliary task to segment foreground regions from background clutter. The guidance signals are provided during training by a saliency detector which highlights object regions in images corresponding to the human visual system. Moreover, for robust classification module building, we introduce a complementary attention mechanism based on the learned segmentation to make the classification module focus on various informative parts of the image. Extensive experiments on 5 popular FSL datasets demonstrate that SGCA can outperform state-of-the-art approaches by a significant margin. In addition, extensions of SGCA to other challenging scenarios, including generalized, transductive and semi-supervised FSL, also verify the effectiveness and flexibility of our proposed approach. [ABSTRACT FROM AUTHOR]

In the field of saliency detection, salient instance segmentation is a novel challenging task that has received widespread attention. Due to the limited scale of the available dataset and the high cost of mask annotations, a substantial quantity of supervision sources is urgently required to train a high-performing salient instance model. To this end, we aim to train a novel salient instance segmentation model by weak supervisions that make full use of the existing salient object detection dataset. In this paper, we present a cyclic global context salient instance segmentation network (CGCNet) supervised by the combination of salient regions and bounding boxes from ready-made salient object detection datasets. To locate salient instances more accurately, a global feature refining layer is designed to expand the size of the features from the region of interest (ROI) to the global field in a scene. Moreover, a labeling updating scheme is embedded in the proposed framework to iteratively update the weak labels. Extensive experimental results demonstrate that our CGCNet trained by weak labels is competitive with the existing fully-supervised state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Spherical images or videos, as typical non-euclidean data, are usually stored in the form of 2D panoramas obtained through an equirectangular projection, which is neither equal area nor conformal. The distortion caused by the projection limits the performance of vanilla Deep Neural Networks (DNNs) designed for traditional euclidean data. In this paper, we design a novel Spherical Deep Neural Network (DNN) to deal with the distortion caused by the equirectangular projection. Specifically, we customize a set of components, including a spherical convolution, a spherical pooling, a spherical ConvLSTM cell and a spherical MSE loss, as the replacements of their counterparts in vanilla DNNs for spherical data. The core idea is to change the identical behavior of the conventional operations in vanilla DNNs across different feature patches so that they will be adjusted to the distortion caused by the variance of sampling rate among different feature patches. We demonstrate the effectiveness of our Spherical DNNs for saliency detection and gaze estimation in $360^\circ$ 360 ∘ videos. For saliency detection, we take the temporal coherence of an observer’s viewing process into consideration and propose to use a Spherical U-Net and a Spherical ConvLSTM to predict the saliency maps for each frame sequentially. As for gaze prediction, we propose to leverage a Spherical Encoder Module to extract spatial panoramic features, then we combine them with the gaze trajectory feature extracted by an LSTM for future gaze prediction. To facilitate the study of the $360^\circ$ 360 ∘ videos saliency detection, we further construct a large-scale $360^\circ$ 360 ∘ video saliency detection dataset that consists of 104 $360^\circ$ 360 ∘ videos viewed by 20+ human subjects. Comprehensive experiments validate the effectiveness of our proposed Spherical DNNs for 360 $^\circ$ ∘ handwritten digit classification and sport classification, saliency detection and gaze tracking in $360^\circ$ 360 ∘ videos. We also visualize the regions contributing to the classification decisions in our proposed Spherical DNNs via the Grad-CAM technique in the classification task, and the results show that our Spherical DNNs constantly leverage reasonable and important regions for decision making, regardless the large distortions. All codes and dataset are available on https://github.com/svip-lab/SphericalDNNs. [ABSTRACT FROM AUTHOR]

Computer vision researchers and decision-makers have struggled to understand how deep neural networks (DNNs) accomplish image classification tasks and interpret their results. Due to a lack of understanding of their internal functioning, these models are commonly referred to as "black boxes." As a part of the development process, the DNNs can be easily explained. This research work introduces an explainable technique for shoulder abnormality detection. The motivation behind this study is to enhance patients' and medical professionals' trust in DNNs technology. DNNs are implemented frequently in the medical domain. The suggested abnormality detector, based on Grad-CAM++, can detect Shoulder X-Rays abnormality. Grad-CAM++ is a common approach that combines the activation maps received from the model to create such a visualization. The average gradient-based terms used in this technique, on the other hand, understate the contribution of the model's identified representations to its predictions. We offer a technique that uses Grad-CAM++ to compute the route integral of the gradient-based terms to address this issue. By assessing different techniques, it is discovered that the recommended procedure can perform very effectively and efficiently in X-ray images provides more visual explanation than existing techniques. [ABSTRACT FROM AUTHOR]

Abstract Researched in the 1980s, multi-sensor data convergence has become a hot issue. Not only does it differ from general signal processing, or single to multiple sensor surveillance and measurement, on the other hand, it is a higher level of integrated decision-making processes based on multiple sensor measurement outcomes, this paper is based on the study of the saliency area detection algorithm of electronic information and image processing based on multi-sensor data fusion, based on the improved FT algorithm and LC algorithm using multi-sensor data fusion technology, a new LVA algorithm is proposed, and these three algorithms are evaluated in an all-round way through various algorithm evaluation indicators such as PR curve, PRF histogram, MAE index, and recognition image rate. The research results show that the LVA algorithm proposed in this paper improves the detection rate of saliency maps by 5–10%.

Salient object detection (SOD) enables machines to recognize and accurately segment visually prominent regions in images. Despite recent advancements, existing approaches often lack progressive fusion of low and high-level features, effective multi-scale feature handling, and precise boundary detection. Moreover, the robustness of these models under varied lighting conditions remains a concern. To overcome these challenges, we present Attention Enhanced Machine Instinctive Vision framework for SOD. The proposed framework leverages the strategy of Multi-stage Feature Refinement with Optimal Attentions-Driven Framework (MFRNet). The multi-level features are extracted from six stages of the EfficientNet-B7 backbone. This provides effective feature fusions of low and high-level details across various scales at the later stage of the framework. We introduce the Spatial-optimized Feature Attention (SOFA) module, which refines spatial features from three initial-stage feature maps. The extracted multi-scale features from the backbone are passed from the convolution feature transformation and spatial attention mechanisms to refine the low-level information. The SOFA module concatenates and upsamples these refined features, producing a comprehensive spatial representation of various levels. Moreover, the proposed Context-Aware Channel Refinement (CACR) module integrates dilated convolutions with optimized dilation rates followed by channel attention to capture multi-scale contextual information from the mature three layers. Furthermore, our progressive feature fusion strategy combines high-level semantic information and low-level spatial details through multiple residual connections, ensuring robust feature representation and effective gradient backpropagation. To enhance robustness, we train our network with augmented data featuring low and high brightness adjustments, improving its ability to handle diverse lighting conditions. Extensive experiments on four benchmark datasets — ECSSD, HKU-IS, DUTS, and PASCAL-S — validate the proposed framework's effectiveness, demonstrating superior performance compared to existing SOTA methods in the domain. Code, qualitative results, and trained weights will be available at the link: https://github.com/habib1402/MFRNet-SOD. • Multi-scale feature extraction enhances SOD accuracy using EfficientNet-B7 backbone. • SOFA module refines spatial features, boosting saliency map precision and sharpness. • CACR block captures multi-scale context with dilated convolutions and channel attention. • Progressive feature fusion and augmentation avoid feature loss and improve robustness under varied conditions. [ABSTRACT FROM AUTHOR]

In real-world datasets, the widespread presence of a long-tailed distribution often leads models to become overly biased towards majority class samples while ignoring minority class samples. We propose a strategy called MASW (MinoritySalMix and adaptive semantic weight compensation) to improve this problem. First, we propose a data augmentation method called MinoritySalMix (minority-saliency-mixing), which uses significance detection techniques to select significant regions from minority class samples as cropping regions and paste them into the same regions of majority class samples to generate brand new samples, thereby amplifying images containing important regions of minority class samples. Second, in order to make the label value information of the newly generated samples more consistent with the image content of the newly generated samples, we propose an adaptive semantic compensation factor. This factor provides more label value compensation for minority samples based on the different cropping areas, thereby making the new label values closer to the content of the newly generated samples. Improve model performance by generating more accurate new label value information. Finally, considering that some current re-sampling strategies generally lack flexibility in handling class sampling weight allocation and frequently require manual adjustment. We designed an adaptive weight function and incorporated it into the re-sampling strategy to achieve better sampling. The experimental results on three long-tailed datasets show that our method can effectively improve the performance of the model and is superior to most advanced long-tailed methods. Furthermore, we extended MinoritySalMix's strategy to three balanced datasets for experimentation, and the results indicated that our method surpassed several advanced data augmentation techniques. • MinoritySalMix fully utilizes the background content in the majority class samples. • Adaptively adjust the label value composition of two categories in the new sample. • The MASW model can significantly improve the performance of baseline methods. [ABSTRACT FROM AUTHOR]

Equipment maintenance support is an important technical measure to maintain the equipment's expected performance. However, the current maintenance supports are mainly completed by maintainers under the guidance of technical manual or additional experts, which may be insufficient for some advanced equipment with rapid update rate and complex inner structure. The rising technology of augmented reality (AR) provides a new solution for equipment maintenance support, while one of the key issues limiting the practical application of AR in maintenance field is the spatial matching issue between virtual space and reality space. In this paper, a virtual-reality spatial matching algorithm is designed to accurately superimpose the virtual information to the corresponding actual scene on the AR glasses. In this algorithm, two methods are proposed to help achieve the stable matching of virtual space and reality space. In detail, to obtain the saliency map with less background interference and improved saliency detection accuracy, a saliency detection method is designed based on the super-pixel segmentation. To deal with the problems of uneven distribution on the feature points and weak robustness to the light changes, a feature extraction and matching method is proposed for acquiring the feature point matching set with the utilization of the obtained saliency map. Finally, an immersive equipment maintenance support system (IEMSS) is developed based on this spatial matching algorithm, which provides the maintainers with immediate and immersive guidance to improve the efficiency and safety in the maintenance task, as well as offers maintenance training for inexperienced maintainers with expanded virtual information in case of limited experts. Several comparative experiments are implemented to verify the effectiveness of proposed methods, and a user study of real system application is carried out to further evaluate the superiority of these methods when applied in the IEMSS. • A saliency detection method is proposed to obtain the quality-improved saliency map. • A feature extraction and matching method is developed to enhance the robustness to various environments. • A virtual-reality spatial matching algorithm is proposed for visualized information superposition. • An immersive equipment maintenance support system is developed for maintenance task. • A user study is carried out to evaluate the proposed system. [ABSTRACT FROM AUTHOR]

In the salient object detection (SOD) models based on convolutional neural network (CNN), the high-level semantic features and low-level features of the image are effectively fused and complementary, which can effectively improve the performance of SOD. However, there are usually great differences between high-level semantic features and low-level features, and low-level features are often rich in noise. How to make full use of different features and avoid noise interference is a hot issue for researchers. Different from the traditional methods, this paper proposes a novel feature extraction and fusion network (EFNet). By setting a middle-level feature extraction module as the medium for the fusion of high-level semantic features and low-level image features, this special module integrates the two by reducing the difference between low-level image features and deep semantic features; In addition, a feature enhancement module is applied to enhance the image features, and the proposed SOD method can obtain good performance. Experimental results on five benchmark datasets show that the proposed method outperforms 15 state-of-the-art methods on five important evaluation metrics. Code will be available at: https://github.com/dc3234/EFNet. [ABSTRACT FROM AUTHOR]

In meaningful secret image sharing (MSIS), a secret image is divided into n shadows. Each shadow is meaningful and similar to the corresponding cover image. Meaningful shadows can reduce the suspicion of attackers in transmission and facilitate shadow management. Previous MSIS schemes always include pixel expansion, and cross-interference from different shadows may exist when cover images are extremely unnatural images with large black and white blocks. In this article, we propose an MSIS with uniform image quality. A threshold t is set to determine the absolute salient regions. More identical bits are allocated according to saliency values in the absolute saliency region, which can improve image quality. In addition, the new identical bits allocation strategy also adjusts the randomness of the shadow images, generating shadows with uniform image quality and avoiding the cross-interference between different shadows. Experimental results show the effectiveness of our proposed scheme. [ABSTRACT FROM AUTHOR]

Due to the rapid development of deep learning, the performance of salient object detection has been constantly refreshed. Nevertheless, it is still challenging for existing methods to distinguish the location of salient objects and retain fine structural details. In this paper, a novel progressive dual-attention residual network (PDRNet) is proposed to exploit two complementary attention maps to guide residual learning, thus progressively refining prediction in a coarse-to-fine manner. We design a dual-attention residual module (DRM) to achieve residual refinement with the help of the dual attention (DA) scheme. Specifically, an attention map and its corresponding reverse attention map are used to make the network be aware of learning residual details from the perspective of the salient and non-salient regions, thus utilizing their complementarity to correct the mistakes of object parts and boundary details. Besides, a hierarchical feature screening module (HFSM) is designed to capture more powerful global contextual knowledge for locating salient objects. It establishes cross-scale skip connections among multi-scale features and utilizes the intra-channel dependency of these scales to enhance information interaction and feature representation. Extensive experiments have proved that our proposed PDRNet performs favorably against 18 state-of-the-art competitors on five benchmark datasets, demonstrating the effectiveness and superiority of our method. [ABSTRACT FROM AUTHOR]

We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models treat this task as a point estimation problem by predicting a single saliency map following a deterministic learning pipeline. We argue that, however, the deterministic solution is relatively ill-posed. Inspired by the saliency data labeling process, we propose a generative architecture to achieve probabilistic RGB-D saliency detection which utilizes a latent variable to model the labeling variations. Our framework includes two main models: 1) a generator model, which maps the input image and latent variable to stochastic saliency prediction, and 2) an inference model, which gradually updates the latent variable by sampling it from the true or approximate posterior distribution. The generator model is an encoder-decoder saliency network. To infer the latent variable, we introduce two different solutions: i) a Conditional Variational Auto-encoder with an extra encoder to approximate the posterior distribution of the latent variable; and ii) an Alternating Back-Propagation technique, which directly samples the latent variable from the true posterior distribution. Qualitative and quantitative results on six challenging RGB-D benchmark datasets show our approach's superior performance in learning the distribution of saliency maps. The source code is publicly available via our project page: https://github.com/JingZhang617/UCNet. [ABSTRACT FROM AUTHOR]