Search

In daily life, some conventional image processing operations, e.g., histogram equalization, and filtering, are widely used to improve the visual quality of digital images. This paper designs an image processing network based on CapsNets (capsule networks), in which additional data can be carried in the processed image. Given an image to be processed, the proposed network is able to achieve some conventional image processing operations with satisfactory results. Meanwhile, additional data can be embedded into the processed image during the process of training, and the existence of additional data cannot be discovered. In this way, additional data can be transmitted secretly via the processed image, which looks normal. Compared with existing data hiding algorithms that embed data by modifying image content, the proposal to embed data during the process of training is more secure. Experimental results verify the effectiveness of the proposed network, including the quality of the processed image, embedding capacity, and security. [ABSTRACT FROM AUTHOR]

• Recent advances in automatic feature detection of fruits is reviewed. • Detection of fruits is explored via image-based approach, and their analysis. • The classification of fruits is divided into supervised and unsupervised tasks. • Watermelon studies focus on spectral features for detection or classification. • This review discusses issues and future research avenues for fruit detection systems. This document provides an overview of advances in the task of automatic feature detection and classification of fruits, with and special interest in watermelon (Citrillus lanatus). The review was written with the objective to highlight the wealth of knowledge that exist in the application of analytical, smart and sensing image recognition techniques commonly used in agro-industry, and the computational approaches used to make the classification possible. Also, an specific interest is put in the contributions made in the development of automatic recognition systems geared towards for watermelon using images, acoustic and spectroscopy methodologies. The importance of this document is that it provides a conceptual summary of the methods for the automatic recognition of fruits, including machine learning and evolutionary computational algorithms to analyze their sensed data. In conclusion this is a first step into recognizing the challenges and opportunities that can be addressed in this field to augment the visibility of these methods and to further modernize the agro-industrial sector. [ABSTRACT FROM AUTHOR]

Nowadays, neural networks have been widely applied in many fields such as pattern recognition, signal and image processing and control theory. Over the past two decades or so, the analysis and synthesis for neural networks have received significant research attention. This paper provides a survey on the analysis and synthesis for neural networks, which is mainly concerned with the recent advances on stability analysis, state estimation and synchronization control for neural networks. First of all, the paper summarizes the recent results on the stability analysis for delayed neural networks, especially for neural networks with multiple discrete delays, neural networks with distributed delays, and discrete-time delayed neural networks. Then, the paper reviews the recent advances regarding the state estimation for neural networks with the emphasis on the network-based state estimation. Subsequently, the paper provides an overview on the synchronization control for neural networks. Finally, the conclusions and further research directions are given. [ABSTRACT FROM AUTHOR]

Automated document processing for tabular information extraction is highly desired in many organizations, from industry to government. Prior works addressed this problem under table detection and table structure detection tasks. Proposed solutions that leverage deep learning approaches have been giving promising results; however, the impact of dataset structures on table structure detection has not been investigated. In this study, we provide a comparison of table structure detection performance with cropped and uncropped datasets. The cropped set consists of only table images that are cropped from documents assuming tables are detected perfectly. The uncropped set consists of regular document images. Experiments show that deep learning models can improve the detection performance by up to 9% in average precision and average recall on the cropped versions. Furthermore, the impact of cropped images is negligible under the Intersection over Union (IoU) values of 50%-70% when compared to the uncropped versions in small-scale datasets. However, beyond 70% IoU thresholds, cropped datasets provide significantly higher detection performance. In a large-scale dataset, cropped version outperforms the uncropped set under all IoU thresholds. [ABSTRACT FROM AUTHOR]

• We study the problem of distributed image-to-image translation. • A novel framework for distributed image-to-image translation. • A unified framework for distributed image-to-image translation and autoencoding. Data-driven paradigms using machine learning are becoming ubiquitous in image processing and communications. In particular, image-to-image (I2I) translation is a generic and widely used approach to image processing problems, such as image synthesis, style transfer, and image restoration. At the same time, neural image compression has emerged as a data-driven alternative to traditional coding approaches in visual communications. In this paper, we study the combination of these two paradigms into a joint I2I compression and translation framework, focusing on multi-domain image synthesis. We first propose distributed I2I translation by integrating quantization and entropy coding into an I2I translation framework (i.e. I2Icodec). In practice, the image compression functionality (i.e. autoencoding) is also desirable, requiring to deploy alongside I2Icodec a regular image codec. Thus, we further propose a unified framework that allows both translation and autoencoding capabilities in a single codec. Adaptive residual blocks conditioned on the translation/compression mode provide flexible adaptation to the desired functionality. The experiments show promising results in both I2I translation and image compression using a single model. [ABSTRACT FROM AUTHOR]

Curvilinear structures are present in various fields in image processing such as blood vessels in medical imaging or roads in remote sensing. Their detection is crucial for many applications. In this article, we propose an unsupervised plug-and-play framework for the segmentation of curvilinear structures that focuses on the preservation of their connectivity. This framework includes an algorithm for generating realistic pairs of connected/disconnected curvilinear structures and a reconnecting regularization operator that can be learned from a synthetic dataset. Once learned, this regularization operator can be plugged into a variational segmentation scheme and used to segment curvilinear structure images without requiring annotations. We demonstrate the interest of our approach on the segmentation of vascular images both in 2D and 3D and compare its results with classic unsupervised and deep learning-based approach. Comparative evaluations against unsupervised classic and deep learning-based methods highlight the superior performance of our approach, showcasing remarkable improvements in preserving the connectivity of curvilinear structures (approximately 90% in 2D and 70% in 3D). We finally showcase the good generalizability behavior of our approach on two different applications : road cracks and porcine corneal cells segmentations. [ABSTRACT FROM AUTHOR]

Breast cancer (BC) is a commonly diagnosed cancer among women nowadays. The cancer cells in the breast tissues are known as BC. Comprehensive research on early-stage BC detection helped to increase the survival rate and reduce the amortality rate associated with this disease. Mammogram scan analysis is a commonly used breast tissue visualization method. This image data is analyzed adequately for accurate BC diagnosis. Region of interest (ROI) identification is crucial in an image-based BC detection system. The ROI detection helps to segment the cancer tissues from the mammogram images by analyzing the heterogeneity among cancerous and normal breast tissues. Early-stage BC issues have homogeneous features as normal breast tissues. So, it is an open challenge for the researchers to develop a more accurate segmentation method during the automatic BC stages detection system. This study introduced fuzzy C means (FCM) distorted contour-based segmentation (FCM DCS) method to address the real detection issues in present studies. It uses the distorted contour (DC) based method to identify the contour of the cancer tissue from mammogram images. The DC method is performed with the help of FCM to identify the cancer tissues. Moreover, a histogram and adaptive equalization method were utilized to reduce image noise and preserve the edge features. The result analysis shows that the FCM-DC methods achieved a maximum accuracy rate (98.76 %) than comparison methods in BC detection. [ABSTRACT FROM AUTHOR]

Person re-identification has gained significant attention in recent years due to its numerous practical applications in video surveillance. However, while artificial intelligence and deep learning methods have enabled substantial progress in particular aspects of this domain, putting together those individual advances to generate practical systems remains a computer vision challenge. Existing methods are typically designed assuming the target person's images are captured under uniform, stable conditions with similar lighting levels, but this assumption may not hold in real-world scenarios, such as outdoor monitoring over 24 h, as image quality can vary considerably throughout day and night. In this paper, we propose a framework that incorporates image enhancement techniques to improve the performance of a person re-identification model. The proposed approach achieves a significant improvement in a demanding re-identification dataset, raising the mAP from 9.0% using a zero-shot baseline to 65.8% through the combined use of low-light image enhancement methods and noise reduction. [ABSTRACT FROM AUTHOR]

High dynamic range (HDR) imaging is an important task in image processing that aims to generate well-exposed images in scenes with varying illumination. Although existing multi-exposure fusion methods have achieved impressive results, generating high-quality HDR images in dynamic scenes remains difficult. The primary challenges are ghosting artifacts caused by object motion between low dynamic range images and distorted content in underexposure and overexposed regions. In this paper, we propose a deep progressive feature aggregation network for improving HDR imaging quality in dynamic scenes. To address the issues of object motion, our method implicitly samples high-correspondence features and aggregates them in a coarse-to-fine manner for alignment. In addition, our method adopts a densely connected network structure based on the discrete wavelet transform, which aims to decompose the input features into multiple frequency subbands and adaptively restore corrupted contents. Experiments show that our proposed method can achieve state-of-the-art performance under different scenes, compared to other promising HDR imaging methods. Specifically, the HDR images generated by our method contain cleaner and more detailed content, with fewer distortions, leading to better visual quality. [ABSTRACT FROM AUTHOR]

In recent years, with the great success of deep learning and especially deep unsupervised learning, many deep architectural clustering methods, collectively known as deep clustering, have emerged. Deep clustering shows the potential to outperform traditional methods, especially in handling complex high-dimensional data, taking full advantage of deep learning. To achieve a comprehensive overview of the field of deep clustering, this review systematically explores deep clustering methods and their various applications. First, the basic network architecture of deep clustering is described in detail, including the common network frameworks, and loss functions. Subsequently, deep clustering is divided into several categories based on the network architecture, and benchmark datasets and evaluation metrics in the field are introduced. Next, the real-world applications of deep clustering are explored in depth, providing successful cases in the fields of bioinformatics, medicine, anomaly detection, and image processing, highlighting the broad applicability of deep clustering in solving real-world challenges. Finally, the paper summarizes its contributions and explores potential directions for future research in deep clustering. • Analyze the theory of deep clustering and its performance advantages and disadvantages. • Summarizes the improvement of deep clustering in recent years, analyzes the improvement effect, and shows it with experimental data. • Finally, the related application research of deep clustering in different fields is introduced. At the same time, we summarize and prospect the improvement and development of deep clustering. [ABSTRACT FROM AUTHOR]

• We propose GATE-CNN architecture for binary classification of brain tumor using MR (Magnetic Resonance) images. • We have implemented our model on three different datasets; all consists of MRI with a significant difference in number of images. • The first dataset consists of tumorous and non-tumorous MRIs, second consists of MRIs with glioma and pitutiary (types of brain tumor), and the third consists of cancerous and non-cancerous (severity of brain tumor) images. • For all three datasets we found better classification results than any of the existing state-of-the-art models. Early and accurate detection is a solitary precaution to overcome the brain tumor. Otherwise, it will result in a deadly disease. Brain tumor (BT) detection using magnetic resonance is an essential and challenging job in the medical domain. To combat the aggressive spreading of BT, clinical imaging (MRI and X-Ray) can be an appropriate method for diagnosis. In this paper, we applied attention based image classification, which is a new paradigm for obtaining improved accuracy than the state-of-the-art models. We propose a GATE-CNN (Graph Attention AutoEncoder-Convolution Neural Network) model in this work for the classification of BT. We calculate the attention values of the neighboring pixels on each and every pixel present in the graph then process the graph using GATE framework and the processed graph with attention values is then passed to CNN framework for generation of final output. Further, we apply Adamax optimizer to optimize the training hyperparameters of CNN. We perform the BT classification using the proposed method on three different datasets among benign and malignant BT for first dataset, glioma and pituitary for second dataset, and normal and abnormal BT images for third dataset; all the three datasets consists of MRI images. We then compare the proposed model with a variety of CNN (Deep-CNN, Multi-Input CNN, and basic CNN) models dealing with several types of medical imaging (thermal image, MRI, X-Ray, etc.). Our model results in the accuracy of 98.27% for first dataset, 99.83% for second dataset, and 98.78% for third dataset demonstrating preferable network performance than the already established state-of-the-art CNN models. [ABSTRACT FROM AUTHOR]

Spectral Clustering (SC) is an important method in areas such as data mining, image processing, computer science and so on. It attracts more and more attention owing to its effectiveness in unsupervised learning. However, SC has poor performance in the high-dimensional data. Traditional SC methods conduct the spectral embedding of the affinity matrix among data at the first beginning, and then obtain clustering results by the K -means clustering. The also have drawbacks int two processing steps: the clustering results are sensitive to the affinity matrix which may be inaccurate and the post-processing K -means may also be limited by its initialization problem. In the paper, a new approach which joints spectral embedding and clustering with structured graph optimization (called JSEGO) is proposed. In the new model, the low-dimensional representation of data can first be obtained by the spectral embedding method, which can handle with the high-dimensional data better. Then, the optimization similarity matrix would be obtained with such the embedded data. Furthermore, the learning structure graph gives feedback to the similarity matrix to generate better spectral embedded data. As a result, better similarity matrix and clustering result can be obtained by the iterations simultaneously, which are often conducted in two separate steps in the spectral clustering. As a result, the drawbacks introduced by the two processing introduces can be solved. At last, we use an alternative optimization method in the new model and conduct the theoretical analysis by comparing this proposed method with K -means clustering. Experiments based on synthetic data and actual benchmark data prove the advantage of this new approach. [ABSTRACT FROM AUTHOR]

• The recent edge detection methods are classified and introduced in detail. • The existing widely used datasets and evaluation criteria for edge detection methods are reviewed. • Future research directions for edge detection are elaborated. Edge detection is one of the most important and fundamental problems in the field of computer vision and image processing. Edge contours extracted from images are widely used as critical cues for various image understanding tasks such as image segmentation, object detection, image retrieval, and corner detection. The purpose of this paper is to review the latest developments on image edge detection. Firstly, the definition and properties of edges are introduced. Secondly, the existing edge detection methods are classified and introduced in detail. Thirdly, the existing widely used datasets and evaluation criteria for edge detection methods are summarized. Finally, future research directions for edge detection are elaborated. [ABSTRACT FROM AUTHOR]

Estimating vehicles' locations is one of the key components in intelligent traffic management systems (ITMSs) for increasing traffic scene awareness. Traditionally, stationary sensors have been employed in this regard. The development of advanced sensing and communication technologies on modern vehicles (MVs) makes it feasible to use such vehicles as mobile sensors to estimate the traffic data of observed vehicles. This study aims to explore the capabilities of a monocular camera mounted on an MV in order to estimate the geolocation of the observed vehicle in a global positioning system (GPS) coordinate system. We proposed a new methodology by integrating deep learning, image processing, and geometric computation to address the observed-vehicle localization problem. To evaluate our proposed methodology, we developed new algorithms and tested them using real-world traffic data. The results indicated that our proposed methodology and algorithms could effectively estimate the observed vehicle's latitude and longitude dynamically. [ABSTRACT FROM AUTHOR]

Fractional calculus is an abstract idea exploring interpretations of differentiation having non-integer order. For a very long time, it was considered as a topic of mere theoretical interest. However, the introduction of several useful definitions of fractional derivatives has extended its domain to applications. Supported by computational power and algorithmic representations, fractional calculus has emerged as a multifarious domain. It has been found that the fractional derivatives are capable of incorporating memory into the system and thus suitable to improve the performance of locality-aware tasks such as image processing and computer vision in general. This article presents an extensive survey of fractional-order derivative-based techniques that are used in computer vision. It briefly introduces the basics and presents applications of the fractional calculus in six different domains viz. edge detection, optical flow, image segmentation, image de-noising, image recognition, and object detection. The fractional derivatives ensure noise resilience and can preserve both high and low-frequency components of an image. The relative similarity of neighboring pixels can get affected by an error, noise, or non–homogeneous illumination in an image. In that case, the fractional differentiation can model special similarities and help compensate for the issue suitably. The fractional derivatives can be evaluated for discontinuous functions, which help estimate discontinuous optical flow. The order of the differentiation also provides an additional degree of freedom in the optimization process. This study shows the successful implementations of fractional calculus in computer vision and contributes to bringing out challenges and future scopes. [ABSTRACT FROM AUTHOR]

Super-resolution (SR) and image enhancement (IE) are two common and important tasks in image processing. Due to poor equipment, illumination, and photographic skills, many low-resolution and low-quality (in terms of color, and contrasts) photos are captured in our daily life. To make the photos more visually pleasing, we usually enlarge and retouch them in two separate steps. In this paper, we propose a new Joint Image Super-resolution and Enhancement Network (JSENet), which is arguably the first end-to-end method based on deep learning for joint SR and IE. Compared with simply conducting these two tasks in parallel or sequentially, JSENet employs the bilateral learning framework to integrate the two tasks together seamlessly. Meanwhile, two lightweight modules are designed for restoring details and generating color transformation coefficients respectively, which ensures that our JSENet can be deployed in real-world applications. Extensive experiments are conducted to demonstrate that JSENet outperforms the state-of-the-art SR-IE methods. [ABSTRACT FROM AUTHOR]

[Display omitted] • A comprehensive survey on image watermarking schemes in different domains. • Introduction of schemes for new digital medias and geometric invariant watermarking. • Explanation of various learning-based watermarking and schemes for deep model. • Discussion and future directions have been given in the field of image watermarking. With the rapid development and popularity of the Internet, multimedia security has become a general essential concern. Especially, as manipulation of digital images gets much easier, the challenges it brings to authentication certification are increasing. As part of the solution, digital watermarking has made significant contributions to image content security and has attracted increasing attention. In this paper, we present a comprehensive review on digital image watermarking methods that were published in recent years illustrating the conventional schemes in different domains. We provide an overview of geometric invariant techniques and emerging watermarking methods for novel medias, such as depth image based rendering (DIBR), high dynamic range (HDR), screen content images (SCIs), and point cloud model. Particularly, as deep learning has achieved a great success in the field of image processing, and has also successfully been used in the field of digital watermarking, learning-based watermarking methods using various neural networks are summarized according to the utilization of neural networks in the single stage training (SST) and double stage training (DST). Finally, we provide an analysis and summary on those methods, and suggest some future research directions. [ABSTRACT FROM AUTHOR]

In this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image processing, especially in image classification. However, image restoration is a fundamental and challenging topic and plays significant roles in image processing, understanding and representation. It typically addresses image deblurring, denoising, dehazing and super-resolution. There are substantial differences in the approaches and mechanisms in deep learning methods for image restoration. Discriminative learning based methods are able to deal with issues of learning a restoration mapping function effectively, while optimisation models based methods can further enhance the performance with certain learning constraints. In this paper, we offer a comparative study of deep learning techniques in image denoising, deblurring, dehazing, and super-resolution, and summarise the principles involved in these tasks from various supervised deep network architectures, residual or skip connection and receptive field to unsupervised autoencoder mechanisms. Image quality criteria are also reviewed and their roles in image restoration are assessed. Based on our analysis, we further present an efficient network for deblurring and a couple of multi-objective training functions for super-resolution restoration tasks. The proposed methods are compared extensively with the state-of-the-art methods with both quantitative and qualitative analyses. Finally, we point out potential challenges and directions for future research. [ABSTRACT FROM AUTHOR]

[Display omitted] • NODEs to provide continuous depth for DNNs rather than stacking layers. • Infinite deep NODE-based NNs with flexible complexity while training. • Better loss landscape, sufficient theoretical support, and visual proofs. • Extensive experiments on several widely used image benchmarks. The depth of the deep neural network (DNN) refers to the number of hidden layers between the input and output layers of an artificial neural network. It usually indicates a certain degree of complexity of the computational cost (parameters and floating point operations per second) and expressiveness once the network structure is settled. In this study, we experimentally investigate the effectiveness of using neural ordinary differential equations (NODEs) as a component to provide further depth in a continuous way to relatively shallower networks rather than stacking more layers (discrete depth), which achieved an improvement with fewer parameters. Experiments are conducted on classic DNNs, the residual networks. Moreover, we construct infinite deep neural networks with flexible complexity based on NODEs, enabling the system to adjust its complexity during training. On a better hidden-space provided by adaptive step DNNs, adaptive step ResNet with NODE (ResODE) is managed to achieve better performances in terms of convergence and accuracy than standard networks, and the improvements are widely observed in popular benchmarks. [ABSTRACT FROM AUTHOR]

Image deraining is essential in both image processing and computer vision. However it is still challenging to remove rain streaks, due to the fact that they are complex and usually have multiple directions. In this paper, we propose a novel infimal convolution type functional based model to remove multi-directional rain streaks. Specifically, we employ total generalized variation (TGV) to represent the rain-free background and m -fold infimal convolution of oscillation TGV ( ICTGV m osci ) to capture the rain streaks with m different directions. The existence of solutions to the proposed model is proved. We also design an algorithm using the classical first-order Primal–Dual method for solving the proposed model. Numerical experiments on both synthetic and real images demonstrate that the proposed model outperforms the state-of-the-art methods quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Intra-domain and inter-domain gaps are widely presented in image processing tasks due to data distribution differences. In the field of image dehazing, particular previous works have paid attention to the inter-domain gap between the synthetic domain and the real domain. However, those methods only establish the connection from the real domain to the synthetic domain without considering the significant distribution shift within the synthetic domain (intra-domain gap). In this work, we propose a Two-Step Dehazing Network (TSDN) with an intra-domain adaptation and a constrained inter-domain adaptation. First, we subdivide the distributions within the synthetic domain into subsets and mine the optimal subset (easy samples) by loss-based supervision. To alleviate the intra-domain gap of the synthetic domain, we propose an intra-domain adaptation to align distributions of other subsets to the optimal subset by adversarial learning. Finally, we conduct the constrained inter-domain adaptation from the real domain to the optimal subset of the synthetic domain, alleviating the domain shift between domains as well as the distribution shift within the real domain. Extensive experimental results demonstrate that our framework performs favorably against the state-of-the-art algorithms both on the synthetic datasets and the real datasets. [ABSTRACT FROM AUTHOR]

Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]

• A CNN for heavy rain removal is proposed based on two networks connected in series. • Novel attention modules are introduced to effectively remove heavy rain effect. • New heavy rain data is proposed with new haze map and various heavy rain models. In this paper, we propose a novel heavy rain removal algorithm using a deep neural network. Unlike most of the existing deraining methods, heavy rain removal is a more challenging task because it is necessary to remove both the rain marks and the haze effects, which are entangled in a complex manner. Motivated by this, we propose a new end-to-end two-stage attention network for single-shot heavy rain removal. The proposed network is connected serially with a removal network and a recovery network, which are based on a newly introduced dilation-wise attention block and skip attention block. Based on these attention techniques, the removal network predicts the heavy rain effect that needs to be removed from a given image, and the recovery network successfully predicts the details that need to be recovered, resulting in a clean image. We also introduce a new realistic RainCityscapes+ dataset, composed of synthesized outdoor images, and demonstrate extensive experiments, the results of which show our approach outperforms the state-of-the-art methods on both real and synthetic datasets quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Within the realm of computer vision, the landscape has been significantly reshaped by the abundance of extensive and diverse datasets, leading to remarkable breakthroughs in image processing. These advancements have reverberated across a wide spectrum of applications, catalyzing transformative outcomes. However, in stark contrast, the field of digital rock analysis finds itself grappling with a conspicuous dearth of data, a challenge that casts a formidable shadow over the effective deployment of computer vision techniques for rock image analysis. In response to this pressing issue, this paper presents a pioneering methodology designed to surmount the hurdles posed by data limitation in the realm of digital rock analysis. At the core of this innovative approach lies the fusion of artificially generated digital rock images, created using a state-of-the-art diffusion model, with their authentic counterparts. This fusion is guided by the overarching objective of augmenting the efficacy of various digital rock analysis applications. This integration endeavors to bridge the gap between the limited available data and the substantial demands of the digital rock analysis domain. The practical significance and potential of this integrated approach are vividly demonstrated through a series of concrete implementations. These include, but are by no means limited to, enhancing image quality to facilitate clearer visualization of intricate rock structures and refining the estimation of petrophysical properties with increased accuracy. [ABSTRACT FROM AUTHOR]

U-Net, known for its simple yet efficient architecture, is widely utilized for image processing tasks and is particularly suitable for deployment on neuromorphic chips. This paper introduces the novel concept of Spiking-UNet for image processing, which combines the power of Spiking Neural Networks (SNNs) with the U-Net architecture. To achieve an efficient Spiking-UNet, we face two primary challenges: ensuring high-fidelity information propagation through the network via spikes and formulating an effective training strategy. To address the issue of information loss, we introduce multi-threshold spiking neurons, which improve the efficiency of information transmission within the Spiking-UNet. For the training strategy, we adopt a conversion and fine-tuning pipeline that leverage pre-trained U-Net models. During the conversion process, significant variability in data distribution across different parts is observed when utilizing skip connections. Therefore, we propose a connection-wise normalization method to prevent inaccurate firing rates. Furthermore, we adopt a flow-based training method to fine-tune the converted models, reducing time steps while preserving performance. Experimental results show that, on image segmentation and denoising, our Spiking-UNet achieves comparable performance to its non-spiking counterpart, surpassing existing SNN methods. Compared with the converted Spiking-UNet without fine-tuning, our Spiking-UNet reduces inference time by approximately 90%. This research broadens the application scope of SNNs in image processing and is expected to inspire further exploration in the field of neuromorphic engineering. The code for our Spiking-UNet implementation is available at https://github.com/SNNresearch/Spiking-UNet. [ABSTRACT FROM AUTHOR]

Addressing the reproducibility crisis in artificial intelligence through the validation of reported experimental results is a challenging task. It necessitates either the reimplementation of techniques or a meticulous assessment of papers for deviations from the scientific method and best statistical practices. To facilitate the validation of reported results, we have developed numerical techniques capable of identifying inconsistencies between reported performance scores and various experimental setups in machine learning problems, including binary/multiclass classification and regression. These consistency tests are integrated into the open-source package mlscorecheck , which also provides specific test bundles designed to detect systematically recurring flaws in various fields, such as retina image processing and synthetic minority oversampling. [ABSTRACT FROM AUTHOR]

We present TNCR, a new table dataset with varying image quality collected from open access websites. TNCR dataset can be used for table detection in scanned document images and their classification into 5 different classes. TNCR contains 9428 labeled tables with approximately 6621 images. In this paper, we have implemented state-of-the-art deep learning-based methods for table detection to create several strong baselines. Deformable DERT with Resnet-50 Backbone Network achieves the highest performance compared to other methods with a precision of 86.7%, recall of 89.6%, and f1 score of 88.1% on the TNCR dataset. We have made TNCR open source in the hope of encouraging more deep learning approaches to table detection, classification and structure recognition. The dataset and trained model checkpoints are available at https://github.com/abdoelsayed2016/TNCR_Dataset. [ABSTRACT FROM AUTHOR]

Global supply chains are kept viable through the information shared through billions of electronic documents, many of which extensively use tables to display critical information. Making effective supply chain decisions requires the extraction of data from these tables which is hindered by the variations in layouts and styles of tables. In this paper, we propose Table Det: a deep learning based methodology to solve table detection and table image classification in data sheet images in a single inference as the first stage of the table text extraction pipeline. TableDet utilizes Cascade R-CNN with Complete IOU (CIOU) loss and a deformable convolution backbone as its underlying architecture to capture the variations in scales and orientations of tables. It also detects text and figures to enhance its table detection performance. We demonstrate the effectiveness of training TableDet with a dual-step transfer learning process and fine-tuning it with Table Aware Cutout (TAC) augmentation strategy. We achieved the highest F1 score for table detection against state-of-the-art solutions on ICDAR 2013 (complete set), ICDAR 2017 (test set) and ICDAR 2019 (test set) with 100%, 99.3% and 95.1% respectively. For the table image classification task we attained 100% recall and above 85% precision on three test sets. This classification capability ensures that all images with tables would be promoted to the next step in the table text extraction pipeline, with a small number of images without tables making it through. [ABSTRACT FROM AUTHOR]

Optic disc and cup segmentation play an essential step towards automatic retinal diagnose system. The task is very challenging since the boundary between optic disc and cup is weak and the existing segmentation network with cross-entropy loss is hard to inject domain-specific knowledge. To solve the problem, we propose a level set based deep learning method for optic disc and cup segmentation. Particularly, we treat the output of the neural network as a level set and add several constraints to make the predicted level set satisfy some characteristics, such as the length constraint and region constraint. The length term lets the boundary tend to smooth while the region term lets the response inside the predicted area tend to be the same. The region term considers the relationship between pixels inside optic disc or cup while the cross-entropy loss treats the segmentation as a pixel-wise classification without considering the relationship between pixels. We conduct extensive experiments on several datasets including ORIGA and REFUGE and DRISHTI-GS dataset. The experiment results verify the effectiveness of our method. [ABSTRACT FROM AUTHOR]

• Synaptic metaplasticity inclusion in convolutional neural networks. • Adaptation to metaplasticity of backpropagation stage of convolutional layers. • Using LeNet-5, AlexNet, GoogLeNet, VGG(16 & 32), ResNet50, DenseNet(121 &169). • Experimentation using MNIST, Fashion MNIST, CIFAR-10 and CIFAR-100 datasets. • Good improvements in performance obtained after introducing metaplasticity. Synaptic metaplasticity is a biological phenomenon shortly defined as the plasticity of synaptic plasticity, meaning that the previous history of the synaptic activity determines its current plasticity. This phenomenon interferes with some of the underlying mechanisms that are considered important in memory and learning processes, such as long-term potentiation and long-term depression. In this work, we provide an approach to include metaplasticity in convolutional neural networks to enhance learning in image classification problems. This approach consists of including metaplasticity as a weight update function in the backpropagation stage of convolutional layers. To validate this proposal, we have been used eight different award-winning convolutional neural networks architectures: LeNet-5, AlexNet, GoogLeNet, VGG16, VGG32, ResNet50, DenseNet121 and DenseNet169; trained with four different popular datasets for benchmarking: MNIST, Fashion MNIST, CIFAR-10 and CIFAR-100. Experimental results show that there is a performance enhancement for each of the convolution neural network architectures in all the datasets used. [ABSTRACT FROM AUTHOR]

Graph matching is a fundamental problem in image processing and computer vision tasks, which can be used to solve the feature correspondence problem. Most graph matching tasks have been proved to be NP-Complete problems which are generally solved by approximate algorithms based on the continuous relaxation scheme. Targeting at these problems, this paper proposes a novel graph matching algorithm based on the alternating direction method of multipliers (ADMM) which typically decomposes a difficult problem into relatively simpler subproblems. Specifically, the matching constraints are decomposed into four continuous constraints and the assignment vector is decomposed into different vectors according to the constraints. Then the transformed objective function is iteratively optimized under the framework of ADMM. The proposed method achieves the state of the art performance, while maintaining a comparable computational complexity. And a discrete solution can be directly obtained without the projection process. Experimental results on both synthetic points and real images demonstrate the effectiveness of the proposed method by comparing it with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks have shown great potential in image recognition tasks. However, the fact that the mechanism of deep learning is difficult to explain hinders its development. It involves a large amount of parameter learning, which results in high computational complexity. Moreover, deep convolutional neural networks are often limited by overfitting in regimes in which the number of training samples is limited. Conversely, kernel learning methods have a clear mathematical theory, fewer parameters, and can contend with small sample sizes; however, they are not able to handle high-dimensional data, e.g., images. It is important to achieve a performance and complexity trade-off in complicated tasks. In this paper, we propose a novel scalable deep convolutional random kernel learning in Gaussian process architecture called SDCRKL-GP, which is characterized by excellent performance and low complexity. First, we successfully incorporated the deep convolutional architecture into kernel learning by implementing the random Fourier feature transform for Gaussian processes, which can effectively capture hierarchical and local image-level features. This approach enabled the kernel method to effectively handle image processing problems. Second, we optimized the parameters of deep convolutional filters and Gaussian kernels by stochastic variational inference. Then, we derived the lower variational bound of the marginal likelihood. Finally, we explored the model architecture design space selection method to determine the appropriate network architecture for different datasets. The design space consists of the number of layers, the channels per layer, and so on. Different design space selections improved the scalability of the SDCRKL-GP architecture. We evaluated SDCRKL-GP on the MNIST, FMNIST, CIFAR10, and CALTECH4 benchmark datasets. Taking MNIST as an example, the error rate of classification is 0.60%, and the number of parameters, number of computations and memory access cost of the architecture are 19.088k, 0.984M, and 1.057M, respectively. The experimental results verified that the proposed SDCRKL-GP method outperforms several state-of-the-art algorithms in both accuracy and speed in image recognition tasks. The code is available at https://github.com/w-tingting/deep-rff-pytorch. [ABSTRACT FROM AUTHOR]

This paper proposed a novel approach to generate face caricatures automatically from a single portrait image. We decompose the process of 3D face caricatures generation into two independent subtasks: appearance transfer of texture and the geometry transfer of mesh. For the appearance transfer, we design a GAN-based network named CariFaceGAN to learn the style mapping from portrait to caricature, in which facial features are leveraged to preserve identity consistency. For geometry transfer, we first learn the transformation of the landmarks between portraits and caricatures in an embedded space obtained with Locally Linear Embedding method, and then Kriging interpolation is used to manipulate the portrait mesh constructed from a single image. The experimental results show that our proposed CariFaceGAN outperforms the state-of-the-art methods in terms of maintaining identity consistency and providing satisfactory visual effects. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks (CNNs) have been successfully applied in the field of image processing, and have been deployed to a variety of artificial intelligence systems. However, such neural models are still considered to be "black box" for most tasks. Two of fundamental issues underlying this problem are as follows: 1. What type of knowledge learned by a neural network in a task was not understandable and predictable? 2. The decision made by a neural model was generally not evaluable. Like neural coding in the brain, some neurons only participated in encoding a particular task. Inspired by this, in this paper, we propose a method to modify traditional CNN models into understandable CNNs, to clarify the information coding in high conv-layers of CNNs and further evaluate the decisions made by a neural model. In our understandable CNN models, each neural node (feature map) in a selected conv-layer was assigned to participate in encoding only one class in a classification task. Our models use the same training data as ordinary models without the need for additional annotations for supervision. We applied our method to the ResNet and DenseNet models. The experiments showed that new models can learn the information coding mode that we expected in an image-recognition task, and, using the pre-assigned coding mode, we can interpret why a neural model makes a right or wrong decision, which decisions are credible, and which are not. [ABSTRACT FROM AUTHOR]

Micro milling is a versatile machining process owing to its capability to machine, by material removal, micro-sized components with complex geometrical features. However, micro milling tools wear quickly, being a key issue to determine tool wear condition in order to prevent excessive tool wear or a sudden tool breakage while machining, which would waste the workpiece. Due to the small size of micro milling tools, direct measurement of the worn tool is not possible. In order to overcome this drawback, this paper presents a new method based on digital image processing where image captures of the micro tool and subsequent analysis provides a valuable information to determine the progression of tool wear. Tool wear is measured in flank wear, crater wear and tool radius reduction. Different approaches are compared so as to determine the best option for every set of images. These methods are based on the use of morphological operations, k-means clustering and Otsu Multilevel algorithm. Results show a good performance with differences of 5% between predicted and actual worn area, which satisfies the industrial requirements. This procedure can be transferred to industrial environments and implemented in collaborative robots, increasing the level of automation and facilitating the decision-making process. [ABSTRACT FROM AUTHOR]

DeepLab—one of the most critical deep neural network models for image segmentation—has achieved the most advanced performance in the field of image semantic understanding. However, the rich parameters and calculation of deep convolutional neural networks (DCNNs) demand further research on neuro-inspired computing chips specifically designed for hardware acceleration and AI applications. This motivates memristive solutions—the memristor integrates a range of merits, such as fast speed and nanoscale device, to provide an ideal implementation for building low-power neural computing chips and ultra-high-density non-volatile memory. Here, this paper proposes a memristive DeepLab (MDeepLab) system with software-hardware co-design, in which atrous convolution is used to enlarge the receptive field of neurons, the measure to avoid the gridding effect is adopted. Also, the weights are stored by leveraging the unit of a single crossbar array and the constant-term circuit, which significantly reduces the number of memristor devices, doubles the density, and nearly halves the power consumption conventional crossbar array. Finally, the effectiveness of the proposed scheme is verified by a series of experimental simulations and result analysis, showing the superiority of MDeepLab. This study is expected to provide a new solution for low-power consumption and real-time image processing of edge devices. [ABSTRACT FROM AUTHOR]

• Three-channel pseudo-color images preprocessing method is designed by concatenating different CT windows. • A dense skip connection encoding–decoding model (DSC-Net) is proposed to perform automatic segmentation of thymoma base on a deep convolutional neural network. • Dense connections are introduced into the architecture of the encoding path across different level feature maps in the DSC-Net. • Different level skip connections are designed between the encoding and decoding path in the DSC-Net. Accurately positioning and segmenting thymoma from computed tomography (CT) images is of great importance for an image-driven thymoma analysis. In clinical practice, the diagnosis and segmentation of thymomas for radiologists are time-consuming and inefficient tasks. Thus, it is necessary to develop a method to accurately and efficiently realize automatic segmentation of thymoma. Here, a dense skip connection encoding–decoding model (DSC-Net), which is a deep convolutional neural network, was proposed to perform automatic segmentation of thymoma with the ability to fuse feature maps under receptive fields of different scales. An image preprocessing method was also proposed to provide much more texture information and enhance the contrast between thymoma and its surrounding tissues. A total of 310 subjects who underwent contrast-enhanced CT scanning were included in this ethically-approved retrospective study. All of the CT slices were manually labeled by four experienced radiologists, and 80% of images were included in the training set and the rest were included in the testing set. The performance of segmentation was evaluated by calculating the accuracy, intersection over union (IoU), and Boundary F1 contour matching score (BFScore) between the predicted segmentation and the manual labels. For segmentation of thymoma in the testing set, the accuracy, IoU and BFScore were 92.96%, 87.86% and 0.9087 respectively. Compared to the U-Net method, the DSC-Net model improved IoU by 3.94%. In addition, the efficacy and robustness of DSC-Net in segmentation of different patients and different types of thymoma classified by the WHO histological classification criteria were verified. The proposed preprocessing method and DSC-Net demonstrated improved performance in segmentation of thymomas, suggesting the ability to provide consistent delineation and assist radiologists in their clinical applications. [ABSTRACT FROM AUTHOR]

Image super-resolution (SR) is a widely used and cost-effective technology in remote sensing image processing. Deep learning-based SR methods have shown promising performance, but they are prone to losing texture details. Instead, generative adversarial nets (GAN)-based methods can generate more visually acceptable results. However, GAN-based SR methods are suffering from scene variance and uncontrollable performance of discriminators as well as unstable training. Besides, both these two methods cannot yield arbitrary high-time SR images. To solve these issues, we propose a novel SR method for remote sensing images using C ascade G enerative A dversarial N ets (CGAN) with introduction of content fidelity and scene constraint, which can achieve arbitrary high-time high-quality SR image. More specifically, the scene-constraint item is incorporated to constrain generated feature for avoiding the risk of scene change. Then, content fidelity is introduced to stabilize the training and avoid gradient vanishing. Besides, an edge enhancement module is designed to preserve edge detail and suppress the noise. CGAN with these three components has achieved higher quality SR results than other recent state-of-the-art methods. Compared with these methods, our proposed method outperformed average increments of 7.3% SSIM, 7.3% FSIM and 6.0% MSIM on WHU-RS19 and NWPU-RESISC45 datasets. In addition, the evaluation of GAN-train and GAN-test gained average increments of 6.3% and 4.5% on the WHU-RS19 and AID datasets, respectively. [ABSTRACT FROM AUTHOR]

We propose a deep fully convolutional neural network with a new type of layer, named median layer, to restore images contaminated by salt-and-pepper (s&p) noise. A median layer simply performs median filtering on all feature channels. By adding this kind of layer into some widely used fully convolutional deep neural networks, we develop an end-to-end network that removes extremely high-level s&p noise without performing any non-trivial preprocessing tasks. Experiments show that inserting median layers into a simple fully-convolutional network with the L 2 loss significantly boosts signal-to-noise ratio. Quantitative comparisons testify that our network outperforms the state-of-the-art methods with a limited amount of training data. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have transformed computer vision, establishing themselves as the current state-of-the-art for image processing. Nevertheless, the training of current large DNN models is one of the main challenges to be solved. In this sense, data-parallelism has been the most widespread distributed training strategy since it is easy to program and can be applied to almost all cases. However, this solution suffers from several limitations, such as its high communication requirements and the memory constraints when training very large models. To overcome these limitations model-parallelism has been proposed, solving the most substantial problems of the former strategy. However, describing and implementing the parallelization of the training of a DNN model across a set of processes deployed on several devices is a challenging task. Current proposed solutions assume a homogeneous distribution, being impractical when working with devices of different computational capabilities, which is quite common on high performance computing platforms. To address previous shortcomings, this work proposes a novel model-parallelism technique considering heterogeneous platforms, where a load balancing mechanism between uneven devices of an HPC platform has been implemented. Our proposal takes advantage of the Google Brain's Mesh-TensorFlow for convolutional networks, splitting computing tensors across filter dimension in order to balance the computational load of the available devices. Conducted experiments show an improvement in the exploitation of heterogeneous computational resources, enhancing the training performance. The code is available on: https://github.com/mhaut/HeterogeneusModelDNN. [ABSTRACT FROM AUTHOR]

There is an increasing demand for automated document processing techniques as the volume of electronic component documents increase. This is most prevalent in the supply chain optimization sector where vast amount of documents need to be processed and is time consuming and prone to error. Detection of tables and table structures serves as a crucial step to automate document processing. While table detection is a well investigated problem, tabular structure detection is more complex, and requires further improvements. To address this, this study proposes a deep learning model that focuses on high precision tabular cell structure detection. The proposed model creates a benchmark for the ICDAR2013 dataset cell structure with comparison to the previous state of the art table detection models as well as proposing alternative models. Our methodology approaches improving table structure detection through the detection of cells instead of row and columns for better generalization capabilities for heterogeneous table structures. Our proposed model advances prior models by improving major parts of the detection pipeline, mainly the two-stage detector, backbone, backbone architecture, and non-maximum-suppression (NMS). TabCellNet consists of Hybrid Task Cascade (HTC) with Combinational Backbone Network (CBNet), dual ResNeXt101 and Soft-NMS to achieve a precision of 89.2% and recall of 98.7% on the hand annotated ICDAR2013 cell structure dataset. [ABSTRACT FROM AUTHOR]

Robust low-rank tensor completion plays an important role in multidimensional data analysis against different degradations, such as Gaussian noise, sparse noise, and missing entries, and has a variety of applications in image processing and computer vision. In this paper, we investigate the problem of low-rank tensor completion with different degradations for third-order tensors, and propose a transformed tensor nuclear norm method combined the tensor ℓ 1 norm with total variational (TV) regularization. Our model is based on a recently proposed algebraic framework in which the transformed tensor nuclear norm is introduced to capture lower transformed multi-rank by using suitable unitary transformations. We adopt the tensor ℓ 1 norm to detect the sparse noise, and the TV regularization to preserve the piecewise smooth structure along the spatial and tubal dimensions. Moreover, a symmetric Gauss–Seidel based alternating direction method of multipliers is developed to solve the resulting model and its global convergence is established under very mild conditions. Extensive numerical examples on both hyperspectral images and video datasets are carried out to demonstrate the superiority of the proposed model compared with several existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Although many hybrid convolutional neural networks using optical convolution processors have been demonstrated for high energy efficiency and fast image processing speed, the need for accurate modeling of the optical convolution hardware and large Fourier kernel sizes result in a long and energy-intensive training process. Here, we demonstrate a hybrid convolutional neural network based on an optimized optical convolution processor—the system uses kernels trained in the spatial domain and compensates the optical path mismatch that arises from the reflection of digital micromirror devices for convolution computation. The spatial-domain convolution kernels are Fourier transformed and then binarized before being programmed into a digital micromirror device in the optical convolution processor. This method minimizes overhead from the optical convolution process in the training of the hybrid convolutional neural network, and it can improve the energy efficiency and reduce the training time of the hybrid optical convolutional neural network. For convolution computation using digital micromirror devices with grayscale images, bit-plane-wise convolution is also presented. We tested the hybrid neural network on MNIST, Fashion-MNIST, and CIFAR-10 color datasets and obtained classification accuracies of 98.7%, 85.8%, and 57.8%, respectively. [ABSTRACT FROM AUTHOR]

Guided image processing techniques are widely used to extract meaningful information from a guiding image and facilitate the enhancement of the guided one. This paper specifically addresses the challenge of guided thermal image super-resolution, where a low-resolution thermal image is enhanced using a high-resolution visible spectrum image. We propose a new strategy that enhances outcomes from current guided super-resolution methods. This is achieved by transforming the initial guiding data into a representation resembling a thermal-like image, which is more closely in sync with the intended output. Experimental results with upscale factors of × 8 and × 16, demonstrate the outstanding performance of our approach in guided thermal image super-resolution obtained by mapping the original guiding information to a thermal-like image representation. [ABSTRACT FROM AUTHOR]

Distributed fiber optic vibration sensing system has been widely used in safety monitoring with distinct advantages, and the feature extraction and classification methods of fiber optic signals directly determine the real-time performance and reliability of the monitoring system. The existing research feature extraction methods are unidimensional and time consuming, which cannot balance the goals of high accuracy and low time consumption for safety monitoring systems. In this paper, we propose an efficient recognition framework for fusing signal time–frequency features, called TFF-CNN, based on Gramian Angular Difference Fields(GADF) and FFT co-generation matrix(FFTT) to manually extract two-dimensional time–frequency image features of fiber optic signals, taking the significant advantages of CNN in image processing and combining a two-channel model and a fusion module that simulates human decision-making behavior. The accuracy of TFF-CNN is 99.30 % and the detection response time is only 0.6 s. Compared with other methods in this field, TFF-CNN has the advantages of low false alarm rate and short time consumption, which is more suitable for deployment in security monitoring field with distributed fiber optic sensing system. Index Terms : Distributed optical fiber vibration sensing system(DVS), Two-dimensional multi-feature, CNN, intrusion detection, real-time monitoring. [ABSTRACT FROM AUTHOR]

• We design a fire-controlled MSPCNN with an automatic parameter setting method. Hereinto, the adaptive parameters α , β , V , and R n are defined to control the firing time of each neuron, and the predetermined parameter P is used for determining the total iteration times of all the neurons. • We propose a color image quantization method and a gallbladder image location method based on the FC-MSPCNN. Compared to other state-of-the-art algorithms, our proposed methods exhibit better image processing performances. A long-term research goal of pulse-coupled neural network (PCNN) is to control neuronal firing states at each iteration. Recently, we propose a fire-controlled MSPCNN model (FC-MSPCNN) and provide a parameter setting method to control firing and fired neurons within an effective pulse cycle. We firstly design the proposed model according to previous prevalent PCNN models. Secondly, the setting methods of the adaptive parameters α , β, V , and R n are given to control neuronal firing time more effectively. Thirdly, a predetermined parameter P will determine the total iteration times of all the neurons. Fourthly, we also propose a color image quantization method and a gallbladder image location method based on the FC-MSPCNN. The evaluation experiments achieve good image processing performances compared to prevalent PCNN models and prove the effectiveness and robustness of the proposed methods. [ABSTRACT FROM AUTHOR]

Due to the ill-posed nature of image restoration, seeking a meaningful image prior is still a great challenge in the field of image processing. The total variation with overlapping group sparsity (OGS-TV) has been successfully applied for image denoising/deblurring. In this paper, we further study the overlapping group sparsity of the image gradient. The sparsity is measured by the ℓ q quasi-norm (0 < q < 1). The proposed regularizer comes down to the well-known hyper-Laplacian prior if the overlapping group size is 1. Although it seems to be a simple extensive study compared with the previous works, its regularization capability and corresponding mathematical problems are still in demand for imaging science. To solve the non-convex and non-smooth minimization problem, we use the alternating direction method of multipliers as the main algorithm framework. The difficult inner subproblem is tackled by the majorization-minimization method with the sophisticatedly derived majorizer. We carry out some numerical experiments to demonstrate the effectiveness of the proposed regularizer in terms of PSNR and SSIM values. [ABSTRACT FROM AUTHOR]

Fast assessment of damaged buildings is important for post-disaster rescue operations. Building damage detection leveraging image processing and machine learning techniques has become a popular research focus in recent years. Although supervised learning approaches have made considerable improvement for damaged building assessment, rapidly deploying supervised classification is still difficult due to the complexity in obtaining a large number of labeled samples in the aftermath of disasters. We propose an unsupervised self-attention domain adaptation (USADA) model, which transforms instances of the source domain to those of the target domain in pixel space, to address the aforementioned issue. The proposed USADA consists of three parts: a set of generative adversarial networks (GANs), a classifier, and a self-attention module. The GAN adapts source domain images to ensure their similarity to target domain images. Once adapted, the classifier can be trained using the adapted images along with the original images of the source domain to classify damaged buildings. The self-attention module is introduced to maintain the foreground of the generated images conditioned on source domain images for generating plausible samples. As a case study, aerial images of Hurricane Sandy, Maria, and Irma, are used as the source and target domain datasets in our experiments. Experimental results demonstrate that classification accuracies of 68.1% and 84.1% are achieved, and our method obtains improvements of 2.0% and 3.6% against pixel-level domain adaptation, which is the basis of our model. [ABSTRACT FROM AUTHOR]

• Mechanism of multi-task auxiliary learning and image fusion method. • The dynamic context aware ability of image fusion network. • The influence of human's top-down subjective attention on image fusion task is studied. • The method of image fusion guided by visual attention is proposed. The human visual perception system has very strong robustness and contextual awareness in a variety of image processing tasks. This robustness and the perception ability of contextual awareness is closely related to the characteristics of multi-task auxiliary learning and subjective attention of the human visual perception system. In order to improve the robustness and contextual awareness of image fusion tasks, we proposed a multi-task auxiliary learning image fusion method guided by subjective attention. The image fusion method effectively unifies the subjective task intention and prior knowledge of human brain. In order to achieve our proposed image fusion method, we first analyze the mechanism of multi-task auxiliary learning, build a multi-task auxiliary learning network. Secondly, based on the human visual attention perception mechanism, we introduce the human visual attention network guided by subjective tasks on the basis of the multi-task auxiliary learning network. The subjective intention is introduced by the subjective attention task model, so that the network can fuse images according to the subjective intention. Finally, in order to verify the superiority of our image fusion method, we carried out experiments on the combined vision system image data set, and the infrared and visible image data set for experimental verification. The experimental results demonstrate the superiority of our fusion method over state-of-arts in contextual awareness and robustness. [ABSTRACT FROM AUTHOR]

Blind image deconvolution is a fundamental task in image processing, computational imaging, and computer vision. It has earned intensive attention in the past decade since the seminal work of Fergus et al. [1] for camera shake removal. In spite of the recent great progress in this field, this paper aims to formulate the blind problem with a simpler modeling perspective. What is more important, the newly proposed approach is expected to achieve comparable or even better performance towards the real blurred images. Specifically, the core critical idea is the proposal of a pure gradient-based discriminative prior for accurate and robust blur kernel estimation. Numerous experimental results on both the benchmark datasets and real-world blurred images in various imaging scenarios, e.g., natural, manmade, low-illumination, text, or people, demonstrate well the effectiveness and robustness of the proposed approach. [ABSTRACT FROM AUTHOR]