Search

Images and other types of unstructural data in the medical domain are rapidly becoming data-intensive. Actionable insights from these complex data present new opportunities but also pose new challenges for classification or segmentation of unstructural data sources. Over the years, medical problems have been solved by combining traditional statistical methods with image processing methods. Both the increase in the size of the data and the increase in the resolution are among the factors that shape the ongoing improvements in artificial intelligence (AI), particularly concerning deep learning (DL) techniques for evaluation of these medical data to identify, classify, and quantify patterns for clinical needs. At this point, it is important to understand how Artificial Neural Networks (ANNs), which are an important milestone in interpreting big data, transform into Deep Convolutional Neural Networks (DCNNs) and to predict where the change will go. We aimed to explain the needs of these stages in medical image processing through the studies in the literature. At the same time, information is provided about the studies that lead to paradigm shift and try to solve the image related medical problems by using DCNNs. With the increase in the knowledge of medical doctors on this subject, it will be possible to look at the solution of new problems in computer science from different perspectives. [ABSTRACT FROM AUTHOR]

Short-term spatio-temporal solar irradiance forecasting plays a pivotal role in scheduling and dispatching energy for distributed energy systems. Fluctuations in cloud cover can be monitored via satellite cloud imagery, which directly impacts irradiance. However, integrating and fusing multi-source heterogeneous data, such as satellite cloud images and ground monitoring data from distributed stations, remains challenging. Here, a spatio-temporal irradiance forecast model is proposed based on multi-modal deep learning model to predict global horizontal irradiance 30 min ahead. To address the feature extraction of heterogeneous data, a dual-channel structure consisting of a time-series processing block and a satellite cloud image processing block is developed to enable parallel processing of multi-modal features. In order To tightly couple cloud images and historical time series at the feature level, maximum mean discrepancy of these two feature is used to help the fusion of heterogeneous data. Furthermore, a self-attention mechanism is employed to construct adaptive inter-region information weights to enhance spatio-temporal representation ability. The evaluation of the method is conducted on open-access datasets from six locations in Jiangsu Province, China. Experimental results demonstrate that the proposed model efficiently utilizes heterogeneous data to improve prediction accuracy under various conditions and enhances model robustness, reducing RMSE by 2.8%–20.58%. Meanwhile, the proposed end-to-end model reduces training and deployment costs for real-world use. [ABSTRACT FROM AUTHOR]

Deep learning has emerged as a powerful tool in various domains, revolutionising machine learning research. However, one persistent challenge is the scarcity of labelled training data, which hampers the performance and generalisation of deep learning models. To address this limitation, researchers have developed innovative methods to overcome data scarcity and enhance deep model learning capabilities. Two prevalent techniques that have gained significant attention are transfer learning and self-supervised learning. Transfer learning leverages knowledge learned from pre-training on a large-scale dataset, such as ImageNet, and applies it to a target task with limited labelled data. This approach allows models to benefit from the learned representations and effectively transfer knowledge to new tasks, resulting in improved learning performance and generalisation. On the other hand, self-supervised learning focuses on training models using pretext tasks that do not require manual annotation, allowing them to learn valuable representations from large amounts of unlabelled data. These learned representations can then be fine-tuned for downstream tasks, mitigating the need for extensive labelled data. In recent years, transfer and self-supervised learning have found applications in various fields, including medical image processing, video recognition, and natural language processing. These approaches have demonstrated remarkable achievements, enabling breakthroughs in areas such as disease diagnosis, object recognition, and language understanding. However, while these methods offer numerous advantages, they also have limitations. For example, transfer learning may face domain mismatch issues between the pre-training and target domains, while self-supervised learning requires careful design of pretext tasks to ensure meaningful representations. This review paper explores the recent applications of these pre-training methods in various fields within the past three years. It delves into the advantages and limitations of each approach, assesses the performance of models employing these techniques, and identifies potential directions for future research. By providing a comprehensive review of current pre-training methods, this article offers guidance for selecting the best technique for specific deep learning applications to address the data scarcity issue. • Differences between Transfer Learning and Self-Supervised Learning. • Applications of Transfer Learning and Self-Supervised Learning. • Advantages and Limitations of Transfer Learning and Self-Supervised Learning. • Recommendations for the Use of Each Approach in Specific Applications. [ABSTRACT FROM AUTHOR]

Supervised and semi-supervised semantic segmentation algorithms require significant amount of annotated data to achieve a good performance. In many situations, the data is either not available or the annotation is expensive. The objective of this work is to show that by incorporating domain knowledge along with deep learning architectures, we can achieve similar performance with less data. We have used publicly available crack segmentation datasets and shown that selecting the input images using knowledge can significantly boost the performance of deep-learning based architectures. Our proposed approaches have many fold advantages such as low annotation and training cost, and less energy consumption. We have measured the performance of our algorithm quantitatively in terms of mean intersection over union (mIoU) and F-score. Our representative model, developed with 23% of the overall data; surpasses the baseline model (> 1.5% F-score gain) on the test data while matching the benchmark performance and, with a suitable augmentation strategy, outperforms the baseline model (11.5% average F-score gain) and the benchmark (> 8.75% average F-score gain) over the selected blind datasets. [ABSTRACT FROM AUTHOR]

The analysis of two-dimensional images enables the detection of structures, objects and their further classification. In the Internet of Things, images are data obtained from cameras or projections of signals. In this paper, we propose a hybridization of the classic RANSAC approach with a selected heuristic algorithm like Grey Wolf Optimizer. This solution allows for increasing the ability to adjust images by analyzing the detected key points of the image. Such hybridization is interesting in terms of the possibility of obtaining stitching accuracy. Moreover, we have introduced the automation of parameter selection to increase the accuracy and speed of the real-time algorithm. For this purpose, an analysis of the results of the algorithm for combining video frames into one large image allowed for a significant reduction in the amount of data processing time. The proposed algorithm was tested on various heuristic algorithms, where their differences and impact on the operation of the method were indicated and discussed. Moreover, the proposition was tested in practical application by automatically creating an image based on video frames recorded by the drone. • Hybridization of RANSAC with a heuristic solution for image stitching purposes. • A method of automating the selection of parameters in the algorithm. • Framework for minimizing the number of calculations in smart devices. • Practical application of the proposed method in IoT solution using a drone. [ABSTRACT FROM AUTHOR]

Lameness is one of the major health problems on dairy farms, which seriously affects dairy cow welfare and increases the risk of premature culling. Accuracy lameness detection ensures timely treatment of hooves and improves the level of health management on dairy farms. Most of the existing lameness detection methods detect one dairy cow, and it is difficult to detect lameness in multiple dairy cows at the same time. In this paper, a lameness detection system based on instance segmentation is constructed to extract the lameness features of multiple dairy cows and automatically detect lameness. First, an improved SOLOv2 network is designed to reduce the semantic gap between low-level and high-level features and improve the precision of dairy cow segmentation. Second, individual matching of dairy cows in different video frames is performed using the Hungarian algorithm. Third, the Canny algorithm is used to extract the neck and back contour features and key gait features of dairy cows. Finally, a deep learning model is constructed, and multiple lameness features are fused to detect the lameness of dairy cows. To evaluate the performance of the constructed system, 172 videos were randomly selected from 246 videos as training videos, and the remaining 74 videos were selected as the test videos. The lameness detection accuracy of the constructed system was 98.65%. The experimental results showed that the constructed system can extract the lameness features of multiple dairy cows at the same time and accurately detect the lameness of dairy cows. [ABSTRACT FROM AUTHOR]

Multi-band image fusion is a vital image processing technique used to combine data from multiple images or sensor bands, particularly in scenarios where a wide range of spectral information is available. Its primary objective is to create a single, comprehensive image that not only visually enhances the scene but also preserves critical details and characteristics present in each source. This paper proposes a novel image fusion technique that combines the useful features from a set of source images to provide a quick visualization of the scene. The approach is highly beneficial for multi-band images, including a large number of bands acquired by sampling the wavelength spectrum at narrow intervals. The fusion process depends on a multi-objective cost function, which optimizes the resultant fused image according to specific desired characteristics. By solving this optimization framework, an optimal set of weights is obtained using a novel Fusion-Driven Atom Search Optimization with Crossover (FASO-C) algorithm for the fusion of the image bands. Crossover operation enhances the exploitation capability of ASO, enabling the global optimal solution to be achieved quickly. To further validate the effectiveness of the proposed fusion model, various experiments were conducted to assess its performance using both subjective and objective metrics, as well as analyze its convergence. [ABSTRACT FROM AUTHOR]

Recently, the digitization of piping and instrumentation diagrams (P&IDs) has become increasingly necessary in the plant industry. In previous studies on line recognition in P&IDs, lines were classified according to line signs after line detection. However, detailed studies on the classification of continuous lines are limited. Herein, a rule-based method for classifying continuous lines in a P&ID is proposed. This method uses the shape and positional relationships between objects in the P&ID to classify continuous lines into eight types: drain, annotation, spec break, dimension, extension, leader, thick continuous, and continuous line. A prototype system was developed to validate the proposed method, and line classification experiments were conducted on actual high-density industrial P&ID systems. The experimental results indicated that the average precision, recall, and F1 score for the five P&IDs were 96.300%, 97.945%, and 96.415%, respectively. These results indicate the excellent performance of the proposed method in classifying continuous lines for various P&IDs. [ABSTRACT FROM AUTHOR]

The processing of noisy images does not provide accurate outcomes. In most existing techniques, many filters have been employed to eradicate the noises and enhance the image quality. However, most of the works fail to generate enhanced outcomes because of degraded capability and increased time consumption. Hence, in the proposed research work, a novel pre-processing method called the optimization assisted cascaded filtering approach (OptCFA) is introduced. In the proposed Opt_CFA model, the Gaussian Amended Bilateral filter (GABF) is designed where the multilevel thresholds are calculated for the GABF outcomes. The filter parameters are optimized using the Amended Pelican optimization algorithm (AmPel) for first stage noise removal. The outcomes are then cascaded with an Extended Savitzky-Golay filter (E_SGF) for second stage noise removal. The proposed Opt_CFA model is a single approach with the integration of GABF, AmPel and E_SGF methods. In the results section, some existing pre-processing filters, like the median, bilateral, and Gaussian filters, are analyzed for comparison. The individual performance of GABF and E_SGF methods used in the proposed model are also analyzed. Through the proposed technique, the noises can be extensively removed, and the image quality can be greatly enhanced. Several existing filters are compared with the proposed model, whereas the results are analyzed using PYTHON. [ABSTRACT FROM AUTHOR]

Machine vision has been increasingly used to address agricultural issues. One such case is corn field harvest losses and image-based object detection approaches, namely image processing, machine learning, and deep learning were investigated to detect and count infield corn kernels, immediately after harvest for combine harvester performance evaluation. A hand-held low-cost RGB camera was used to collect images with kernels of different backgrounds, based on which a 420 images dataset (200, 40, and 180 for training, validation, and testing, respectively) was generated. Three different models for kernel detection were constructed based on image processing, machine learning, and deep learning. For the imaging processing method, the images were preprocessed (color thresholding, graying, and erosion), followed by Hough circle detection to identify kernels. For the machine learning (cascade detector) and deep learning (Mask R-CNN, EfficientDet, YOLOv5, and YOLOX), models were trained, validated, and tested. Experimental results showed the overall performance of the deep learning network YOLOv5 was superior to the other approaches, with a small model size (89.3 MB) and a high model average precision (78.3 %) for object detection. The detection accuracy, undetection rate and F 1 value were 90.7 %, 9.3 %, and 91.1 %, respectively, and the average detection rate was 55 fps. This study demonstrates that the YOLOv5 model has the potential to be used as a real-time, reliable, and robust method for infield corn kernel detection. [ABSTRACT FROM AUTHOR]

Predictive learner modelling is crucial for personalized education. While convolutional neural networks (CNNs) have shown great success in education, their potential in learner modelling via image data is unexplored. This research introduces a novel and interpretable approach for Image-based Learner Modelling (ImageLM) using CNNs and transfer learning to model learners' performance and accordingly classify their computational thinking solutions. The approach integrates Grad-CAM, enabling it to provide insights into its decision-making process. Findings show that our custom CNN outperforms other models (namely ResNet, VGG, and Inception), with 83% accuracy in predicting solution correctness. More importantly, the ImageLM approach identifies the regions that contribute the most to the predictions, shedding light on learners' computational thinking knowledge and advancing toward trustworthy AI for education. These results underline the potential of utilizing imagery data from learners' activities during the learning process to predict their performance, especially in challenging environments like programming where traditional feature extraction and learning might struggle. [ABSTRACT FROM AUTHOR]

Imaging spectroscopy integrates traditional computer vision and spectroscopy into a single system and has gained widespread acceptance as a non-destructive scientific instrument for a wide range of applications. The current state of imaging spectroscopy spans diverse applications including but not limited to air-borne and ground-based computer vision systems. This paper presents the current state of research and industrial applications including precision agriculture, material classification, medical science, forensic science, face recognition and document image analysis, environment monitoring, and remote sensing, which can be aided through imaging spectroscopy. In this regard, we further discuss a comprehensive list of applications of imaging spectroscopy, pre-processing techniques, and spectral image acquisition systems. Likewise, publicly available databases and current software tools for spectral data analysis are also documented in this review. This review paper, therefore, could potentially serve as a reference and roadmap for people looking for literature, databases, applications, and tools to undertake additional research in imaging spectroscopy. • A survey on the current state of imaging spectroscopy. • Pre-processing techniques and data acquisition methods are highlighted. • We provide applications of imaging spectroscopy in various domains. • Publicly available databases and software tools for spectral data are presented. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) technology has significant potential in Earth sciences, particularly in mineral identification for industrial exploration, geological mapping, and archaeological research. However, traditional methods are time-consuming, expensive, and complex. And existing mineral identification methods based on mineral photos face several critical challenges, including lack of consideration for natural image features captured in real environments, limitations of single-label classification which does not align with multi-mineral occurrences in nature, and growing computational complexity as the number of identifiable mineral labels increases. Therefore, this paper proposes an efficient mineral identification model based on multi-label image classification, focusing on natural environmental features. First, realistic feature datasets are created by simulating mineral photos in real environments. Then, the model uses the query-label (Query2Label) framework, with MaxViT-T (Multi-Axis Vision Transformer-Tiny) as the feature extraction network and the asymmetric loss function. Knowledge distillation is employed to improve identification accuracy while reducing computational complexity. The proposed model achieves an impressive average identification accuracy of 84.74% on a dataset of 495,756 mineral photos, surpassing existing models like ResNet-101, ML-GCN (Multi-Label Graph Convolutional Network), and SRN (Spatial Regularization Net). It maintains a lower parameter count and computational complexity. In the end, ablation experiments demonstrate the effectiveness of each optimization scheme. • Selecting image noise and color shift methods for realistic mineral data simulation. • Proposing an enhanced two-stage image multi-Label identification model framework. • Applying knowledge distillation for multi-label classification subtask decomposition. • Being with low computational complexity and high mineral identification accuracy. [ABSTRACT FROM AUTHOR]

Feature extraction from planetary remote sensing images is a primary image processing task for object recognition, crater counter, morphological structure dimension and unpaired image co-registration. The paper presents a novel methodology to extract features directly from Bayer Pattern raw planetary images. Gradient information is extracted from Bayer Pattern image using standard edge operator that follows Color Difference Constancy (CDC) assumption. The proposed method's advantage is that it can skip the computationally intensive image processing pipeline and can have the Bayer Pattern raw planetary image flow directly for useful information extraction. Sobel Edge Detector is applied on Indian Mars Color Camera (MCC) Bayer Pattern raw images, and Gradient Magnitude Map (GMM) is generated at different Martian terrains. In addition, we have developed a direct image co-registration approach for MCC Bayer intensity raw image with respect to Mars Digital Image Model (MDIM) 2.1 reference using Mode-Mean Combo Patch Filler and Gradient Intensity induced Scale Invariant Feature Transform (GI-SIFT) based feature matching. The outlier matched points are removed by Feature Similarity Score guided Random Sample Consensus (FSS-RANSAC) estimation technique. The visual evaluation and quantitative metrics indicate that GMM from MCC Bayer Pattern raw image has negligible degradation with respect to GMM extracted from MCC demosaic image. The Root Mean Square Error (RMSE) is computed at different Mars regions, and it is found that the average image co-registration accuracy is less than 0.5 pixel. [ABSTRACT FROM AUTHOR]

• Cosmesis is an important outcome post breast conservation surgery and radiotherapy. • Objective evaluation of cosmesis uses handcrafted features from breast photographs. • A deep learning-based feature extraction method from breast region is proposed. • Novel ways to preprocess the regions before the feature extraction are presented. • The performance of our method is validated in several external datasets. Adverse cosmesis or poor aesthetic appearance of the breast after breast conservation surgery (BCS) and/or radiation therapy (RT) is highly correlated with quality-of-life (QOL) in breast cancer patients. Existing methods for objective assessment of cosmesis utilize human-designed features that are calculated using outlines and/or key-points from breasts. These are complicated and time consuming to obtain. Our purpose is to develop and validate a deep neural network (DNN)-based approach for classifying adverse cosmesis from frontal digital images of patients who have undergone BCS and RT using an approximate region-of-interest (ROI) containing the breast. We used 6 datasets (5 from BCS/RT-related clinical trials) to conduct our study. Training subsets (80% of patients) from 3 datasets (RAPID, OPAR, and PORTO) were used to train 3 independent support vector classifiers for classifying adverse cosmesis using the features extracted from preprocessed, approximate breast ROIs passed through a pretrained DNN, called ResNet-18. The trained classifiers/models were evaluated through 5 experiments: holdout validation, treatment-arm specific validation in RAPID, external validation (included datasets from the remaining 3 trials), assessment of the effect of preprocessing, and comparison with BCCT.core software. Area-under-the receiver operating characteristics (AUC) was used as the primary evaluation measure. Treatment-arm specific probability values were compared by the Mann-Whitney U test. AUC (0.78, 0.88, 0.94) values increased from RAPID to OPAR and PORTO respectively for holdout validation. The median probability of adverse cosmesis was higher in the same treatment arm in accordance with the interim results of the RAPID trial and differed significantly between the treatment arms (p = 0.006). For external validation, best AUC values were obtained by the model trained in RAPID (AUCs > 0.78). Preprocessing improved the performance to a large extent (average AUC: 0.87 vs 0.79 for internal validation, 0.78 vs 0.64 for external validation). AUCs of the individual models using DNN-based features were higher in 7 of 9 validation tests and statistically similar to the human-designed, dimensionless features extracted from BCCT.core software. Appropriately preprocessed, approximate breast ROIs can be represented by DNN-based features to identify adverse cosmesis from frontal images of breast cancer patients treated with BCS/RT. Adaptations to this approach may be necessary for its applicability to images with non-frontal views and different quality. This work serves as a foundation for developing robust DNN-based platforms for evaluating cosmesis in clinical trials. [ABSTRACT FROM AUTHOR]

Image matching involves identifying the template region in the search image after selecting it in the template image. Image matching, as a basic task, is a pre-order step of many image processing. Because of the different imaging principles of visible and infrared images, the difference between visible and infrared images is large. Compared with the matching of visible images, it is more difficult to match visible and infrared images. Due to the characteristics of infrared images, many targets in infrared images do not have clear outlines, which makes it difficult to distinguish targets and backgrounds in infrared images during the matching process. Regarding the issue above, we propose a network named VINet for visible infrared image matching, and design a feature extraction network AM-Net based on Inception-v3. In order to improve the feature expression ability of the network, we added a parameter-free attention mechanism to the AM-Net network, which improved the network expression ability without introducing new parameters. In addition, we added DropBlock to the AM-Net network, which can achieve better regularization through the DropBlock. To accurately differentiate the background from the target and obtain the target position, we integrate target-aware graph attention in VINet and employ the CIoU loss during training. To address the limited number of visible and infrared image matching datasets, we repurpose existing datasets by relabeling them to obtain new training and testing sets. The experimental results indicate that our method can achieve better matching results compared with other state-of-the-art methods. • We proposed a network called VINet for visible and infrared image matching. • AM-Net we designed can effectively extract the features of visible-infrared images. • Target-aware graph attention and CIoU loss can improve matching performance of VINet. • Our reconstructed data sets contains abundant features. • VINet gets the highest mAP on the all test images. [ABSTRACT FROM AUTHOR]

• An efficient dataset synthesis method for occluded rockfill particle image segmentation. • An occlusion-aware gradation detection method based on amodal instance segmentation. • A comparison between occlusion-aware and ordinary image-based gradation detection. Image-based gradation detection methods of rockfill materials mostly ignore occluded regions of particles, and for the mainstream methods with deep learning, dataset annotation is time-consuming and labour-intensive. This study proposes an efficient virtual-to-real dataset synthesis method for rapid dataset synthesis and occlusion-aware gradation detection. Instead of photographing or scanning real particles with large time cost and labour input, Diffusion-GAN, which is trained with 600 virtual images generated by 3D modeling, efficiently generates 50,000 various individual particle images to synthesize initial stacked particle images and amodal annotations. Post-processing CycleGAN is subsequently proposed to preserve the background with image processing based on CycleGAN, which superiorly converts the style of synthetic images from virtual to real. The proposed dataset synthesis method is 50 times faster than manual labelling. Occlusion-aware gradation detection employs Bilayer Convolutional Network (BCNet) to predict both visible and occluded areas of particles, whose maximum absolute error is 4.72%, less than the error of 7.73% produced by ignoring occluded regions. The AP 50 of BCNet trained with the synthetic dataset is 0.941, extraordinarily close to the result trained with the real dataset. [ABSTRACT FROM AUTHOR]

This paper presents an horizontal technique to classify the main directions of orientation of the planar surfaces that conform solid bodies. When mathematically representing these surfaces via 3D point clouds, a cascade or chain process using combined information of the point positions and of their detected normal vectors is applied to solidly identify the planar surfaces composing them and to retrieve their orientations. This technique distinguishes itself from others in the field due to (i) a specific filtering inspired by image processing techniques that aims to mitigate the effects that surface rugosity-induced mathematical noise can induce on the classification; and (ii) also due to the data representation on the unit-sphere using the von Mises–Fisher mixture model. The extents and limits of the proposed process are presented using several validation cases of increasing complexity, and its results and possibilities for horizontal applications over many kinds of real-life bodies are also analysed. • A system joining cross-field techniques classifies orientations in 3D point clouds. • Rugosity in real surfaces equals image noise affecting the orientations. • Denoised sphere orientations is a robust framework for orientation segmentation. • This technology combination provides an expert solution applicable to different fields. [ABSTRACT FROM AUTHOR]

With the development of vision technology, image set classification (ISC) has flourished in the image processing field. Different from the one-shot image classification, ISC focuses on the set rather than a one-shot image. Hence, ISC can synthesize the abundant set information to alleviate various appearance variations. Despite the great success of the existing ISC methods, there are still some problems: (1) They usually face an expensive time complexity, which directly limits the practical application; (2) They largely ignore the intrinsic relationships between different sets. In light of this, we propose a novel Discrete Aggregation Hashing (DAH) for fast ISC. To be specific, to extract more semantic information from each set and each sample, we adopt the same projection standard to embed dual semantic labels (i. e., sample label and set label) into instance and set hash codes. Then we regard set hash codes as set-specific centers. A hashing aggregation strategy is proposed to learn compact discriminative instance hash codes via iteratively aggregating intrinsic neighborhood representations around each central node. Therefore, instance hash codes can obtain greater intra-set compactness and inter-set separability. Extensive experiments demonstrate that our DAH can obtain promising performance and outperform these state-of-the-art ISC methods on four image set datasets. [ABSTRACT FROM AUTHOR]

To make the face hallucination process robust to impulse noise, a new outlier elimination based iterative neighbor representation (OEINR) algorithm is proposed in this work. The proposed algorithm computes the fidelity in the low-resolution (LR) space and the locality adaptor in the high-resolution (HR) space. It helps the proposed algorithm in obtaining better reconstruction weights and preserving individual characteristics of an input in the output faces. To compute the HR space-based locality adaptor, an HR face (named the assisting face) is generated by proposing a new objective function utilizing the merits of sparsity. Further, the iterative approach is applied to continuously improve the quality of the assisting face which succors in achieving better locality constraint, and hence better reconstruction weights. Moreover, outliers (impulse noise-contaminated pixels) are eliminated from weights calculation as well as the assisting face generation process which helps the proposed algorithm in minimizing the effect of impulse noise. The results obtained from the experiments conducted on public FEI, CAS+PEAL, CMU+MIT face databases, and surveillance videos and images exhibit the superiority of the proposed OEINR algorithm over existing comparative face hallucination methods. • Outlier elimination-based iterative neighbor representation algorithm is developed. • Fidelity in the LR and locality adaptor in HR space is computed. • Iterative approach is applied to continuously improve the performance. • The results exhibit the superiority of the proposed OEINR algorithm. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have achieved impressive success in the multi-modal image processing (MIP) area. However, many existing CNN approaches fuse the features of the target and guidance images only once, which may cause a loss of information. To alleviate this problem, we present a multi-level bilateral interactive attention network (MBIAN) to fuse the features of the target and guidance images by their progressive interaction at different levels. Concretely, for each level, a bilateral interactive attention block (BIAB) is proposed to fuse the information of target and guidance images and refine their features. As the core component of our BIAB, a novel bilateral interactive attention layer (BIAL) is designed, where target and guidance images can mutually determine the attention weights. In addition, in each BIAB, long and short local shortcuts are employed to further facilitate the flow of information. Numerical experiments are conducted for three different problems, including panchromatic guided multi-spectral image super-resolution, near-infrared guided RGB image denoising, and flash-guided no-flash image denoising. The results demonstrate the versatility and superiority of MBIAN in terms of quantitative metrics and visual inspection, against 14 popular and state-of-the-art methods. • Bilateral interactive attention block is a novel progressive interactive method. [ABSTRACT FROM AUTHOR]

The habitats of marine life, characteristics of species, and the diverse mix of maritime industries around these habitats are of interest to many researchers, authorities, and policymakers whose aim is to conserve the earth's biological diversity in an ecologically sustainable manner while being in line with indispensable industrial developments. Automated detection, locating, and monitoring of marine life along with the industry around the habitats of this ecosystem may be helpful to (i) reveal current impacts, (ii) model future possible ecological trends, and (iii) determine required policies which would lead accordingly to a reduced ecological footprint and increased sustainability. New automatic techniques are required to observe this large environment efficiently. Within this context, this study aims to develop a novel platform to monitor marine ecosystems and perform bio census in an automated manner, particularly for birds in regional aerial surveys since birds are a good indicator of overall ecological health. In this manner, a new non-parametric approach, WILDetect, has been built using an ensemble of supervised Machine Learning (ML) and Reinforcement Learning (RL) techniques. It employs several hybrid techniques to segment, split and count maritime species – in particular, birds – in order to perform automated censuses in a highly dynamic marine ecosystem. The efficacy of the proposed approach is demonstrated by experiments performed on 26 surveys which include Northern gannets (Morus bassanus) by utilising retrospective data analysis techniques. With this platform, by combining multiple techniques, gannets can be detected and split automatically with very high sensitivity (Se) (≈ 0.97), specificity (Sp) (≈ 0.99), and accuracy (Acc) (≈ 0.99) — these values are validated by precision (Pr) (≈ 0.98). Moreover, the evaluation of the system by the APEM staff, which uses a completely new evaluation dataset gathered from recent surveys, shows the viability of the proposed techniques. The experimental results suggest that similar automated data processing techniques – tailored for specific species – can be helpful both in performing time-intensive marine wildlife censuses efficiently and in establishing ecological platforms/models to understand the underlying causes of trends in species populations along with the ecological change. • This study aims to develop a methodology for detecting maritime bio ecosystems. • The methodology employs Machine Learning and Reinforcement Learning techniques. • It performs bio census automatically, particularly for birds. • More explicitly, It employs several hybrid techniques to segment, split and count birds. • The effectiveness of the approach is demonstrated with aerial surveys. [ABSTRACT FROM AUTHOR]

As one of the challenging tasks in the remote sensing (RS), object detection has been successfully applied in many fields. Convolution neural network (CNN) has recently attracted extensive attention and is widely used in the natural image processing. Nevertheless, RS images have cluttered scenes compared with natural images. As a result, the existing detectors perform poorly in RS images, especially with the complicated backgrounds. Moreover, the detection inference time and model volume of detectors in RS images often go unrecognized. To address the above issues, this study proposes a novel method for object detection in RS images, which is called the consistency- and dependence-guided knowledge distillation (CDKD). To this end, the spatial- and channel-oriented structure discriminative modules (SCSDM) are put forward to extract the discriminative spatial locations and channels to which the teacher model pays attention. SCSDM improves the feature representation of the student model by effectively eliminating the influence of noises and the complicated backgrounds. Then, the consistency and dependence of the features between the teacher model and the student model are constructed under the guidance of SCSDM. Experimental results over public datasets for RS images demonstrate that our CDKD method surpasses the state-of-the-art methods effectively. Most of all, on the RSOD dataset, our CDKD method achieves 92% mean average precision with 3.3 M model volume and 588.2 frames per second. [ABSTRACT FROM AUTHOR]

Sparse regularization has been attracting much attention in industrial applications over the past few decades. By exploiting the latent data structure in low-dimensional subspaces, a significant amount of research achievements have been realized in signal/image processing, pattern recognition and system identification, etc. However, very few systematic review or comprehensive survey are reported for sparse regularization including fundamentals, state-of-the-art methodologies, and applications on fault diagnosis. To fill this gap, this article conducts an in-depth review of the state-of-the-art technologies of sparse regularization, and the R&D of sparse regularization applied to fault diagnosis will also be summarized. Specifically, we discuss the rationales of cause formulation, algorithm idea, algorithm merits, algorithm demerits and computing techniques for each category. The availability and practicability of several representative models of sparse regularization are investigated with real-world experimental datasets. Finally, benefiting from theoretical developments of the sparse regularization, open/upcoming challenges, instructive perspectives, as well as possible future trends of the sparse regularization for prognostic and health management (PHM) are discussed. [ABSTRACT FROM AUTHOR]

While the recurrent neural network (RNN) has achieved remarkable performance on dynamic and control tasks, its applications to image processing, particularly target detection are limited. Challenges arise from differences between the two domains, such as the way for merging time information into static problems and variances of dynamic and static solving methods. To this end, we first extend the existing constrained energy minimization (CEM)-based detection scheme to a dynamic version, e.g. dynamic reinforced CEM (DRCEM), which injects the dynamic information. After that, aided by the rigorous mathematical derivation and optimization theory, the DRCEM is merged into the RNN solution framework. To enhance the robustness and convergence of the existing RNN solutions for improving DRCEM performance, the nonlinear and bounded-constraint RNN (NBCRNN) is designed by developing a novel nonlinear activation function, then applying the proposed model to implement the DRCEM scheme. The corresponding theorem results reveal the proposed model possesses global convergence and enhanced robustness. Compared to state-of-the-art works, the DRCEM solved by the NBCRNN model detection method achieves better detection accuracy, with 1.82% improvement in terms of the Kappa coefficient, and reduces the residual error from 1 0 − 4 to 1 0 − 7 . Furthermore, our detection method is able to preserve the detection accuracy in presence of noise perturbated. To the best of our knowledge, it is the first work to develop the zeroing-type RNN for hyperspectral image target detection. The code and models are publicly available at Github DRCEM_NBCRNN Code Implementation. • A novel hyperspectral image target detection scheme is proposed. • Detection scheme possesses more accuracy with less prior information. • A neural network is designed to improve detection performance. • The proposed neural network can achieve better performance. [ABSTRACT FROM AUTHOR]

Crop row detection is the basis for the visual navigation of agricultural machinery. Previous research has typically developed crop detection schemes based on specific application objects, with cumbersome image processing steps. A compact and efficient deep learning-based crop row detection network, named E2CropDet, is proposed in this study. E2CropDet models each crop row as a complete and independent object to enable an end-to-end detection manner with no involvement of image post-processing. Generic backbones are used as feature extractors in E2CropDet. Line-shaped proposals are developed as pre-defined detection anchors based on the shape characteristics and distribution pattern of crop rows. The feature vector obtained by pooling along the slender crop rows is fed into fully connected layers after aggregating contextual information. Then the final centreline extraction results are obtained by classification and regression. With ResNet-34 as the backbone, the proposed model results in a lateral deviation of 5.945 pixels for centerline extraction, which exceeds the semantic segmentation-based (7.153) and the Hough transform-based (17.834) approaches. Additionally, benefiting from an end-to-end pipeline that requires no post-process, it achieves a remarkable detection speed of 166 frames per second. [ABSTRACT FROM AUTHOR]

• We develop a strategy to map aerial views and plan safe routes for ground robots. • The strategy is capable of determining whether target locations are reachable or not. • We propose a new method, dataset and metrics to evaluate path planning strategies. • We consistently define the quality of a strategy in terms of accuracy and timing. • The evaluation method can serve as a standard for comparing path planning procedures. Autonomous robot navigation consists of establishing a sequence of actions that guarantee safe and efficient locomotion for a robotic system. This work focuses on outdoor ground robot navigation guided by aerial images. We propose a pipeline of image processing, traversability estimation, terrain representation and route planning capable of assessing whether a desired target location is reachable or not. We also propose a dataset and an evaluation method to quantify the average quality of generated paths and measure how well a planning strategy finds possible routes and rejects impossible ones. This evaluation was applied to identify a parameter combination that maximized path feasibility recognition and average path quality. Our results show that the proposed strategy identifies the correct action in 85.8% of the test cases and takes an average of 2.3 s for mapping and 100 ms for path planning over 300 × 300 m2 areas in images of size 1000 × 1000 pixels. We successfully employed our proposed method in a small scale experiment to guide our Hydra v3 mini rover. [ABSTRACT FROM AUTHOR]

• A novel deep multibranch multitask neural network architecture. • The different branches process the input image at different scales. • A trainable crop strategy to feed branches with the most informative regions. • The injection of hand-crafted features inside the network for painting categorization. • A new dataset composed of 100k paintings from 1508 artists, 125 styles, 41 genres. We propose a novel deep multibranch and multitask neural network for artist, style, and genre painting categorization. The multibranch approach allows us to exploit at the same time the coarse layout of the painting and the fine-grained structures by using painting crops at different resolutions that are wisely extracted using a Spatial Transformer Network trained to identify the most discriminative subregions of paintings. The effectiveness of the proposed network is proved in experiments that are performed on a new dataset originally sourced from wikiart.org and hosted by Kaggle, and made suitable for artist, style and genre multitask learning. The dataset here proposed and made available for research is named MultitaskPainting100k , and is composed by 100K paintings, 1508 artists, 125 styles and 41 genres annotated by human experts. Among the different variants of the proposed network, the best method achieves accuracy levels of 56.5%, 57.2%, and 63.6% on the MultitaskPainting100k dataset for the tasks of artist, style and genre prediction respectively. [ABSTRACT FROM AUTHOR]

• A deep learning based method for detecting QRS complexes. • A fully convolutional network model with a large receptive field. • A labeling process to generate ground-truth data. • Postprocessing to improve the accuracy of QRS detection. • Performance of the proposed method verified with real data. A capacitive electrocardiogram (cECG) signal is considered a promising alternative to a conventional contact electrocardiogram (ECG) signal because the cECG signal can serve the same purpose as the contact ECG signal but can be measured during daily life without causing a subject to feel uncomfortable. However, the cECG signal has a limitation in that detection of QRS complexes, which is a fundamental step to analyze heart condition, is not easy. That is because the cECG signal is sensitive to noise, especially motion noise. This paper proposes a method to detect QRS complexes in cECG signals degraded by motion noise. The proposed method is based on fully convolutional networks (FCNs) and mainly consists of three parts: the generation of ground-truth data, the FCN model, and postprocessing. A labeling process for generating the ground-truth data is proposed. Then, an FCN model that is suitable for cECG signals is proposed. The proposed FCN model uses filters of a large size to achieve a large receptive field, unlike the common FCN models used in image processing. The receptive field is sufficiently large to involve information about adjacent QRS complexes, such as the time interval between the QRS complexes and its variability. By considering the information, the proposed FCN model can reliably classify QRS complexes even in cECG signals degraded by motion noise. Additionally, postprocessing, which consists of an accumulation step and a non-maximum suppression step, is proposed to complement the proposed FCN model. In experiments with real data, the proposed method showed an average sensitivity of 96.94%, positive predictive value of 99.13%, and F1 score of 98.02%. These results demonstrate that the proposed method overcomes the limitation of a cECG signal and helps the cECG signal be widely utilized for medical or healthcare applications. [ABSTRACT FROM AUTHOR]

• Effective strategies for noisy image segmentation using adaptive wavelet shrinkage. • Edge enhancement in wavelet domain improve the results. • Significantly better performance than other methods on severely noisy images. • No parameter-tuning required for different noise levels and types • Very consistent results even when the noise level has a large variation. Noisy image segmentation is a hot topic in natural, medical, and remote sensing image processing. It is among the non-trivial problems of computer vision having to address denoising and segmentation at the same time. Fuzzy C-means (FCM) is a clustering algorithm that has been shown to be effective at dealing with both segmentation-oriented denoising and segmentation at the same time. Moreover, with a high level of noise and other imaging artifacts, FCM loses its ability to perform image segmentation effectively. This paper introduces a Particle Swarm Optimization (PSO)-based feature enhancement approach in the wavelet domain for noisy image segmentation. This approach applies adaptive wavelet shrinkage using FCM clustering performance as an evaluation mechanism and also as the segmentation algorithm. The PSO-based process helps to enhance intensity features for clustering-based denoising, and also provides adaptivity for the system that performs well on a range of real, synthetic, and simulated noisy images with different noise levels and range/spatial properties. Furthermore, the algorithm applies edge enhancement based on Canny edge detector in order to further improve accuracy. Experiments are presented using three different datasets each degraded with different types of common noise. The presented algorithms show effective and consistent performance over a range of severe noise levels without the need for any parameter tuning. [ABSTRACT FROM AUTHOR]

Highlights • Novel approach to automated medical x-ray image evaluation. • Simplified x-ray image descriptors tailored for respiratory diseases detection. • Devoted fitness condition for x-ray image processing. • Tailored neural and heuristic approaches for x-ray images processing. • Model of intelligent expert system devoted for automated assistance. Abstract Background and objective The X-ray screening is one of the most popular methodologies in detection of respiratory system diseases. Chest organs are screened on the film or digital file which go to the doctor for evaluation. However, the analysis of x-ray images requires much experience and time. Clinical decision support is very important for medical examinations. The use of Computational Intelligence can simulate the evaluation and decision processes of a medical expert. We propose a method to provide a decision support for the doctor in order to help to consult each case faster and more precisely. Methods We use image descriptors based on the spatial distribution of Hue, Saturation and Brightness values in x-ray images, and a neural network co-working with heuristic algorithms (Moth-Flame, Ant Lion) to detect degenerated lung tissues in x-ray image. The neural network evaluates the image and if the possibility of a respiratory disease is detected, the heuristic method identifies the degenerated tissues in the x-ray image in detail based on the use of the proposed fitness function. Results The average accuracy is 79.06% in pre-detection stage, similarly the sensitivity and the specificity averaged for three pre-classified diseases are 84.22% and 66.7%, respectively. The misclassification errors are 3.23% for false positives and 3.76% for false negatives. Conclusions The proposed neuro-heuristic approach addresses small changes in the structure of lung tissues, which appear in pneumonia, sarcoidosis or cancer and some consequences that may appear after the treatment. The results show high potential of the newly proposed method. Additionally, the method is flexible and has low computational burden. [ABSTRACT FROM AUTHOR]

Highlights • We propose a multi-objective version of the Multiverse Optimizer (MOMVO). • The problem of image thresholding is solved by a multi-objective algorithm. • The Otsu and Kapur's methods are combined in a multi-objective problem. • The MOMVO is tested over different images for thresholding. • Comparisons support the performance of the proposed MOMVO. Graphical abstract Abstract Image segmentation is among the most important techniques in image processing, and many methods have been developed to perform this task. This paper presents a new multi-objective metaheuristic based on a multi-verse optimization algorithm to segment grayscale images via multi-level thresholding. The proposed approach involves finding an approximate Pareto-optimal set by maximizing the Kapur and Otsu objective functions. Both Kapur's and Otsu's methods are highly used for image segmentation performed by means of bi-level and multi-level thresholding. However, each of them has certain characteristics and limitations. Several metaheuristic approaches have been proposed in the literature to separately optimize these objective functions in terms of accuracy, whereas only a few multi-objective approaches have explored the benefits of the joint use of Kapur and Otsu's methods. However, the computational cost of Kapur and Otsu is high and their accuracy needs to be improved. The proposed method, called Multi-objective Multi-verse Optimization, avoids these limitations. It was tested using 11 natural grayscale images and its performance was compared against three of well-known multi-objective algorithms. The results were analyzed based on two sets of measures, one to assess the performance of the proposed method as a multi-objective algorithm, and the other to evaluate the accuracy of the segmented images. The results showed that the proposed method provides a better approximation to the optimal Pareto Front than the other algorithms in terms of hypervolume and spacing. Moreover, the quality of its segmented image is better than those of the other methods in terms of uniformity measures. [ABSTRACT FROM AUTHOR]

The environment sensing and image processing are integrated with the Internet of Things (IoT), which is considered a novel platform for monitoring plant health. The main limitation of the existing approaches is the inability to accurately classify root diseases. This paper presents a novel model for categorizing the root infection in the IoT network. The goal of the proposed model is to enhance crop productivity by controlling the plantation platform at less cost. Here, the Remora chicken swarm optimization with SqueezeNet (RCSO-based SqueezeNet) is employed for categorizing the root disease. The SqueezeNet training is done using RCSO, which is formed by integrating the Remora optimization algorithm (ROA) and chicken swarm optimization (CSO). This model assists to mine the hidden knowledge from images and attains high accuracy, while classifying the disease present in the roots. Here, IoT is incorporated wherein the IoT nodes help to accumulate the information of roots and sends this information to the base station (BS), wherein the classification of root disease is achieved. The RCSO-based SqueezeNet outperformed with the highest energy of 0.176 J, Packet delivery ratio (PDR) of 0.590, smallest distance of 12.476 m, highest accuracy of 91.2%, sensitivity of 93.1% and specificity of 90.9%. [ABSTRACT FROM AUTHOR]

Many applications of image processing, statistical modeling of frequency domain coefficients is considered by researchers as an approach. In this paper, we present a probabilistic graphical model based on the Vine Copula approach called SherVine(Shearlet Vine) to model the shearlet transform coefficients jointly. The proposed model is an extension of joint modeling with the help of Copula's theory, which can diversely model a pair of shearlet coefficients. Also, the graphical representation of SherVine leads to a better understanding of dependencies types between shearlet coefficients. In the second part of this paper, we propose a novel image quality assessment(IQA) approach based on SherVine called IQA-SherVine. IQA-SherVine uses the parameters of the SherVine model as a feature in the IQA framework. The extracted features are modeled using the decision tree algorithm in IQA-SherVine. The results show that IQA-SherVine performs very well compared to the statistical and deep learning based state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a new large-scale publicly available color constancy dataset which we are calling the Shadows & Lumination dataset. The dataset contains 2500 minimally processed images from various indoor, outdoor, and night-time scenes. This dataset is GDPR-compliant, as we masked out all sensitive private information from the images. Unlike most other color constancy datasets, our dataset contains real-world images with two illuminants is appropriate for multi-illuminant estimation. In addition to the illumination, we provide a binary segmentation mask for each image. In the segmentation mask, we divide the image into two regions, where each region is illuminated by only one of the illuminants. We give an explanation of the methodology used to create the dataset. For dataset creation, we used five cameras: Canon 5D, Canon 550D, Sony α 300 , Panasonic FZ1000, and the Motorola one fusion+ mobile camera. Finally, we tested several state-of-the-art illumination estimation and image segmentation models on our dataset. The dataset is publicly available 1 1 bit.ly/shal_dataset Direct link.. This paper also benchmarks several illumination estimation methods as well as several image segmentation methods on our dataset. • Large-scale public dataset for two-illuminant color constancy. • Detailed manually annotated per pixel illumination segmentation mask included. • Image acquisition and labeling methodology provided. • Existing state-of-the-art methods were tested on the new dataset. [ABSTRACT FROM AUTHOR]

Digital watermarking is of critical importance in multimedia copyright protection. However, with the development of Internet technologies, the combined attacks of complex environments challenge to the robustness of traditional digital watermarking schemes. In order to tackle this problem, a robust periodic blind watermarking scheme based on sub-block mapping and block encryption is proposed in this paper to enhance robustness under combined attacks. Concretely, a watermark is encoded by the periodic encoding method to raise the fault tolerance rate. Then a block encryption technique is designed to encrypt periodic sub-blocks by utilizing different encoding methods to maintain the security of copyright information. To achieve high image quality, this scheme employs a sub-block mapping mechanism to map the encrypted periodic watermark into a smaller watermark to minimize the visual difference between host image and watermarked image. Finally, an improved Singular Value Decomposition (SVD) is conducted to embed a quaternary watermark that can extend to a gray-scale watermark. Experimental results demonstrate that the proposed watermarking scheme has good visual quality of watermarked images and is robust to image processing attacks, especially in cropping, scaling and combined attacks, which is better than other watermarking schemes. [ABSTRACT FROM AUTHOR]

In the leather industry, identifying species of leather holds a significant step toward consistent global leather trade. Intertwining image processing and a learning algorithm with leather science can enhance the predictability of leather species. Hence, this paper aims to learn the pore-pattern variability between each species from digital microscopic leather images. These images undergo image pre-processing to generate leather images with highlighted pores, less susceptible to noise. This work also proposes an Entropy-based Otsu's thresholding with Component-area-histogram Analysis (E O C A) to achieve an adequate hair-pore segmentation, irrespective of any species. Goodness and discrepancy measures validate the generosity of the proposed E O C A method. Morphological, geometrical, and statistical features estimate the pattern-variability of each species. To ascertain the discriminatory behavior of these features, this work performs the feature classification using K N N , N B , D T , S V M , and M L P classifiers. The classification accuracy signifies the efficiency of the pre-processing and the proposed E O C A method in estimating species-definite features. The performance comparison determines M L P with 98.75% accuracy as an appropriate leather species learning model. Thus, the present work contributes to automatic leather species identification by learning and interpreting species-definite features. It also lays the design and development of a human-machine interactive platform to revolutionize the leather trade. [ABSTRACT FROM AUTHOR]

Image processing filter intellectual property (IP) cores perform different critical operations on images such as blurring, sharpening, edge detection, etc., which are essential functions in several multimedia systems. The design process of the image filter cores as a dedicated IP needs to consider multiple objectives such as area, delay, and security against various hardware threats such as IP piracy (counterfeiting and cloning) and fraudulent claim of IP ownership during design space exploration (DSE) in high level synthesis (HLS). This paper presents the following novelties: (a) particle swarm optimization (PSO) driven exploration of secured image processing filter IP core datapath architecture during HLS capable of providing detective control against IP piracy threat (b) the proposed approach incorporates embedding of covert security constraints based on the proposed multi-phase encryption algorithm during performing area-delay tradeoff via PSO based DSE. The proposed approach achieves stronger security in terms of digital proof of > 2X and higher tamper tolerance as well as attains a lower design cost of 56.92% in terms of area-delay than recent approaches. The proposed approach also achieves optimal solution as evident from the optimality analysis performed on metrics such as generational distance, spacing, spreading, and weighted sum. [ABSTRACT FROM AUTHOR]

Nowadays, the potential benefits and implementation of autonomous driving have attracted widespread attention from both industry and academia. This study will solve view-invariant object detection and semantic key-point pose assumptions from a single RGB image. A machine learning method for estimating the absolute pose of an on-road vehicle for autonomous driving from monocular vision alone without the help of additional sensors is a complex task. The main purpose of this work is to identify other vehicles on the road and estimate their exact angular position from a single image with improved accuracy. The focus of the study is to create a new algorithm by applying a potentially deep convoluted neural network followed by a repetitive neural structure for more accurate 6D pose inference. A 6D pose hypothesis is presented in this study, based on a deep hybrid architecture for individual vehicles of an end-to-end approach to a task consisting of a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN). In this work, we will use a large-scale dataset consistent with the understanding of a 3D car instance called ApolloCar3D. The data set contains 5,277 real-life street scenes with examples of about 60K cars. By comparison, the ApolloCar3D is twenty times larger than the PASCAL3D+ and KITTI datasets. Ultimately, the idea is to efficiently eliminate motionless cars and predict the next pose given in the speed context, allowing a comprehensive evaluation, and passing the output through LSTM (long short-term memory) with an additional filter layer. The new filter added to the LSTM will efficiently filter and isolate stationary or parking vehicles and focus on on-road vehicles. Since the LSTM has a non-linear high-dimensional hidden memory state, it can preserve the past continuity of each generation's data history and pay more attention to those road vehicles rather than parked or stationary vehicles to act accordingly. So for each new vehicle, the pose estimator classifier can use LSTM memory and compare the historical pose with the newly filtered data. The successful implementation of this innovative concept will lead to significant improvements in the real-life traffic situation in the field of computer vision and autonomous driving. [ABSTRACT FROM AUTHOR]

Highlights • The proposed systems explore different Gabor orientations for locating flat regions. • Candidate pixels are detected for flat regions with uniform Gabor orientations. • The Markov Chain Code process is used in a new way for groping candidate pixels. • The Chi square similarity measure has been used for finding safe landing zone. • The proposed system outperforms the existing systems in terms of recall, precision and F-measure. Abstract As the demand increases for the use Unmanned Aerial Vehicles (UAVs) to monitor natural disasters, protecting territories, spraying, vigilance in urban areas, etc., detecting safe landing zones becomes a new area that has gained interest. This paper presents an intelligent system for detecting regions to navigate a UAV when it requires an emergency landing due to technical causes. The proposed system explores the fact that safe regions in images have flat surfaces, which are extracted using the Gabor Transform. This results in images of different orientations. The proposed system then performs histogram operations on different Gabor-oriented images to select pixels that contribute to the highest peak, as Candidate Pixels (CP), for the respective Gabor-oriented images. Next, to group candidate pixels as one region, we explore Markov Chain Codes (MCCs), which estimate the probability of pixels being classified as candidates with neighboring pixels. This process results in Candidate Regions (CRs) detection. For each image of the respective Gabor orientation, including CRs, the proposed system finds a candidate region that has the highest area and considers it as a reference. We then estimate the degree of similarity between the reference CR with corresponding CRs in the respective Gabor-oriented images using a Chi square distance measure. Furthermore, the proposed system chooses the CR which gives the highest similarity to the reference CR to fuse with that reference, which results in the establishment of safe landing zones for the UAV. Experimental results on images from different situations for safe landing detection show that the proposed system outperforms the existing systems. Furthermore, experimental results on relative success rates for different emergency conditions of UAVs show that the proposed intelligent system is effective and useful compared to the existing UAV safe landing systems. [ABSTRACT FROM AUTHOR]

Highlights • A three step combination of different Neural Networks for discrimination. • The algorithm works by decomposing and remembering previously presented images. • The training of each of the networks is separated according to each of its properties. • The algorithm perform better at discrimination than individual methods. • The algorithm can remember longer sequences than individual methods. Abstract Although done nearly effortlessly by humans, digital systems cannot easily recognize images or predictions from recent observations. Tackling these limitations by proposing novel algorithms to improve the performance of image processing would have widespread implications in a variety of fields, including robotics, manufacturing, biomedicine, and automation. To provide a computer with this combined ability and transform it into an intelligent system, an algorithm must combine memory with an image decomposition procedure. Artificial neural networks (ANNs) are algorithms that aim to solve tasks such as classification, clustering, pattern recognition, and prediction by resembling brain connections. Specifically, three ANNs have excelled in specific areas: deep neural networks (DNNs), which use intrinsic connections to create prediction maps; long short-term memory neural networks (LSTMs), which use recurrent connections to emulate a type of memory; and convolutional neural networks (CNNs), which can decompose complex data through layers for simpler analysis. Although these algorithms can solve certain tasks of image sequence prediction, they cannot easily solve entire problems on their own. Nevertheless, combining these networks may enable solving such problems with ease. Thus, this article evaluates the combination of ANNs into two novel algorithms developed with the aim of improving image sequence prediction: (i) a combination of CNNs and LSTMs to form a CLNN and (ii) a combination of CNNs, LSTMs, and DNNs to form a CLDNN. Although the developed algorithms require a longer training time, they require less training epochs to have better accuracy than their predecessors. Furthermore, both developed methods were capable of accurately performing the image sequence prediction task, outperforming each individual method, as well as predicting longer and greater numbers of sequences correctly. Overall, the developed algorithms were able to better decompose inputs, remember previous inputs, and more accurately predict sequences of images. This allows the prediction of the next step in the sequence, which can be used as part of an intelligent system to make an analysis and an informed decision on the next course of action. [ABSTRACT FROM AUTHOR]

Highlights • A new sorting method based on VIKOR is proposed. • The proposed method is employed to evaluate the suppliers’ environmental performance. • The results proved the efficiency of the proposed method. • The method has a great potential to be applied in different domains. Abstract Depleting natural resources and limited amount of landfill areas have forced many governments to impose stricter measures on environmental performance. In order to comply with those measures and to have a better environmental image, companies are investing heavily in environmental, social and economic responsibility issues. Moreover, they continuously track the environmental performance of their suppliers. Many green supplier evaluation and ranking methodologies were proposed in the literature in order to assist companies in the environmental performance evaluation of suppliers. However, the number of studies on the sorting of suppliers based on environmental criteria is very limited. In this study, we fill this research gap by proposing a novel VIKOR-based green supplier sorting methodology called VIKORSORT. This methodology evaluates the environmental performance of suppliers and sorts them into the predefined ordered classes. The proposed methodology can easily be embedded into an expert system which can suggest a suitable green supplier development program for each class. [ABSTRACT FROM AUTHOR]

Highlights • The CNN model, with the circular Hough transformation, was applied for the first time on retinal images. • The CNN model was used together with image processing methods to increase the success rate. • The success rate of exudate detection is 99.18%. • Developing a clinical decision support system to aid practitioners. Abstract In this study, a combined approach of circular Hough transform and Convolutional Neural Network (CNN) algorithms was proposed for detecting exudates, which is one of the signs of diabetic retinopathy disease. The proposed approach was assessed using DiaretDB0, DiaretDB1 and DrimDB public datasets. This approach consists of visual enhancement with basic pre-processing methods, the segmentation of the OD with the help of circular Hough transformation to ignore the optical disc (OD) regions from the image, and the CNN-based exudate detection system to automatically detect the exudates in the retinal image. In pre-processing and segmentation of the OD region step, adaptive histogram equalization, Canny edge detection algorithm and circular Hough conversion methods are applied respectively to improve retinal images and prevent interference with OD, which is an anatomical region. The images obtained by segmenting and discarding the OD are trained with CNN and subjected to binary classification as exudated and exudate-free image. Then, the method developed with the images not included in the training set was found to have a correct classification ratio of 99.17% in DiaretDB0, 98.53% in DiaretDB1 and 99.18% in DrimDB. This suggests that the results of the proposed approach are more successful than the results obtained using CNN-only or image processing methods alone. Finally, it is seen that the proposed method that applying CNN to the output image of the image processing result, is more successful than the other methods. [ABSTRACT FROM AUTHOR]

Highlights • Convexity is used as a cue for fruit recognition in RGB-D images. • A novel convex template instance (CTI) descriptor is used for fruit recognition. • A dataset of four sorts of fruits acquired in an unstructured environment is provided. • Nearest neighbor approach is used for classification. • Advantages of the CTI descriptor are low dimensionality and fast computation. Abstract Automatic fruit picking is a challenging problem in robotics with a wide application field. A prerequisite for realization of a robotic fruit picker is its ability to detect fruits in tree tops. An expert system, which would be able to compete with human perception, must be capable of recognizing fruits among leaves and branches under uncontrolled conditions, where fruits are occluded and shaded. In this paper, a novel approach for fruit recognition in RGB-D images based on detection and classification of convex surfaces is proposed. The input RGB-D image is first segmented into convex surfaces by a region growing procedure. Each convex surface is then described by an appropriate descriptor and classified with the aid of the associated descriptor. A novel descriptor of approximately convex surfaces is proposed, which we named Convex Template Instance (CTI) descriptor. It is based on approximating surfaces by convex polyhedrons with quantized face orientations, where every polyhedron face corresponds to one descriptor component. Computation of the proposed descriptor is simple and can be performed very efficiently. The proposed CTI descriptor is compared to the SHOT descriptor, a standard descriptor for 3D point clouds. Two variants of the both CTI and SHOT descriptor are evaluated, a variant which uses color and a variant which does not. A k-nearest neighbor classifier is used to classify detected surfaces into two classes: fruit and other. The main advantage of the proposed expert system in comparison to other fruit recognition solutions is its computational efficiency, which is of great importance for its target application – an automatic fruit picker. The proposed approach is evaluated using a challenging dataset containing RGB-D images of four fruit sorts acquired under uncontrolled conditions, which has been made publicly available to the scientific community, allowing benchmarking of novel fruit recognition methods. [ABSTRACT FROM AUTHOR]

Highlights • Investigation into the development of a multi-disciplinary diagnosis framework for Tuberculosis. • Tuberculosis-specific antibody detection in real time using mobile phone. • Exploration of image processing technique to analyse contents of plasmonic ELISA without experts. • Enhancement of detection accuracy using mobile enabled expert system up to 98.4%. Abstract This paper presents an investigation into the development of an intelligent mobile-enabled expert system to perform an automatic detection of tuberculosis (TB) disease in real-time. One third of the global population are infected with the TB bacterium, and the prevailing diagnosis methods are either resource-intensive or time consuming. Thus, a reliable and easy–to-use diagnosis system has become essential to make the world TB free by 2030, as envisioned by the World Health Organisation. In this work, the challenges in implementing an efficient image processing platform is presented to extract the images from plasmonic ELISAs for TB antigen-specific antibodies and analyse their features. The supervised machine learning techniques are utilised to attain binary classification from eighteen lower-order colour moments. The proposed system is trained off-line, followed by testing and validation using a separate set of images in real-time. Using an ensemble classifier, Random Forest, we demonstrated 98.4% accuracy in TB antigen-specific antibody detection on the mobile platform. Unlike the existing systems, the proposed intelligent system with real time processing capabilities and data portability can provide the prediction without any opto-mechanical attachment, which will undergo a clinical test in the next phase. [ABSTRACT FROM AUTHOR]

Automated Teller Machine (ATM) offers great convenience to many people by allowing quick bank transactions and cash withdrawal. However, ATM machines are also vulnerable to attacks when they are unattended during non-office hours and public holidays. Recently, many ATM machines were reported being removed from the premises or damaged using various methods in order to steal the cash inside. Due to this reason, many ATM sites are actually equipped with video surveillance systems to monitor the environment for crime prevention. However, it is difficult for security personnel to pin-point the crime scene in real time when the number of surveillance cameras increases. In this paper, a real time security expert video surveillance system was proposed to detect the suspicious behaviour by utilizing image processing techniques. The proposed expert system, hereafter referred to as ArchCam, is capable in detecting suspicious behaviours that attempt to remove or attack the ATM machines and provide early warning to the centralized video surveillance system. The suspicious behaviour that ArchCam detects include squatting/climbing (attempt to remove security alarm system or place a bomb) and carrying “belt shape” object (attempt to remove the ATM). The squatting/climbing activity is detected through novel technique to estimate the height of the moving object(s) in front of ATM. On the other hand, the “belt shape” object is detected through estimation of object width by using region splitting and merging technique. With the intelligence of detecting suspicious behaviour, the proposed expert system can effectively alert the security personnel to take proactive actions before the cash is being robbed from the ATM machines. This greatly reduces the effort for security personnel as they only need to observe the camera videos with suspicious behaviour, which on the other hand help to improve the possibility of detecting actual crime scene in real time. ArchCam was implemented in an embedded system with GPU platform and has been verified in a simulated ATM setup with good detection accuracy and fast computational timing performance. [ABSTRACT FROM AUTHOR]

The quality of document images has direct impacts on the performance of document image processing systems. Document Image Quality Assessment (DIQA) is, therefore, of fundamental importance to a numerous document processing applications. As manual quality assessment is almost impossible for a huge volume of document images generated in day-to-day life, it is critical to develop intelligent machine operated methods to estimate the quality of document images. In this paper, a blind document image quality assessment method is proposed to deal with the problem of DIQA in real scenarios, as reference images are not always available. To estimate the quality of a document image, the document is first sampled into a set of patches. The extracted patches are then filtered out based on their level of foreground information using a patch selection strategy. For every selected patch, a cluster assignment is then performed to obtain its quality from a quality aware bag of visual words constructed using k-means clustering. An average pooling is finally employed to estimate the quality of the input document image. To evaluate the proposed method, a dataset composed of document images and three scene image datasets were considered for experimentation. The results obtained from the proposed method demonstrate the effectiveness of the proposed DIQA method. These achievements in applied computational intelligence, expert and decision support systems make a good foundation for creating practical tools to automate document image forgery detection, and archiving process. [ABSTRACT FROM AUTHOR]

The paper deals with unconstrained face recognition task for the small sample size problem based on computation of distances between high-dimensional off-the-shelf features extracted by deep convolution neural network. We present the novel statistical recognition method, which maximizes the likelihood (joint probabilistic density) of the distances to all reference images from the gallery set. This likelihood is estimated with the known asymptotically normal distribution of the Kullback–Leibler discrimination between nonnegative features. Our approach penalizes the individuals if their feature vectors do not behave like the features of observed image in the space of dissimilarities of the gallery images. We provide the experimental study with the LFW (Labeled Faces in the Wild), YTF (YouTube Faces) and IJB-A (IARPA Janus Benchmark A) datasets and the state-of-the-art deep learning-based feature extractors (VGG-Face, VGGFace2, ResFace-101, CenterFace and Light CNN). It is demonstrated, that the proposed approach can be applied with traditional distances in order to increase accuracy in 0.3–5.5% when compared to known methods, especially if the training and testing images are significantly different. [ABSTRACT FROM AUTHOR]

Classifier training plays an important role in image classification, while a good classifier could more effectively exploit the discriminative information of input features to separate the difficult samples. Inspired by the recent advance of representation based classifiers and the success of multi-layer architectures in visual recognition, we propose a multi-layer dictionary pair learning based classifier to enhance the image classification performance. With the multi-layer structure and a nonlinear feature transform in each layer, the proposed classifier learning model could accumulate stronger discrimination capability than the previous single-layer representation based classifiers. Furthermore, to make our learning model applicable to datasets with a larger amount of samples, we propose an online training algorithm which updates model parameters with data batches. The so-called online multi-layer dictionary pair learning (OMDPL) method is evaluated on benchmark image classification datasets. With the same input features, OMDPL exhibits better classification performance than other popular classifiers. [ABSTRACT FROM AUTHOR]

Despite the imitability of the signatures due to presence of numerous image processing programs, online verification systems could provide sufficient security for e-signatures. Recent developments in touchscreen technology and android programming also lead to utilization of hidden interfaces stealthily collecting the unique characteristics and storing the key features aside from geometrics. Therefore, we initially designed a signing interface for touchscreens which stealthily collects the precise coordinates while an individual is signing on the screen by fingertips. Even if the coordinate data is extracted as a matrix consisting of x and y values with corresponding time, the speed array is consequently calculated to investigate the higher frequency regions. The speed data processed by continuous wavelet transformations (CWT) to reveal the frequency information of the signing speed with respect to time information. The grayscale spectrograms created by wavelet transforms are converted into arrays for subsequent training session performed by support vector machines (SVM). The trained network successfully classified further attempts of the real and fake signatures with 1.67% false negative (FNR), 3.33% false positive rates (FPR) and 3.41% equal error rate (EER) for 120 signatures, even though the signature is totally public. For understanding the validity of the CWT and SVM running consecutively, the experiments are re-conducted for the signatures taken from SVC2004 and SUSIG public databases. [ABSTRACT FROM AUTHOR]