Search

Multi-modal retinal image registration plays an important role in the ophthalmological diagnosis process. The conventional methods lack robustness in aligning multi-modal images of various imaging qualities. Deep-learning methods have not been widely developed for this task, especially for the coarse-to-fine registration pipeline. To handle this task, we propose a two-step method based on deep convolutional networks, including a coarse alignment step and a fine alignment step. In the coarse alignment step, a global registration matrix is estimated by three sequentially connected networks for vessel segmentation, feature detection and description, and outlier rejection, respectively. In the fine alignment step, a deformable registration network is set up to find pixel-wise correspondence between a target image and a coarsely aligned image from the previous step to further improve the alignment accuracy. Particularly, an unsupervised learning framework is proposed to handle the difficulties of inconsistent modalities and lack of labeled training data for the fine alignment step. The proposed framework first changes multi-modal images into a same modality through modality transformers, and then adopts photometric consistency loss and smoothness loss to train the deformable registration network. The experimental results show that the proposed method achieves state-of-the-art results in Dice metrics and is more robust in challenging cases.

The field of medical image processing is rapidly adopting artificial intelligence. Its use is required for many applications in the healthcare industry. A machine can learn from experience without explicit programming thanks to computer education. It is an area within AI. Deep learning, a kind of machine learning, infers critical features for image processing via multiple layer processing and mathematical operations based on artificial neural networks. In the field of healthcare, which encompasses medicine and dentistry, artificial intelligence has several applications. Early melanoma skin cancer identification is necessary for effective therapy. Melanoma, among the various types of skin cancer, has recently gained international recognition as the most deadly one since it is much more likely to spread to other body regions if detected and treated quickly. Clinical diagnosis of various ailments is increasingly using non-invasive medical computer vision or medical image processing. These methods offer an automatic image processing tool that makes it possible to examine the lesion quickly and precisely. The procedures used in this study included building a database of dermoscopy images, preprocessing, segmenting using thresholding, extracting statistical features using asymmetry, border, colour, diameter, etc., and choosing features based on the total dermoscopy score, principal component analysis (PCA), and convocation neural network classification (CNN). According to the findings, a classification accuracy of 90.1% was attained. [ABSTRACT FROM AUTHOR]

In this paper, we conduct an extensive study on the use of pre-trained convolutional neural networks (CNNs) for omnidirectional image quality assessment (IQA). To cope with the lack of available IQA databases, transfer learning from seven pre-trained CNN models is investigated over retraining on standard 2D databases. In addition, we explore the influence of various image representations and training strategies on the model’s performance. A comparison of the use of projected versus radial content, and multichannel CNN versus patch-wise training is also covered. The experimental results on two publicly available databases are used to draw conclusions about which strategy best fits the visual quality prediction and at which computational cost. The analysis shows that retraining CNN models on 2D IQA databases improves the prediction accuracy. The latter and the required computational time are found to be significantly affected by the training strategy. Cross-database evaluations demonstrate that the nature and variety of the content impact the generalization ability of the models. Finally, we show that conclusions coming from other image processing communities may not hold for IQA. The provided discussion shall provide insights and recommendations when using pre-trained CNNs for omnidirectional IQA. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

Arbitrary style transfer aims at migrating the style of a reference style painting to a target content image. Existing methods find it challenging to achieve good content fidelity and style migration at the same time. Moreover, they all rely on manually defined content and style, which is of limited universality and robustness. In this paper, we propose to introduce contrastive learning into style transfer, instructing the network to automatically learn to model the structural content and artistic style based on natural contrastive relationships in style transfer. Compared with existing methods, our learned modeling of content and style is more robust and universal. In addition, we further propose instance-wise contrastive style losses and a patch-wise contrastive content loss to guide style transfer. Combining the proposed contrastive losses and two self-reconstruction strategies, we develop a new style transfer framework, which is pluggable and can be flexibly applied to various style transfer modules. Experimental results demonstrate that our method has strong flexibility and synthesizes stylized images with higher quality. [ABSTRACT FROM AUTHOR]

Cartoonization as a special type of artistic style transfer is a difficult image processing task. The current existing artistic style transfer methods cannot generate satisfactory cartoon-style images due to that artistic style images often have delicate strokes and rich hierarchical color changes while cartoon-style images have smooth surfaces without obvious color changes, and sharp edges. To this end, we propose a cartoon loss based generative adversarial network (CartoonLossGAN) for cartoonization. Particularly, we first reuse the encoder part of the discriminator to build a compact generative adversarial network (GAN) based cartoonization architecture. Then we propose a novel cartoon loss function for the architecture. It can imitate the process of sketching to learn the smooth surface of the cartoon image, and imitate the coloring process to learn the coloring of the cartoon image. Furthermore, we also propose an initialization strategy, which is used in the scenario of reusing the discriminator to make our model training easier and more stable. Extensive experimental results demonstrate that our proposed CartoonLossGAN can generate fantastic cartoon-style images, and outperforms four representative methods. [ABSTRACT FROM AUTHOR]

A novel image processing scheme based on flash memory is proposed as a highly efficient pre-processing unit capable of Discrete Cosine Transform (DCT) operation for image compression and feature extractions. By using matrix splitting algorithms and optimizations of operation bias, the DCT operation is demonstrated successfully based on the 65 nm NOR flash memory. The processing accuracy, as well as its variations, can be well controlled by setting the working points at the saturation region with further circuit optimizations. The data compression rate is ~15.6%, showing that the proposed scheme can provide a fundamental solution to accelerate various neural network training, image recognitions, and retrievals. [ABSTRACT FROM AUTHOR]

Mobile applications that can be used for the training and classification of plant diseases are described in this paper. Professional agronomists can select the species and their diseases that are supported by the developed tool and follow an automatic training procedure using a small number of indicative photographs. The employed classification method is based on features that represent distinct aspects of the sick plant such as, for example, the color level distribution in the regions of interest. These features are extracted from photographs that display a plant part such as a leaf or a fruit. Multiple reference ranges are determined for each feature during training. When a new photograph is analyzed, its feature values are compared with the reference ranges, and different grades are assigned depending on whether a feature value falls within a range or not. The new photograph is classified as the disease with the highest grade. Ten tomato diseases are used as a case study, and the applications are trained with 40–100 segmented and normalized photographs for each disease. An accuracy between 93.4% and 96.1% is experimentally measured in this case. An additional dataset of pear disease photographs that are not segmented or normalized is also tested with an average accuracy of 95%. [ABSTRACT FROM AUTHOR]