Search

The complex flow of a supersonic jet (Mach number 2.4–2.5) around the blunt aerodynamic body (consisting of a cylinder with a diameter of 16 mm and a truncated cone) was studied using the high-speed shadowgraph technique and computer vision-based visual data processing. The problem considered is directly related to the rocket and space systems stages safe separation problem in a sufficiently dense atmosphere. For systems with a sequential arrangement of stages, their separation often occurs under the rocket engine jet action. The occurrence of lateral forces in violation of axial symmetry can lead to overturning of the discarded stage and undesirable interaction with the outgoing stage. The goals of the paper were to study the pulsation characteristics of the incoming supersonic jet streamlining the body, as well as the pulsations of the bow shock wave in both axisymmetric and asymmetric flows. The recording frame rate in the experiments was up to 775 000 fps. Computer vision and various machine learning approaches were used to process a large number of shadowgraph images. Canny edge detection and Hough transform, as well as a convolutional neural network (CNN) based on YOLOv2 architecture, were used to automatically track the position of the bow shock relative to the body. Jet sound generation was studied - the amplitude dependence on the distance from the nozzle and the evolution in time. The position of the bow shock relative to the streamlined body and flow pulsations over time were analyzed. Bow shock pulsation spectra at zero angle of attack were compared to those at a small angle of attack (α = 2.5°–3°). The power spectral density was also estimated. The slope of the spectra for the zero angle of attack case was found to be horizontal and the spectra for the small angle of attack case had a slope close to −5/3. The strongest of the measured bow shock frequencies were found to be close to those of the jet mixing layer pulsations. The hardware signal spectrum was also measured and this signal can be considered as noise. The described approach makes it possible to obtain the physical characteristics of the flow at any point of interest. It was also shown that the automation of flow visualization experiments using computer vision techniques allows to increase the speed of data processing and obtaining new physical information, which may be important for engineers developing aircraft and spacecraft technologies. • Supersonic jet flow around a blunt body was studied experimentally and also numerically. • Computer vision tools were applied to supersonic flow investigation. • Spectra of jet generated acoustic waves were determined. • Time dependent bow shock positions were automatically measured by the neural network. • Amplitudes of bow shock pulsations and flow fluctuations depend on the angle of the body relative to the jet direction. [ABSTRACT FROM AUTHOR]

Electron paramagnetic resonance imaging (EPRI) is an advanced tumor oxygen concentration imaging method. Now, the bottleneck problem of EPRI is that the scanning time is too long. Sparse reconstruction is an effective and fast imaging method, which means reconstructing images from sparse-view projections. However, the EPRI images sparsely reconstructed by the classic filtered back projection (FBP) algorithm often contain severe streak artifacts, which affect subsequent image processing. In this work, we propose a feature pyramid attention-based, residual, dense, deep convolutional network (FRD-Net) to suppress the streak artifacts in the FBP-reconstructed images. This network combines residual connection, attention mechanism, dense connections and introduces perceptual loss. The EPRI image with streak artifacts is used as the input of the network and the output-label is the corresponding high-quality image densely reconstructed by the FBP algorithm. After training, the FRD-Net gets the capability of suppressing streak artifacts. The real data reconstruction experiments show that the FRD-Net can better improve the sparse reconstruction accuracy, compared with three existing representative deep networks. [ABSTRACT FROM AUTHOR]

Smart manufacturing uses emerging deep learning models, and particularly Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), for different industrial diagnostics tasks, e.g., classification, detection, recognition, prediction, synthetic data generation, security, etc., on the basis of image data. In spite of being efficient for these objectives, the majority of current deep learning models lack interpretability and explainability. They can discover features hidden within input data together with their mutual co-occurrence. However, they are weak at discovering and making explicit hidden causalities between the features, which could be the reason behind the particular diagnoses. In this paper, we suggest Causality-Aware CNNs (CA-CNNs) and Causality-Aware GANs (CA-GANs) to address the issue of learning hidden causalities within images. The core architecture includes an additional layer of neurons (after the last convolution-pooling and just before the dense layers), which learns pairwise conditional probabilities (aka causality estimates) for the features. Computations for these neurons are driven by the adaptive Lehmer mean function. Learned causalities are merged with the features during flattening and (via fully connected layers) influence the classification outcomes. Such causality estimates can be done for the mixed inputs where images are combined with other data. We argue that CA-CNNs not only improve the classification performance of normal CNNs but also open additional opportunities for the explainability of the models' outcomes. We consider as an additional advantage for CA-CNNs (if used as a discriminator within CA-GANs) the possibility to generate realistically looking images with respect to the causalities. [ABSTRACT FROM AUTHOR]

• QR codes for the toxicity dataset have been generated. • Four models were compared with RF and KNN. • Compound classification is crucial in the field of chemistry. • The choice of model and algorithm influences the results. • The study presents a comparison of models and algorithms. Chemical compound classification, toxicity prediction, and environmental risk assessments are critically important in various applications within the field of chemistry. Deep learning models provide highly effective tools for extracting features from complex large datasets and performing classification tasks. Four different deep learning models, namely ResNet50V2, VGG19, InceptionV3, and MobileNetV2, have been compared with the random forest (RF) and k-nearest neighbors (KNN) algorithms. The results obtained from experiments conducted using QRCODE images of the Tox21SMILES dataset demonstrate the effectiveness of deep learning models for classifying chemical compounds and showcase the performance of different classification algorithms. The findings of the study thoroughly evaluate the performance of deep learning models and classification algorithms in the task of chemical classification. While ResNet50V2 and VGG19 models achieve high accuracy and precision, InceptionV3 and MobileNetV2 models provide more balanced results. Additionally, in terms of classification algorithms, the k-nearest neighbors (KNN) algorithm generally outperforms the Random Forest (RF) algorithm. Although the RF algorithm achieves good accuracy, the KNN algorithm proves to be more effective in terms of sensitivity and F1 score. These results emphasize the factors to consider when choosing which deep learning model or classification algorithm to use in chemical classification tasks. In conclusion, this study presents a comprehensive analysis comparing the performance of deep learning models and classification algorithms in chemical classification tasks. The selection of the most suitable model and algorithm for a specific task supports achieving better results in the classification of chemical compounds and related applications. [Display omitted] [ABSTRACT FROM AUTHOR]

Adversarial embedding, which can deceive the CNN-based steganalyzers, has emerged as an effective strategy to improve image steganography security. However, its efficacy might be easily weakened when confronting re-trained or unknown steganalyzers. In this work, the security of adversarial embedding-based image steganography is further improved by ensemble steganalysis and adversarial perturbation minimization. Different from the existing works that rely on a single targeted steganalyzer, the proposed approach develops an ensemble steganographic classifier, which leverages the majority voting rule to smartly select those pixels that are more suitable for adversarial embedding. To mitigate the interference caused by adversarial embedding, two strategies are adopted. Firstly, a cover image is divided into two non-overlapping regions in terms of pixel gradient amplitude. The regions with higher gradient amplitudes are progressively conducted with adversarial embedding until the targeted steganalyzer is effectively deceived. Secondly, the embedding costs are fine-tuned to minimize the degradation of image quality. Extensive experimental results demonstrate that the proposed approach achieves superior steganography security. Under black-box attacks, with S-UNIWARD and HILL as baseline methods and Deng-Net as the targeted steganalyzer, the proposed approach improves the average detection accuracy of 4.88% and 2.47% for S-UNIWARD and HILL, respectively. In comparison, the existing works only achieve improvements of 2.88% and 2.93% for S-UNIWARD, and 1.44% and 1.12% for HILL, respectively. [ABSTRACT FROM AUTHOR]

Confronted with the challenge of malicious bot abuse disrupting daily life, bot detection has become a crucial field of research. Among the various methods proposed, a behavior-driven framework analyzing mouse movements stands out for its widespread applicability, adaptability, and non-intrusive nature. Traditionally, this area of research has relied on manually extracted features from mouse dynamics and classical machine learning techniques, which often struggle to identify complex behavior patterns. To address these limitations, our research turns to Convolutional Neural Networks (CNNs), which have proven effective in image processing, to explore their application in bot detection using mouse movement data. We employ various visual representation techniques to convert mouse movements into trajectory images suitable for CNN analysis. Our empirical findings show that among all the lightweight networks tested, EfficientNet_b3 not only achieves the highest accuracy, with an impressive precision of 99.82%, but also surpasses more complex models like ResNet50 in terms of detection accuracy and modeling speed. These results not only enhance the practical application of deep learning in bot detection but also offer valuable insights into more effective strategies for advancing bot detection technologies. [ABSTRACT FROM AUTHOR]

Mud layer height of thickener is the key quality index of thickening process which is difficult to achieve real-time detection with existing methods in reality. While the need of developing a soft sensor model which can be used for real-time detection of mud layer height, we proposed an end-to-end mud layer height prediction method with attention mechanism-based convolutional neural network (CNN). The dynamic features are firstly extracted from the image samples based on CNN, and then two types of attention mechanism are embedded sequentially to contribute to more precise prediction results. Compared with the traditional spatial attention mechanism, the regional spatial attention mechanism we proposed selectively divides the spatial feature map into regions, while regions containing important features are assigned larger weights. Adding the channel and regional spatial attention mechanism in CNN not only effectively improve both the precision and calculation speed, but also affect the dimension of the output feature map, so as to avoid the loss of channel or spatial attention information of the feature map. To verify the validity of the proposed method, different attention mechanisms are embedded in the CNN, and the corresponding experiments are carried out on the dataset of the thickener mud layer. The experimental results demonstrate the feasibility and effectiveness of the mud layer height prediction method. • A novel convolution neural network model combined with attention mechanism is designed for mud layer height of the thickener prediction. • A regional spatial attention mechanism is proposed which divides the feature map into regions in the spatial dimension according to the distribution of features, and the weights of each region are constructed to increase the weights of the regions containing the key information. • Applying the proposed method of DAM with region divided to the thickening process improves the network features extraction capability, and the prediction accuracy. [ABSTRACT FROM AUTHOR]

• Proposed model achieves state-of-the-art results on tested datasets. • Proposed neural network is simple and suitable for real-time applications. • Simple image augmentation results in improved cross-camera illumination estimation. • Architecture with small kernal outperforms larger kernels in speed and accuracy. [Display omitted] Images have an ever-increasing presence in our daily lives. This increases the need for accurate and efficient image processing. One of the first processing steps in modern cameras is image white-balancing, the process of making the image invariant to the illumination of the scene. This can be achieved by estimating the illumination of the scene, which is used to chromatically adapt the image. Many existing state-of-the-art approaches use pre-trained models as feature extractors. These models are pre-trained on ImageNet and usually have several million parameters. In this paper, we introduce a simple convolutional neural network without pre-trained layers, that achieves state-of-the-art results. The model contains five convolutional layers, and all of them have a small kernel of size (1,1). Experiments with different model complexities and different kernel sizes have shown that high-level semantic information obtained using larger kernels is not required to achieve state-of-the-art results. Cross camera experiments were also performed and they showed that simple image pre-processing can significantly decrease the effect of camera-sensor on the method. The proposed method has less than 22 000 parameters and achieves state-of-the-art results. The model was tested on three different datasets: the Cube+ dataset, the NUS-8 dataset, and the Intel-TAU dataset. [ABSTRACT FROM AUTHOR]

Super-resolution (SR) and image enhancement (IE) are two common and important tasks in image processing. Due to poor equipment, illumination, and photographic skills, many low-resolution and low-quality (in terms of color, and contrasts) photos are captured in our daily life. To make the photos more visually pleasing, we usually enlarge and retouch them in two separate steps. In this paper, we propose a new Joint Image Super-resolution and Enhancement Network (JSENet), which is arguably the first end-to-end method based on deep learning for joint SR and IE. Compared with simply conducting these two tasks in parallel or sequentially, JSENet employs the bilateral learning framework to integrate the two tasks together seamlessly. Meanwhile, two lightweight modules are designed for restoring details and generating color transformation coefficients respectively, which ensures that our JSENet can be deployed in real-world applications. Extensive experiments are conducted to demonstrate that JSENet outperforms the state-of-the-art SR-IE methods. [ABSTRACT FROM AUTHOR]

Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]

• A CNN for heavy rain removal is proposed based on two networks connected in series. • Novel attention modules are introduced to effectively remove heavy rain effect. • New heavy rain data is proposed with new haze map and various heavy rain models. In this paper, we propose a novel heavy rain removal algorithm using a deep neural network. Unlike most of the existing deraining methods, heavy rain removal is a more challenging task because it is necessary to remove both the rain marks and the haze effects, which are entangled in a complex manner. Motivated by this, we propose a new end-to-end two-stage attention network for single-shot heavy rain removal. The proposed network is connected serially with a removal network and a recovery network, which are based on a newly introduced dilation-wise attention block and skip attention block. Based on these attention techniques, the removal network predicts the heavy rain effect that needs to be removed from a given image, and the recovery network successfully predicts the details that need to be recovered, resulting in a clean image. We also introduce a new realistic RainCityscapes+ dataset, composed of synthesized outdoor images, and demonstrate extensive experiments, the results of which show our approach outperforms the state-of-the-art methods on both real and synthetic datasets quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Modifying the existing models of classifiers' operation is primarily aimed at increasing the effectiveness as well as minimizing the training time. An additional advantage is the ability to quickly implement a given solution to the real needs of the market. In this paper, we propose a method that can implement various classifiers using the federated learning concept and taking into account parallelism. Also, an important element is the analysis and selection of the best classifier depending on its reliability found for separated datasets extended by new, augmented samples. The proposed augmentation technique involves image processing techniques, neural architectures, and heuristic methods and improves the operation in federated learning by increasing the role of the server. The proposition has been presented and tested for the fruit image classification problem. The conducted experiments have shown that the described technique can be very useful as an implementation method even in the case of a small database. Obtained results are discussed concerning the advantages and disadvantages in the context of practical application like higher accuracy. [ABSTRACT FROM AUTHOR]

Riverside monitoring systems are used for controlling the passage of ships, counting them to prevent overcrowding in a port, or raising an alarm if the ship is unknown or not safe. This type of control and analysis is commonly carried out by many people who supervise CCTV in real time. In this paper, we present an alternative approach to automatic image analysis using a variety of artificial intelligence techniques. Based on collaborative learning, these are punished if they make an incorrect classification. The main advantage is the possibility of continually increasing the amount of knowledge during system operation. However, overtraining is possible, so each time, the best classifier is chosen. Another advantage for practical use is the small database, which allows for the quick and practical implementation of such a system. To verify its effectiveness, this ship classification system was tested on data obtained in a Polish city, Szczecin, as part of a bigger project for classifying inland ships and publicly available databases (for more general ship problems). • A cascade of classifiers with a priority of classification for nonconventional ships. • A model of classification improved by forcing retraining based on new samples. • Experimental data based on a case study on the river in a Polish city Szczecin. [ABSTRACT FROM AUTHOR]

This study presents an imaging-based deep learning tool to measure the fuel regression rate in a 2D slab burner experiment for hybrid rocket fuels. The slab burner experiment is designed to verify mechanistic models of reacting boundary layer combustion in hybrid rockets by the measurement of fuel regression rates. A DSLR camera with a high intensity flash is used to capture images throughout the burn and the images are then used to find the fuel boundary to calculate the regression rate. A U-net convolutional neural network architecture is explored to segment the fuel from the experimental images. A Monte-Carlo Dropout process is used to quantify the regression rate uncertainty produced from the network. The U-net computed regression rates are compared with values from other techniques from literature and show error less than 10%. An oxidizer flux dependency study is performed and shows the U-net predictions of regression rates are accurate and independent of the oxidizer flux, when the images in the training set are not over-saturated. Training with monochrome images is explored and is not successful at predicting the fuel regression rate from images with high noise. The network is superior at filtering out noise introduced by soot, pitting, and wax deposition on the chamber glass as well as the flame when compared to traditional image processing techniques, such as threshold binary conversion and spatial filtering. U-net consistently provides low error image segmentations to allow accurate computation of the regression rate of the fuel. • Accurate regression rate measurement using U-net neural network for a slab burner. • Network is superior at filtering out noise compared to other basic image processing. • U-net neural network is shown to be independent of incoming oxidizer mass flux. • Model-generated regression rate uncertainty is quantified using Monte-Carlo Dropout. [ABSTRACT FROM AUTHOR]

This study develops two different approaches to perform temporal and spatial measurements of surface wave profile for experimental studies in transparent wave flumes. Both are based on image acquisition and processing with an Internet of Things (IoT) system consisting of three sets of GoPro camera cum Raspberry Pi connected wirelessly together in a local LAN. The first approach uses advanced edge algorithms with perspective transformation of the multiple cameras for the detection, while the second approach adopts Convolutional Neural Network (CNN) algorithms instead with training of the processed image data using information from additional discrete probes installed. Their accuracy is assessed under a range of experimental conditions of regular and irregular waves with different wave heights and periods, based on metrics that consist of the average errors of the predicted water surface profile as well as position errors for wave crests and troughs. The effects on the measurement accuracy due to the image acquisition frequency, camera resolution and camera location are also investigated. The results show that higher wave steepnesses generally lead to larger detection errors, and measurements for irregular waves are also more challenging. In addition, positioning the cameras closer to the wave flume sidewalls yields better detection results as expected, particularly in resolving wave crests and troughs, although the field of view narrows at the same time. However, higher video frequencies and camera resolutions might not necessarily improve the accuracy contrary to common expectation due to jaggedness in the image processing. Overall, both approaches are shown to be viable for the measurement of wave profile in the laboratory. The first approach is more straight forward in terms of implementation, and it performs well for regular wave conditions. The second approach requires more complex training of the neural networks, but its accuracy is significantly higher particularly for irregular waves. [ABSTRACT FROM AUTHOR]

• Three-channel pseudo-color images preprocessing method is designed by concatenating different CT windows. • A dense skip connection encoding–decoding model (DSC-Net) is proposed to perform automatic segmentation of thymoma base on a deep convolutional neural network. • Dense connections are introduced into the architecture of the encoding path across different level feature maps in the DSC-Net. • Different level skip connections are designed between the encoding and decoding path in the DSC-Net. Accurately positioning and segmenting thymoma from computed tomography (CT) images is of great importance for an image-driven thymoma analysis. In clinical practice, the diagnosis and segmentation of thymomas for radiologists are time-consuming and inefficient tasks. Thus, it is necessary to develop a method to accurately and efficiently realize automatic segmentation of thymoma. Here, a dense skip connection encoding–decoding model (DSC-Net), which is a deep convolutional neural network, was proposed to perform automatic segmentation of thymoma with the ability to fuse feature maps under receptive fields of different scales. An image preprocessing method was also proposed to provide much more texture information and enhance the contrast between thymoma and its surrounding tissues. A total of 310 subjects who underwent contrast-enhanced CT scanning were included in this ethically-approved retrospective study. All of the CT slices were manually labeled by four experienced radiologists, and 80% of images were included in the training set and the rest were included in the testing set. The performance of segmentation was evaluated by calculating the accuracy, intersection over union (IoU), and Boundary F1 contour matching score (BFScore) between the predicted segmentation and the manual labels. For segmentation of thymoma in the testing set, the accuracy, IoU and BFScore were 92.96%, 87.86% and 0.9087 respectively. Compared to the U-Net method, the DSC-Net model improved IoU by 3.94%. In addition, the efficacy and robustness of DSC-Net in segmentation of different patients and different types of thymoma classified by the WHO histological classification criteria were verified. The proposed preprocessing method and DSC-Net demonstrated improved performance in segmentation of thymomas, suggesting the ability to provide consistent delineation and assist radiologists in their clinical applications. [ABSTRACT FROM AUTHOR]

Developmental dysplasia of the hip (DDH) metrics based on 3-D ultrasound have proven more reliable than those based on 2-D images, but to date have been based mainly on hand-engineered features. Here, we test the performance of 3-D convolutional neural networks for automatically segmenting and delineating the key anatomical structures used to define DDH metrics: the pelvis bone surface and the femoral head. Our models are trained and tested on a data set of 136 volumes from 34 participants. For the pelvis, a 3D-U-Net achieves a Dice score of 85%, outperforming the confidence-weighted structured phase symmetry algorithm (Dice score = 19%). For the femoral head, the 3D-U-Net had centre and radius errors of 1.42 and 0.46 mm, respectively, outperforming the random forest classifier (3.90 and 2.01 mm). The improved segmentation may improve DDH measurement accuracy and reliability, which could reduce misdiagnosis. [ABSTRACT FROM AUTHOR]

Accurately measuring the pellet size distribution during the pelletizing process is critical, because the pellet size distribution is a key quality indicator and affects the efficiency of the blast furnace. In this study, a method based on a convolutional neural network is proposed to measure the pellet size distribution in the stable area of a rotary disc. The proposed network only uses simple convolution layers in sequence to realize multiscale performance; furthermore, it uses a bottleneck layer and a concatenate path to improve the computing efficiency and alleviate the vanishing gradient problem, respectively. Various experimental results demonstrate that the proposed network outperforms other comparison methods in segmenting overlapped pellets, especially pellets of different sizes. The pellet size distribution obtained by the proposed method agreed well with the manual sieving results. Moreover, the computing time of the proposed method can meet the requirements of online size distribution measurement. [Display omitted] • Proper imaging system is designed to capture the dense pellet images. • A novel deep learning method is proposed to segment the overlapped pellets. • The proposed method achieves higher segmentation results than comparative methods. • Computing time of the proposed method can meet the online measurement. [ABSTRACT FROM AUTHOR]

We propose a retinal vessel segmentation algorithm based on a convolution neural network and connected domain detection. After defining the discriminant matrix, we constructed and trained a convolutional neural network model, which can realize the mapping relationship from eye fundus grayscale to the discriminant matrix. This model achieves the preliminary segmentation of retinal vessels. The prediction of uncertain pixels is revised by using the geometric characteristics of the vessels and through the analysis of connected regions. The experimental results show good generalization ability, the average segmentation accuracy, specificity, and sensitivity are 96.64%, 97.96%, and 80.68%, respectively. [ABSTRACT FROM AUTHOR]

• TrioConvTomatoNet, a novel architecture of deep convolutional neural network is designed for the classification of tomato leaf diseases. • This research suggests improving a network's learning ability by incorporating stochastic gradient descent optimizers. • It is also to be noted that the inclusion of both real-time and existing datasets for classification strengthens the diverse data collection. • Shown result in the proposed architecture would certainly be a suitable for practical applicability in agricultural settings. Tomato leaf diseases are a common threat to long-term tomato protection that affects many farmers worldwide. Computer-assisted technology is more prevalent for early and accurate diagnosis of tomato leaf diseases, which may reduce the likelihood that the plant will suffer further harm. The advancement of deep learning techniques has a lot of potential for accurate classification of leaf diseases through an automated feature extraction topology. By changing the network layers and their parameters in the feature extraction topology, it is possible to extract the exact features and improve the classification accuracy. In this study, TrioConvTomatoNet, novel deep convolutional neural network architecture, is proposed, which includes a 3-series convolution layer under each stage of the feature extraction topology to capture and integrate the information from the leaf images efficiently. To improve the network's learning ability and perform accurate classifications, the proposed method incorporates the stochastic gradient descent optimizer. In order to facilitate proper learning, a separate dataset preparation strategy was followed for tomato leaf image dataset collection, which includes existing and real-time images. This research is carried out with varying the number of convolution layers to express the effectiveness of the proposed method in capturing and integrating information from the leaf images. Also, the extensive experiments on an unseen dataset collection with and without augmentation express the superiority and robustness of TrioConvTomatoNet in terms of both accuracy and speed when compared to state-of-the-art methods. The proposed method achieves remarkable accuracy of about 99.39% in disease classification while processing both existing database and real-time images, making it suitable for practical deployment in agricultural settings. [ABSTRACT FROM AUTHOR]

This study presents a deep convolutional neural network metasensor for measuring crack width from high-resolution images and brightness profiles. Procedures of training, testing, data drift evaluation and fine-tuning were proposed. The metasensor enables repeated, semi-automated crack measurements in the same set of multiple locations over subsequent multi-stage observations, a unique and important feature for accurately assessing the progress of self-healing. It was employed to investigate the autogenous self-healing of 75-month-old high-strength concrete samples extracted from a structural member. The crack-width measurements, aided by X-ray computed tomography and electron microscopy data, helped to identify differences between the self-healing of externally exposed and deep-lying materials as a result of the natural local porosity differentiation. • Proposed deep CNN metasensor for crack width assessment from high-resolution images. • Trained metasensor using considerable set of manual reference measurements. • Achieved semi-automatic, multi-position and multi-stage self-healing evaluation. • Shown differences in healing of exposed and deeper materials, explained by porosity. • Demonstrated effective self-healing in mature high-strength concrete. [ABSTRACT FROM AUTHOR]

Predictive learner modelling is crucial for personalized education. While convolutional neural networks (CNNs) have shown great success in education, their potential in learner modelling via image data is unexplored. This research introduces a novel and interpretable approach for Image-based Learner Modelling (ImageLM) using CNNs and transfer learning to model learners' performance and accordingly classify their computational thinking solutions. The approach integrates Grad-CAM, enabling it to provide insights into its decision-making process. Findings show that our custom CNN outperforms other models (namely ResNet, VGG, and Inception), with 83% accuracy in predicting solution correctness. More importantly, the ImageLM approach identifies the regions that contribute the most to the predictions, shedding light on learners' computational thinking knowledge and advancing toward trustworthy AI for education. These results underline the potential of utilizing imagery data from learners' activities during the learning process to predict their performance, especially in challenging environments like programming where traditional feature extraction and learning might struggle. [ABSTRACT FROM AUTHOR]

Virtual reality is becoming more and more improved primarily due to numerous applications and the powers of mobile devices. Using various sensors, precise displays and high computing powers smartphone are becoming devices that make the boost in technology. Now it is necessary to efficiently use various sensors without affecting system operation and improve control abilities for various purposes. Especially in practical applications received by mass users such as games and any kind of experience. In this article, we propose a system that allows to extend the perception of the virtual world by conveying information about the user's movements in reality into the supervised model. The system retrieves data from several sources, quickly analyzes them using artificial intelligence techniques, and returns information to the mobile phone about the activity that is being processed. The concept extends the understanding of today's virtual reality by allowing the user to move and perform simple gestures in a specially designed room. Moreover, we propose multiplayer mode in virtual reality, where players are in different places. The proposed architecture of the system has been tested on simple applications, and the results show high potential for implementations in various apps by achieving almost 90% efficiency in changing player direction in real time and only 7.5% of collision cases. • A closed space architecture for games in virtual reality with the use of additional sensors. • Analysis of the use of machine learning techniques for rapid data classification in real-time. • Mathematical model and analysis of player movement in real-time. • Multiplayer approach in virtual reality. • A simple multiplayer VR game architecture in real-time. [ABSTRACT FROM AUTHOR]

This paper proposes a new haze layer based single image dehazing algorithm. The residual images, which exists between the hazy images and the clear images, will be firstly obtained by haze layers through an end-to-end mapping from the original hazy images. Then the designed convolutional neural network can remove the residual image from the given hazy image to obtain a recovered dehazed image. In order to remove the halos and block artefacts in the dehazed images, a guided filter is applied before generating the final dehazed images, which will also increase the reality of the result images. Since the proposed haze layer based dehazing algorithm directly learns the residual images, it enjoys a relatively high learning rate and low computation. We tested and compared the proposed algorithm with other dehazing methods in various variables, such as the density of the fog and the testing scenes, e.g. real-world images or synthetic images. Experimental results showed the advancement of the proposed method in many metrics. [ABSTRACT FROM AUTHOR]

Edge detection plays an important role in image processing. With the development of deep learning, the accuracy of edge detection has been greatly improved, and people have more requirements for edge detection tasks. Most edge detection algorithms are binary edge detection methods, but there are usually multiple categories of edges in an image. In this paper, we present an accurate multi-category edge detection network, the richer category-aware semantic edge detection network (R-CASENet). In order to make full use of convolutional neural network's powerful feature expression capabilities, we attempt to use more information from feature maps for edge feature extraction and classification. Using the ResNet101 network as the backbone, firstly we merge the building blocks in different composite blocks and down-sample to obtain the feature maps. Then we fuse the feature maps in different composite blocks to obtain the final fused classifier. Experimental results show that R-CASENet can achieve state-of-the-art performance on the large SBD dataset. Furthermore, to get precise one-pixel width edges, we also propose an edge refinement network (ERN) structure. The proposed scheme is an end-to-end method and the proposed ERN can reduce redundant points and improve computational efficiency, especially for further image processing. [ABSTRACT FROM AUTHOR]

For quantitative neuroimaging studies using multi-echo gradient echo (mGRE) images, additional T 1 -weighted magnetization prepared rapid gradient echo (MPRAGE) images are often acquired to supplement the insufficient morphometric information of mGRE for tissue segmentation which require lengthened scan time and additional processing such as image registration. This study investigated the feasibility of generating synthetic MPRAGE images from mGRE images using a deep convolutional neural network. Tissue segmentation results derived from the synthetic MPRAGE showed good agreement with those from actual MPRAGE (DSC = 0.882 ± 0.017). There was no statistically significant difference between the mean susceptibility values obtained with the regions of interest from synthetic and actual MPRAGEs and high correlation between the two measurements. [ABSTRACT FROM AUTHOR]

In this paper, a joint baud-rate and modulation format identification (BR-MFI) is proposed based on asynchronous delay tap picture (ADTP) analyzer by using convolutional neural network (CNN). Considering 8 types of signals under different channel conditions of OSNR, CD and DGD, the proposed BR-MFI can achieve 100% accuracy after 6 training epochs, and just 2 epochs for MFI. Here, two test number of samples are about 15% of total samples. This paper also investigates the influence of CNN structure on the identification accuracy. The results show that CNN has better performance for image processing than back-propagation Artificial Neural Network (BP-ANN). [ABSTRACT FROM AUTHOR]

• Taking advantage of the fact that CatBoost does not require significant preprocessing of category features, the internal construction uses unique thermal coding for categorical features with small bases, enabling the model to automatically transform category-based features into numerical features. • The proposed model supports automatic finding of class feature combinations, which significantly extends the dimensionality of license plate hyperspectral image features by maximizing the relationship between different features, further improving the effectiveness and generalization ability of the model. • By comparing with traditional image processing methods, the proposed CatBoost-based license plate hyperspectral image classification model has better performance in processing applications. The rapid development of artificial intelligence technology has brought profound technical changes to many traditional industries, among which intelligent transportation has become a hot spot for development in the traditional transportation field. Remote sensing license plate image recognition technology is extensively emphasized in such domains as intelligent transportation and intelligent vehicle management. However, in the practical traffic environment, low visibility scenes caused by complex environmental factors such as rain, snow, haze and cloudy days influence the recognition and classification of license plates, while the distortion of license plate images that may be caused by irregular movements of vehicles bring challenges to license plate recognition classification. The CNN-CatBoost model proposed in this paper divides the license plate recognition classification into two stages. The first stage uses the excellent performance of convolutional neural network in processing image data to extract various license plate image features; the second stage uses the CatBoost module to further process the image feature data and finally obtain the remote sensing license plate information. The model achieves outstanding results in the experiments and has practical application value. Through comparison with other network models, the CNN-CatBoost model proposed in this paper has superior performance. [ABSTRACT FROM AUTHOR]

The present work focuses on designing a robust deep-learning model to track bubble dynamics in a vertical rectangular mini-channel. The rectangular mini-channel is heated from one side with a constant heat flux, resulting in the creation of bubbles. Images of the bubbles are recorded using a high-speed camera, which serve as the input data for the deep learning model. The raw image data acquired from the high-speed camera is inherently noisy due to the presence of shadows, reflections, background noise, and chaotic bubbles. The objective is to extract the mask of the bubble given all these challenging factors. Transfer learning is adopted to eliminate the need for a large dataset to train the deep learning model and also to reduce computational costs. The trained model is then validated against the validation datasets, demonstrating an accuracy of 98% while detecting the bubbles. The model is then evaluated on different experimental conditions, such as lighting, background, and blurry images with noise. The model demonstrates high robustness to different conditions and is able to detect the edges of the bubbles and classify them accurately. Moreover, the model achieves an average intersection over union of 85%, indicating a high level of accuracy in predicting the masks of the bubbles. The method enables accurate recognition and tracking of individual bubble dynamics, capturing their coalescence, oscillation, and collisions to estimate local parameters by proving the bubble masks. This allows for a comprehensive understanding of their spatial–temporal behaviour, including the estimation of local Reynolds numbers. [Display omitted] • A deep-learning model to track local bubble behaviour. • 25% improvement over conventional techniques. • The model effectively tackles noisy and complex image conditions. • It can handle the onset of bubble generation, coalescence, and oscillation. • Flexible to different flow regimes through transfer learning. [ABSTRACT FROM AUTHOR]

• MS3Net is designed with a dual-stream symmetric structure for HSI classification. • Stratified-split module can promote the extraction of multiscale features. • Quadra-view attention can focus on valuable features from different dimensions. • Experimental results on four HSI datasets show the effectiveness of our method. Recently, hyperspectral image (HSI) classification has become a promising research direction in remote sensing image processing. Many HSI classification methods have been proposed based on convolutional neural networks (CNNs) and attention mechanisms (AMs). However, most current CNN-based methods only consider extracting features at a single scale in HSI, which may ignore the delicate features of some objects. Moreover, present AMs primarily focus on one feature dimension, such as spatial or channel attention, while disregarding dimension interaction. To conquer the above issues, a novel multiscale stratified-split symmetric network with quadra-view attention, namely MS3Net, is proposed for HSI classification. Generally, the proposed MS3Net has a dual-stream symmetric pipeline, which can better extract HSI's spectral signatures and spatial features. Specifically, the proposed MS3Net consists of three modules: a multiscale feature extraction module, a feature enhancement module, and a feature fusion module. Firstly, a stratified-split module is designed to extract multiscale spectral and spatial features. In addition, to reduce the complexity of the model, we designed pseudo-3-D spectral and spatial convolution to replace the traditional 3-D convolution operation. Secondly, a novel quadra-view attention module is proposed, guiding the model to focus on important features from multiple dimensions. Finally, the selective kernel feature fusion module is introduced, which can dynamically integrate spectral and spatial features. Experimental results on four benchmark HSI datasets with different scenes and resolutions confirm the visual and quantitative superiority of the proposed MS3Net over the state-of-the-art related methods in this research direction. [ABSTRACT FROM AUTHOR]

• A set of processing procedure for images captured by UAV is designed. • The improved U-net is used for semantic segmentation to screen the effective region. • The improved VGG-19 is used to locate and identify the disease on the target leaf. • The testing phase is performed under a testing set of different origin from the training set. In order to realize the acquisition of plant leaf images by agricultural drones and perform regional segmentation and disease recognition on leaf cluster images over a wide area, this paper takes grape leaves as an example and designs a set of processing procedures combining the improved U-net and VGG-19 networks. This paper aims to address the difficulty of obtaining target leaves for recognition in complex environments due to the presence of multiple invalid leaves. First, the feature extraction process is performed on the training and validation datasets to reduce the input parameters of the network and increase the training speed. Second, the test data is divided into three datasets: low-light, jitter interference, and two-factor combination. After recovery processing, the Multi-fusion U-net network is fed to locate diseased leaves. Finally, the improved VGG-19 network was used again to locate and identify the disease. Experimental results show that the proposed procedure achieves satisfactory performance in UAV image processing. The average accuracy of segmentation reaches 71.91%, and the identification rate of disease location is increased by 12.33% after segmentation, which provides a strong practical basis for the implementation of unmanned smart ecological farms. [ABSTRACT FROM AUTHOR]

Preparing the infrastructure for analyzing, recognizing, and characterizing microorganisms in the Metaverse can transform the fields of biology, medicine, and drug discovery. Accordingly, the realization of digital twins of microorganisms, such as viruses, fungi, algae, bacteria, protozoa, archaea, and multicellular animal parasites, can streamline the applicability of the Metaverse and similar emerging technologies like Cyber-Physical Healthcare Systems (CPHS). This is why a new approach to the digital twinning of bacteria has been presented in this research. This method of digital twinning can revolutionize the research and study of bacteria because it allows us to separate useful and harmful bacterial species, increasing the efficiency of treatment noticeably. This innovative method can easily be used by clinics, medical centers, or even by private users physically and virtually and can be adapted to every centralized or decentralized Metaverse Platform. To determine the proper treatment, biologists have always tried to identify the correct bacterial species that prompted a bacterial infection. They use various indicators, such as the bacterial cell's shape and the size of the colony formed by the bacteria, to classify different types of bacteria with different biochemistries and shapes. However, it is challenging because of the extensive similarities between some species. For instance, such similarities exist between Staphylococcus aureus and Staphylococcus saprophyticus, which has caused numerous false diagnostic reports by operators. Wrong species reports bring about treatment failure and increase antibiotic resistance issues. Therefore, the digital twins of bacteria cover all of their identifiable characteristics and overcome many limitations to study them. In this approach, DTL techniques, including MobileNetV2, EfficientNetV2-S, and ResNet-50, have been employed to build digital twins of bacteria species. A hybrid dataset was used for training and evaluation. Among the models, EfficientNetV2-S exhibited the best performance, with a validation accuracy of 99.58% and a test accuracy of 99.33%. The results showed the ability of deep learning models to make bacteria digital twins based on image processing from different labs, thus assisting experts in speeding up the process and reducing diagnostic errors. In addition, in the mispredicted cases, the correct species was among the first three choices of the model. Therefore, not only can experts use DTL approaches to speed up the digital twin realization of microorganisms in the Metaverse, but they can also use these methods to reduce diagnostic errors. • Introducing a technique to streamline digital twinning of microorganisms using CNN. • Describing machine learning digital twins of microorganisms in the Metaverse. • Presenting an exact, economical digital twinning approach via smartphone photography. • Providing a comprehensive bacteria species dataset for future method development. [ABSTRACT FROM AUTHOR]

Using remotely sensed imagery to identify biophysical components across landscapes is an important avenue of investigation for ecologists studying ecosystem dynamics. With high-resolution remotely sensed imagery, algorithmic utilization of image context is crucial for accurate identification of biophysical components at large scales. In recent years, convolutional neural networks (CNNs) have become ubiquitous in image processing, and are rapidly becoming more common in ecology. Because the quantity of high-resolution remotely sensed imagery continues to rise, CNNs are increasingly essential tools for large-scale ecosystem analysis. We discuss here the conceptual advantages of CNNs, demonstrate how they can be used by ecologists through distinct examples of their application, and provide a walkthrough of how to use them for ecological applications. CNNs enable ecologists to identify biophysical components in high-resolution remotely sensed imagery by leveraging spatial context, and are particularly effective when ecological components have distinct shapes. CNNs can be used for both object detection, where key components are identified throughout an image, and semantic segmentation, where each pixel is classified individually. CNN accuracy is similar to human-level classification accuracy, but is consistent and fast, enabling rapid application over very large areas and/or through time. [ABSTRACT FROM AUTHOR]

• Automatic Plant disease diagnosis for early disease symptoms. • Novel image processing algorithm in combination with machine learning methods. • A testing on wheat was performed with 7 mobile devices over a period of 4 seasons. • Obtained results reveal AuC metrics higher than 0.80 for all the analyzed diseases. Fungal infection represents up to 50 % of yield losses, making it necessary to apply effective and cost efficient fungicide treatments, whose efficacy depends on infestation type, situation and time. In these cases, a correct and early identification of the specific infection is mandatory to minimize yield losses and increase the efficacy and efficiency of the treatments. Over the last years, a number of image analysis-based methodologies have been proposed for automatic image disease identification. Among these methods, the use of Deep Convolutional Neural Networks (CNNs) has proven tremendously successful for different visual classification tasks. In this work we extend previous work by Johannes et al. (2017) with an adapted Deep Residual Neural Network-based algorithm to deal with the detection of multiple plant diseases in real acquisition conditions where different adaptions for early disease detection have been proposed. This work analyses the performance of early identification of three relevant European endemic wheat diseases: Septoria (Septoria triciti), Tan Spot (Drechslera triciti-repentis) and Rust (Puccinia striiformis & Puccinia recondita). The analysis was done using different mobile devices, and more than 8178 images were captured in two pilot sites in Spain and Germany during 2014 , 2015 and 2016. Obtained results reveal an overall improvement of the balanced accuracy from 0.78 (Johannes et al., 2017) up to 0.87 under exhaustive testing, and balanced accuracies greater than 0.96 on a pilot test performed in Germany. [ABSTRACT FROM AUTHOR]

Abstract Low-light is a challenging environment for image processing and computer vision tasks, either in contrast enhancement for better visibility and quality, or application oriented tasks such as detection. We found that the current trend of low-light enhancement research is heavily on quality improvement. In this work, we aim to shift the focus towards a more functional direction, that is enhancement that prioritizes feature retrieval. For this reason, we first propose to model low-light enhancement as a set of localized functions using Gaussian Process (GP) that is trained at runtime using data from a simple Convolutional Neural Network (CNN) to provide the necessary feature information as reference. The CNN is in turn trained using large amount of synthetic data, based upon the luminance distribution of real world low-light images to learn the relationship between features and pixels. Secondly, we also proposed two new evaluation metrics to better assess enhancement algorithms to support high level computer vision tasks, namely, local features matching and intensity histogram similarity. In our experiments, our proposed low-light enhancement framework outperforms the state-of-the-arts with significant improvement in Recall, F 1 , and F 2 -score of SIFT features matching, and achieve comparable results for l 1 -norm distance of histograms as well as the PSNR. Moreover, our analysis of the performance showed that the PSNR quality metric is not only unable to assess the practicality of the results, but also inappropriately gives high assessment to low visual quality images. Highlights • Gaussian Process with CNN data is proposed for lowlight image contrast enhancement. • Enhancement aimed for high level computer vision applications instead of aesthetics. • New evaluation metrics are proposed. • Proposed method shows competitive results with the state-of-the-art. [ABSTRACT FROM AUTHOR]

Abstract Pan-sharpening is a domain-specific task of satellite imagery fusion. However, most traditional methods fuse the panchromatic image and the multispectral images in linear manners, which lead to severe spectral and spatial distortions. In the meanwhile, discriminative learning methods are limited in specialized satellites and tasks. In this paper, we make an attempt to integrate a deep prior with model-based optimization scheme for pan-sharpening. The proposed deep prior is based on a convolutional neural network which is composed of the proposed problem-specific recursive block and is trained in gradient domain. We plug the trained prior in place of the spatial preservation term in model-based optimization scheme, and address it with the alternating direction method of multipliers. Final experimental results demonstrate that the proposed model can overcome the restriction of linear model, and greatly reduce spectral and spatial distortions. Compared with several discriminative learning methods, our model tends to achieve promising generalization across different satellites. [ABSTRACT FROM AUTHOR]

In this study, we present new deep learning (DL) method for fusing multi-focus images. Current multi-focus image fusion (MFIF) approaches based on DL methods mainly treat MFIF as a classification task. These methods use a convolutional neural network (CNN) as a classifier to identify pixels as focused or defocused pixels. However, due to unavailability of labeled data to train networks, existing DL-based supervised models for MFIF add Gaussian blur in focused images to produce training data. DL-based unsupervised models are also too simple and only applicable to perform fusion tasks other than MFIF. To address the above issues, we proposed a new MFIF method, which aims to learn feature extraction, fusion and reconstruction components together to produce a complete unsupervised end-to-end trainable deep CNN. To enhance the feature extraction capability of CNN, we introduce a Siamese multi-scale feature extraction module to achieve a promising performance. In our proposed network we applied multiscale convolutions along with skip connections to extract more useful common features from a multi-focus image pair. Instead of using basic loss functions to train the CNN, our model utilizes structure similarity (SSIM) measure as a training loss function. Moreover, the fused images are reconstructed in a multiscale manner to guarantee more accurate restoration of images. Our proposed model can process images with variable size during testing and validation. Experimental results on various test images validate that our proposed method yields better quality fused images that are superior to the fused images generated by compared state-of-the-art image fusion methods. [ABSTRACT FROM AUTHOR]

Abstract Understanding environmental factors that influence forest health, as well as the occurrence and abundance of wildlife, is a central topic in forestry and ecology. However, the manual processing of field habitat data is time-consuming and months are often needed to progress from data collection to data interpretation. To shorten the time to process the data we propose here Habitat-Net: a novel deep learning application based on Convolutional Neural Networks (CNN) to segment habitat images of tropical rainforests. Habitat-Net takes color images as input and after multiple layers of convolution and deconvolution, produces a binary segmentation of the input image. We worked on two different types of habitat datasets that are widely used in ecological studies to characterize the forest conditions: canopy closure and understory vegetation. We trained the model with 800 canopy images and 700 understory images separately and then used 149 canopy and 172 understory images to test the performance of Habitat-Net. We compared the performance of Habitat-Net to the performance of a simple threshold based method, manual processing by a second researcher and a CNN approach called U-Net, upon which Habitat-Net is based. Habitat-Net, U-Net and simple thresholding reduced total processing time to milliseconds per image, compared to 45 s per image for manual processing. However, the higher mean Dice coefficient of Habitat-Net (0.94 for canopy and 0.95 for understory) indicates that accuracy of Habitat-Net is higher than that of both the simple thresholding (0.64, 0.83) and U-Net (0.89, 0.94). Habitat-Net will be of great relevance for ecologists and foresters, who need to monitor changes in their forest structures. The automated workflow not only reduces the time, it also standardizes the analytical pipeline and, thus, reduces the degree of uncertainty that would be introduced by manual processing of images by different people (either over time or between study sites). Highlights • Canopy closure and understory vegetation density are measurements that are of high relevance for foresters and ecologists. • Currently, months to years are needed to manually process forest habitat field data. • Deep learning can be used to significantly shorten the time while maintaining accuracy. • We propose Habitat-Net, a Convolutional Neural Network based method, to segment color images of canopy and understory. • Habitat-Net decreases the time needed to process habitat images, while offering improved performance and consistency. [ABSTRACT FROM AUTHOR]

Abstract This paper proposes an object detection model named Feature Enriching Object Detection Framework with Weak Segmentation Loss (FEOD) based on convolutional neural networks, which is an improvement of You Only Look Once (YOLO). To overcome the shortcoming of insensitivity to small objects, a novel feature enriching module is proposed to augment the semantic information of the feature in the shallow layer within a typical deep detector. Meanwhile, Focal Loss is also introduced to our model to further improve the algorithm performance. To obtain features more suitable for object detection, a more powerful feature extractor – DetNet is used as the backbone. An mAP of 83.8 in VOC2007 test and an mAP of 82.1 in VOC2012 test are achieved with a FPS of 11.3 with a NVIDIA GTX1080Ti GPU. For a lower resolution version, we achieve an mAP of 81.9 and 80.9 respectively in VOC2007 and VOC2012 tests with FPS of 17. Comparisons with other object detection algorithms have shown that our method works well and achieves a trade-off of accuracy and speed. [ABSTRACT FROM AUTHOR]

Abstract Crack detection is a critical task in monitoring and inspection of civil engineering structures. Image classification and bounding box approaches have been proposed in existing vision-based automated concrete crack detection methods using deep convolutional neural networks. The current study proposes a crack detection method based on deep fully convolutional network (FCN) for semantic segmentation on concrete crack images. Performance of three different pre-trained network architectures, which serves as the FCN encoder's backbone, is evaluated for image classification on a public concrete crack dataset of 40,000 227 × 227 pixel images. Subsequently, the whole encoder-decoder FCN network with the VGG16-based encoder is trained end-to-end on a subset of 500 annotated 227 × 227-pixel crack-labeled images for semantic segmentation. The FCN network achieves about 90% in average precision. Images extracted from a video of a cyclic loading test on a concrete specimen are used to validate the proposed method for concrete crack detection. It was found that cracks are reasonably detected and crack density is also accurately evaluated. Highlights • Crack classifiers built on pre-trained networks achieve at least 97.8% in accuracy. • Semantic segmentation method produces about 90% in average precision. • Semantic segmentation method can capture crack size reasonably. [ABSTRACT FROM AUTHOR]

Highlights • This work investigates computational method for automatic detection of spinal cord in planning CT scan for radiotherapy. • The spinal cord was detected in CT scan using adaptive template matching, IMSLIC and convolutional neural network. • The method was applied with 36 CT scans with an average of 200 slices. • The method uses image processing and deep learning techniques. • The method achieved an accuracy of 92.55%, specificity of 92.87% and sensitivity of 89.23% with 0.065 of false positives per images. Abstract Background and Objective The spinal cord is a very important organ that must be protected in treatments of radiotherapy (RT), considered an organ at risk (OAR). Excess rays associated with the spinal cord can cause irreversible diseases in patients who are undergoing radiotherapy. For the planning of treatments with RT, computed tomography (CT) scans are commonly used to delimit the OARs and to analyze the impact of rays in these organs. Delimiting these OARs take a lot of time from medical specialists, plus the fact that involves a large team of professionals. Moreover, this task made slice-by-slice becomes an exhaustive and consequently subject to errors, especially in organs such as the spinal cord, which extend through several slices of the CT and requires precise segmentation. Thus, we propose, in this work, a computational methodology capable of detecting spinal cord in planning CT images. Methods The techniques highlighted in this methodology are adaptive template matching for initial segmentation, intrinsic manifold simple linear iterative clustering (IMSLIC) for candidate segmentation and convolutional neural networks (CNN) for candidate classification, that consists of four steps: (1) images acquisition, (2) initial segmentation, (3) candidates segmentation and (4) candidates classification. Results The methodology was applied on 36 planning CT images provided by The Cancer Imaging Archive, and achieved an accuracy of 92.55%, specificity of 92.87% and sensitivity of 89.23% with 0.065 of false positives per images, without any false positives reduction technique, in detection of spinal cord. Conclusions It is demonstrated the feasibility of the analysis of planning CT images using IMSLIC and convolutional neural network techniques to achieve success in detection of spinal cord regions. [ABSTRACT FROM AUTHOR]

Abstract We propose a sparse Convolutional Autoencoder (CAE) for simultaneous nucleus detection and feature extraction in histopathology tissue images. Our CAE detects and encodes nuclei in image patches in tissue images into sparse feature maps that encode both the location and appearance of nuclei. A primary contribution of our work is the development of an unsupervised detection network by using the characteristics of histopathology image patches. The pretrained nucleus detection and feature extraction modules in our CAE can be fine-tuned for supervised learning in an end-to-end fashion. We evaluate our method on four datasets and achieve state-of-the-art results. In addition, we are able to achieve comparable performance with only 5% of the fully-supervised annotation cost. [ABSTRACT FROM AUTHOR]

Highlights • Convolutional neural networks derive a robust distance measure for ISH images. • Olfactory bulb (OB) tissue sections are well-classified by CNNs. • A high-resolution atlas of registered OB ISH sections was created. • Gene expression patterns in the OB clustered into lamina-specific groups. • Our code is publicly available, for extension to other brain areas. Abstract Background The Allen Mouse Brain Atlas allows study of the brain's molecular anatomy at cellular scale, for thousands genes. To fully leverage this resource, one must register histological images of brain tissue – a task made challenging by the brain's structural complexity and heterogeneity, as well as inter-experiment variability. New method We have developed a deep-learning based methodology for classification and registration of thousands of sections of brain tissue, using the mouse olfactory bulb (OB) as a case study. Results We trained a convolutional neural network (CNN) to derive an image similarity measure for in-situ hybridization experiments, and embedded these in a low-dimensional feature space to guide the design of registration templates. We then compiled a high quality, registered atlas of gene expression for the OB (the first such atlas for the OB, to our knowledge). As proof-of-principle, the atlas was clustered using non-negative matrix factorization to reveal canonical expression motifs, and to identify novel, lamina-specific marker genes. Comparison with existing methods Our method leverages virtues of CNNs for a set of important problems in molecular neuroanatomy, with performance comparable to existing methods. Conclusion The atlas we have complied allows for intra- and inter-laminar comparisons of gene expression patterns in the OB across thousands of genes, as well identification of canonical expression profiles through clustering. We anticipate that this will be a useful resource for investigators studying the bulb's development and functional topography. Our methods are publicly available for those interested in extending them to other brain areas. [ABSTRACT FROM AUTHOR]

Abstract A novel multi-focus image fusion algorithm based on deep support value convolutional neural network (DSVCNN) is proposed for multi-focus image fusion. First, a deep support value training network is presented by replacing the empirical risk minimization-based loss function by a loss function based on structural risk minimization during the training of convolutional neural network (CNN). Then, to avoid the loss of information, max-pooling/subsampling of the feature mapping layer of a conventional convolutional neural network, which is employed in all conventional CNN frameworks to reduce the dimensionality of the feature map, is replaced by standard convolutional layers with a stride of two. The experimental results demonstrate that the suggested DSVCNN-based method is competitive with current state-of-the-art approaches and superior to those that use traditional CNN methods. [ABSTRACT FROM AUTHOR]

Highlights • A new attention based CNN-LSTM network is proposed for script identification. • It is based on local and global feature extraction and dynamically weighting them. • Attention used twice to give priority to significant features and to focus on more relevant patches. • The method has been tested on SIW-13, CVSI-2015, ICDAR-17 and MLe2e datasets. Abstract Script identification plays a significant role in analysing documents and videos. In this paper, we focus on the problem of script identification in scene text images and video scripts. Because of low image quality, complex background and similar layout of characters shared by some scripts like Greek, Latin, etc., text recognition in those cases become challenging. In this paper, we propose a novel method that involves extraction of local and global features using CNN-LSTM framework and weighting them dynamically for script identification. First, we convert the images into patches and feed them into a CNN-LSTM framework. Attention-based patch weights are calculated applying softmax layer after LSTM. Next, we do patch-wise multiplication of these weights with corresponding CNN to yield local features. Global features are also extracted from last cell state of LSTM. We employ a fusion technique which dynamically weights the local and global features for an individual patch. Experiments have been done in four public script identification datasets: SIW-13, CVSI2015, ICDAR-17 and MLe2e. The proposed framework achieves superior results in comparison to conventional methods. [ABSTRACT FROM AUTHOR]

Highlights • The CNN model, with the circular Hough transformation, was applied for the first time on retinal images. • The CNN model was used together with image processing methods to increase the success rate. • The success rate of exudate detection is 99.18%. • Developing a clinical decision support system to aid practitioners. Abstract In this study, a combined approach of circular Hough transform and Convolutional Neural Network (CNN) algorithms was proposed for detecting exudates, which is one of the signs of diabetic retinopathy disease. The proposed approach was assessed using DiaretDB0, DiaretDB1 and DrimDB public datasets. This approach consists of visual enhancement with basic pre-processing methods, the segmentation of the OD with the help of circular Hough transformation to ignore the optical disc (OD) regions from the image, and the CNN-based exudate detection system to automatically detect the exudates in the retinal image. In pre-processing and segmentation of the OD region step, adaptive histogram equalization, Canny edge detection algorithm and circular Hough conversion methods are applied respectively to improve retinal images and prevent interference with OD, which is an anatomical region. The images obtained by segmenting and discarding the OD are trained with CNN and subjected to binary classification as exudated and exudate-free image. Then, the method developed with the images not included in the training set was found to have a correct classification ratio of 99.17% in DiaretDB0, 98.53% in DiaretDB1 and 99.18% in DrimDB. This suggests that the results of the proposed approach are more successful than the results obtained using CNN-only or image processing methods alone. Finally, it is seen that the proposed method that applying CNN to the output image of the image processing result, is more successful than the other methods. [ABSTRACT FROM AUTHOR]

Abstract The number of kernels in a convolutional neural network (CNN) can have a significant impact on the performance of the CNN in the aspects of accuracy and computation efficiency. However, existing approaches to determining the number of convolution kernels are mainly conducted through a manual process, which suffers from the problems of potential overfitting, instability and inefficiency. In response to these problems, this paper presents a corner radiation area adaptation (CRAA) based method to automatically determine the number of convolution kernels. CRAA is evaluated in comparison with three representative methods on multiple public data sets. Experimental results show that CRAA is robust to the number of convolution kernels which is well adapted to a specific data set and achieves a higher level of classification accuracy by 3% when spending the same period of time in classification. More importantly, CRAA reduces the computational time by 15% in comparison with the three representative approaches when reaching the same level of accuracy in classification. [ABSTRACT FROM AUTHOR]

Highlights • We propose a personality prediction method based on image scene perception. • Users with the same personality have similar preferences for image scenes. • A number of image scenes are recognized to solve a user-centric prediction problem. Graphical abstract Abstract People tend to have different preferences for image scenes, which are indicative of their personality traits. Inspired by the relationship between users' attributed personality traits and preferred image scenes, this letter proposes a personality prediction method based on image scene perception probability. Firstly, image scenes are recognized based on the convolutional neural network. Secondly, the probability distribution of perceived image scenes is calculated. Finally, a regression model is trained using the scene probability distribution features to predict the attributed personality traits. The experimental results on the annotated PshycoFlickr dataset show that the proposed method is superior to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• A surrogate model for distributed hydrological models is developed using deep networks. • The surrogate model can handle high-dimensional input–output maps of VIC. • The surrogate model can capture the dynamic relationship between the time-varying input and output. • The surrogate model outperforms LSTM in approximating spatial patterns of VIC output. Surrogate modelling methods have been used to estimate spatially distributed parameters of land surface hydrological models due to their computational efficiency. However, traditional surrogate techniques have trouble in modelling high-dimensional input–output maps. In addition, their applications to a distributed model over large spatial and temporal domains are very unlikely to produce consistent spatial pattern and dynamic relationship of model input and output, leading to unbalanced model performance across different spatiotemporal domains. To address these issues, we employ a deep autoregressive neural network to construct an accurate and reliable surrogate system of distributed dynamic model outputs. The network uses a deep convolutional encoder-decoder architecture to take full advantage of the deep convolutional layers in high-dimensional image processing and spatial feature extraction. Moreover, the autoregressive strategy makes the network good at handling dynamic relationship between the time-varying model input and output. We apply this deep learning (DL) network to building a surrogate model of the Variable Infiltration Capacity (VIC) model simulated soil moisture over the Huaihe river basin of China. The results show that the adopted deep autoregressive neural network can provide an accurate surrogate model of the high-dimensional dynamic relationship between 15 input fields and 1 output field, each covering 408 grids, over multiple years. This surrogate model significantly outperforms the popular long short-term memory (LSTM) model, achieving an average mean squared error (MSE) of 0.26 and an average R 2 of 0.76 using the test datasets. The surrogate's MSE and R 2 performance represents an average improvement of 43% and 33%, respectively, compared to the LSTM model. In addition, it shows better spatial performance than the LSTM model, demonstrating its superior performance in approximating spatial patterns of VIC simulated soil moisture. The accuracy of this surrogate method and its ability to handle high-dimensional data is promising for advancing parameter uncertainty quantification of distributed land surface hydrological models, which usually have high dimensionality due to the large number of variables involved, including distributed parameters, input forcing variables, and output variables. [ABSTRACT FROM AUTHOR]

• We propose a spatial linkage module (SLM) to address the issue of lost low-level feature information. • We propose a channel linkage module (CLM) to capture the dependency relationship of feature channels. • We propose a transmodal synaptic linkage (TSL) module to guide the feature fusion within the SCLMnet. • We conduct extensive comparative and ablation experiments on three public datasets for lung and lung lobe segmentation. Lung and lung lobe segmentation are two crucial techniques for lung imaging analysis that interact in clinical settings. Lung segmentation assists physicians in comparing different images to select the most appropriate surgical plan, while lobe segmentation provides precise anatomical information to help plan surgical procedures. However, inaccurate lung segmentation edges, mis-segmented lobe boundaries, and tiny targets pose challenges. Therefore, we propose a dual-branch guided convolutional neural network, SCLMnet, for lung and lung lobe segmentation. To completely leverage the semantic information of feature maps, the first branch adds a spatial linkage module (SLM) to focus on low-level features at different spatial levels, highlighting feature representations of lung edges and lung lobe boundaries. A channel linkage module (CLM) is added by matrix inner product to model channel relations, emphasizing the relevance and similarity of feature maps and capturing the interdependency of high-level feature channels to highlight feature representations of the entire lung lobe region. Transmodal synaptic linkage (TSL) and multi-scale fusion strategy guide the feature information of the CLM and SLM and the deep features extracted by the second branch ResUNet to jointly explore useful information in chest computer tomography (CT) images. To evaluate the performance of the state-of-the-art model, we use three publicly available datasets: LUNA16, COVID-19-CT-Seg, and VESSEL12. Compared to the existing methods, SCLMnet achieves average Dice scores of 92.17%, 97.80%, and 99.12%, respectively, demonstrating remarkable performance, which suggests that lung and lung lobe segmentation using CT images with SCLMnet can play an essential role in clinical research. [ABSTRACT FROM AUTHOR]