Search

Deep learning, specifically the Convolutional Neural Network (CNN), has been a significant technology tool for image processing and human health. CNNs, which mimic the working principles of the human brain, can learn robust representations of images. However, CNNs are susceptible to noise interference, which can impact classification performance. Choosing the right activation function can improve CNNs performance and accuracy. This research aims to test the accuracy of CNN with ResNet50, VGG16, and GoogleNet architectures combined with several activation functions such as ReLU, Leaky ReLU, Sigmoid, and Tanh in the classification of images that experience Poisson noise. Poisson noise is applied to each test data to evaluate CNN accuracy. The data used in this study consists of three scenarios of different numbers of classes, namely 3 classes, 5 classes, and 10 classes. The results showed that combining ResNet50 with the ReLU activation function produced the best performance in class recognition in each scenario of the number of classes experiencing Poisson noise interference. The model achieved 97% accuracy for 3-class data, 95% for 5-class data, and 90% for 10-class data. These results show that using ResNet50 with the ReLU activation function can provide excellent resistance to Poisson noise in image processing. It was found that as the number of classes increases, the accuracy of image recognition tends to decrease. This shows that the more complex the image classification task is with a larger number of classes, the more difficult it is for CNNs to distinguish between different classes. Doi: 10.28991/ESJ-2024-08-02-014 Full Text: PDF

Breast cancer is the most widespread form of cancer diseases among women. Such oncoviral cancer starts in the epithelial lining of the lobules or ducts in the breast gland tissue. Identifying and classifying breast cancer presents a significant challenge for researchers and scientists. Neural networks have emerged as a powerful tool for classifying cancer data through feature extraction. This paper addresses the challenge of accurately classifying breast cancer using a novel approach that combines a hybrid sigmoid activation function (HSAF) with transfer learning, utilizing the pre-trained EfficientNetB6 model. The HSAF is specifically designed to capture complex patterns within histopathological images, while transfer learning leverages prior knowledge from the pre-trained model. In our experimental approach, we employ a breast histopathological image dataset, dividing it into three segments: 60% for training, 20% for validation, and 20% for testing. Furthermore, data augmentation techniques are performed to increase the size of training data. The experimental results of this research indicate an impressive precision, recall, and F1 score of 91%. Furthermore, our proposed model is compared to existing methods, demonstrating its efficiency. We also conduct a comparative study of activation functions (AFs), highlighting the classification performance of HSAF for breast cancer. This research not only advances our ability to classify breast cancer more accurately but also serves as a catalyst for raising awareness and alleviating concerns related to breast cancer. By integrating advanced technology and innovative techniques, this paper aims to make a meaningful contribution to the early detection and effective treatment of this widespread and life-affecting disease. [ABSTRACT FROM AUTHOR]

Recent research has found that the activation function (AF) plays a significant role in introducing non-linearity to enhance the performance of deep learning networks. Researchers recently started developing activation functions that can be trained throughout the learning process, known as trainable, or adaptive activation functions (AAF). Research on AAF that enhances the outcomes is still in its early stages. In this paper, a novel activation function ‘ErfReLU’ has been developed based on the erf function and ReLU. This function leverages the advantages of both the Rectified Linear Unit (ReLU) and the error function (erf). A comprehensive overview of activation functions like Sigmoid, ReLU, Tanh, and their properties have been briefly explained. Adaptive activation functions like Tanhsoft1, Tanhsoft2, Tanhsoft3, TanhLU, SAAF, ErfAct, Pserf, Smish, and Serf is also presented. Lastly, comparative performance analysis of 9 trainable activation functions namely Tanhsoft1, Tanhsoft2, Tanhsoft3, TanhLU, SAAF, ErfAct, Pserf, Smish, and Serf with the proposed one has been performed. These activation functions are used in MobileNet, VGG16, and ResNet models and their performance is evaluated on benchmark datasets such as CIFAR-10, MNIST, and FMNIST. [ABSTRACT FROM AUTHOR]

The problem of inverting dynamic complex matrices remains a central and intricate challenge that has garnered significant attention in scientific and mathematical research. The zeroing neural network (ZNN) has been a notable approach, utilizing time derivatives for real-time solutions in noiseless settings. However, real-world disturbances pose a significant challenge to a ZNN's convergence. We design an accelerated dual-integral structure zeroing neural network (ADISZNN), which can enhance convergence and restrict linear noise, particularly in complex domains. Based on the Lyapunov principle, theoretical analysis proves the convergence and robustness of ADISZNN. We have selectively integrated the SBPAF activation function, and through theoretical dissection and comparative experimental validation we have affirmed the efficacy and accuracy of our activation function selection strategy. After conducting numerous experiments, we discovered oscillations and improved the model accordingly, resulting in the ADISZNN-Stable model. This advanced model surpasses current models in both linear noisy and noise-free environments, delivering a more rapid and stable convergence, marking a significant leap forward in the field. [ABSTRACT FROM AUTHOR]

In this paper we consider a new class of RBF (Radial Basis Function) neural networks, in which smoothing factors are replaced with shifts. We prove under certain conditions on the activation function that these networks are capable of approximating any continuous multivariate function on any compact subset of the d-dimensional Euclidean space. For RBF networks with finitely many fixed centroids we describe conditions guaranteeing approximation with arbitrary precision. [ABSTRACT FROM AUTHOR]

In this study, the advancing and receding contact angles of three types of pure liquid on four different surfaces have been measured. The contact angles of the droplet have been measured by analysing the captured photo by tilted plate method. Droplet behaviour has been modelled by two experimental equations. Additionally, the data has been modelled by an artificial neural network using genetic algorithm and a hyperbolic tangent activation function. The input parameters are density, molecular weight of pure liquid and solid, viscosity and surface tension of pure liquid, roughness of the solid surface and the two outputs are advancing and receding contact angles of the droplet. Number of 81 data points were used for training, 27 data for validation and 28 data for testing. The topography of {7,7,2} for artificial neural network has been proposed. The resulting RMS errors were 8%, 8% and 7% for training, validation and testing, respectively. [ABSTRACT FROM AUTHOR]

Aiming at the existing deep learning classification model for power corridor point cloud still need to improve the classification efficiency and the robustness of the classification model to meet the requirements of practical applications. An improved classification model based on PointNetþþ is proposed. Based on the fact that the main features of the power corridor scene are power lines, poles, and vegetation, the initial data are first optimally filtered, and then the ensemble abstraction module of the classical PointNetþþ is modified to better adapt to the power corridor scene. Finally, h-Swish is used as the activation function to realize the accurate classification of the features of the power corridor scene, and the training time of deep learning is also greatly reduced. The experimental results show that the improved algorithm achieves an average F1 value of 97.58%, which is 3.62 percentage points higher than the classical PointNetþþ. Therefore, the algorithm has great potential in point cloud classification. [ABSTRACT FROM AUTHOR]

Existing learning-based image deblurring algorithms tend to focus on single source of image information, and the network structure and dynamic scene blur characteristics make it difficult to recover the missing details of the image. Therefore, a novel dynamic scene deblurring framework is proposed based on hybrid activation and edge-assisted dual-branch residuals. Specifically, the network's ability to learn nonlinear features is enhanced by different activation functions, and the feature utilization at different semantic levels is improved by improving the traditional residual structure. In particular, the fixed-parameter training method is adopted to reduce ringing artifacts. And a new dual-source edge extraction algorithm is designed that organically combines edge information from different sources as network inputs. The experimental results demonstrate that our algorithm not only shows advantages in objective evaluation metrics PSNR, SSIM and VIF, but also achieves satisfactory results in subjective visual effects. Source code is publicly available at: https://github.com/Mangolzh/HN.git. [ABSTRACT FROM AUTHOR]

Funahashi established that the space of two-layer feedforward neural networks is dense in the space of all continuous functions defined over compact sets in n-dimensional Euclidean space. The purpose of this short survey is to reexamine the proof of Theorem 1 in Funahashi [3]. The Tietze extension theorem, whose proof is contained in the appendix, will be used. This paper is based on harmonic analysis, real analysis, and Fourier analysis. However, the audience in this paper is supposed to be researchers who do not specialize in these fields of mathematics. Some fundamental facts that are used in this paper without proofs will be collected after we present some notation in this paper. [ABSTRACT FROM AUTHOR]

We propose a neural-network-based watermarking method that introduces the quantized activation function that approximates the quantization of JPEG compression. Many neural-network-based watermarking methods have been proposed. Conventional methods have acquired robustness against various attacks by introducing an attack simulation layer between the embedding network and the extraction network. The quantization process of JPEG compression is replaced by the noise addition process in the attack layer of conventional methods. In this paper, we propose a quantized activation function that can simulate the JPEG quantization standard as it is in order to improve the robustness against the JPEG compression. Our quantized activation function consists of several hyperbolic tangent functions and is applied as an activation function for neural networks. Our network was introduced in the attack layer of ReDMark proposed by Ahmadi et al. to compare it with their method. That is, the embedding and extraction networks had the same structure. We compared the usual JPEG compressed images and the images applying the quantized activation function. The results showed that a network with quantized activation functions can approximate JPEG compression with high accuracy. We also compared the bit error rate (BER) of estimated watermarks generated by our network with those generated by ReDMark. We found that our network was able to produce estimated watermarks with lower BERs than those of ReDMark. Therefore, our network outperformed the conventional method with respect to image quality and BER. [ABSTRACT FROM AUTHOR]

Copyright of International Journal of Engineering Research & Development (IJERAD) is the property of International Journal of Engineering Research & Development and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This article addresses the semantic segmentation of laparoscopic surgery images, placing special emphasis on the segmentation of structures with a smaller number of observations. As a result of this study, adjustment parameters are proposed for deep neural network architectures, enabling a robust segmentation of all structures in the surgical scene. The U-Net architecture with five encoder–decoders (U-Net5ed), SegNet-VGG19, and DeepLabv3+ employing different backbones are implemented. Three main experiments are conducted, working with Rectified Linear Unit (ReLU), Gaussian Error Linear Unit (GELU), and Swish activation functions. The applied loss functions include Cross Entropy (CE), Focal Loss (FL), Tversky Loss (TL), Dice Loss (DiL), Cross Entropy Dice Loss (CEDL), and Cross Entropy Tversky Loss (CETL). The performance of Stochastic Gradient Descent with momentum (SGDM) and Adaptive Moment Estimation (Adam) optimizers is compared. It is qualitatively and quantitatively confirmed that DeepLabv3+ and U-Net5ed architectures yield the best results. The DeepLabv3+ architecture with the ResNet-50 backbone, Swish activation function, and CETL loss function reports a Mean Accuracy (MAcc) of 0.976 and Mean Intersection over Union (MIoU) of 0.977. The semantic segmentation of structures with a smaller number of observations, such as the hepatic vein, cystic duct, Liver Ligament, and blood, verifies that the obtained results are very competitive and promising compared to the consulted literature. The proposed selected parameters were validated in the YOLOv9 architecture, which showed an improvement in semantic segmentation compared to the results obtained with the original architecture. [ABSTRACT FROM AUTHOR]

Wide application of proportional-integral-differential (PID)-regulator in industry requires constant improvement of methods of its parameters adjustment. The paper deals with the issues of optimization of PID-regulator parameters with the use of neural network technology methods. A methodology for choosing the architecture (structure) of neural network optimizer is proposed, which consists in determining the number of layers, the number of neurons in each layer, as well as the form and type of activation function. Algorithms of neural network training based on the application of the method of minimizing the mismatch between the regulated value and the target value are developed. The method of back propagation of gradients is proposed to select the optimal training rate of neurons of the neural network. The neural network optimizer, which is a superstructure of the linear PID controller, allows increasing the regulation accuracy from 0.23 to 0.09, thus reducing the power consumption from 65% to 53%. The results of the conducted experiments allow us to conclude that the created neural superstructure may well become a prototype of an automatic voltage regulator (AVR)-type industrial controller for tuning the parameters of the PID controller. [ABSTRACT FROM AUTHOR]

The selection of weight initialization in an artificial neural network is one of the key aspects and affects the learning speed, convergence rate and correctness of classification by an artificial neural network. In this paper, we investigate the effects of weight initialization in an artificial neural network. Nguyen-Widrow weight initialization, random initialization, and Xavier initialization method are paired with five different activation functions. This paper deals with a feedforward neural network, consisting of an input layer, a hidden layer, and an output layer. The paired combination of weight initialization methods with activation functions are examined and tested and compared based on their best achieved loss rate in training. This work aims to better understand how weight initialization methods in neural networks, in combination with activation functions, affect the learning speed in comparison after a fixed number of training epochs. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging has a strong capability respecting distinguishing surface objects due to the ability of collect hundreds of bands along the electromagnetic spectrum. Hyperspectral image classification, one of the major tasks of hyperspectral image processing, is challenging process due to the characteristics of the considered dataset. Along with a variety of traditional algorithms, the convolutional neural network (CNN) has gained popularity in recent days thanks to its excellent performance. Activation functions and optimizers have a crucial role in learning process of CNN model. In this paper, a comparative analysis using a set of different activation functions and optimizers was performed. For this purpose, six different activation functions, LReLU, Mish, PReLU, ReLU, Sigmoid, and Swish, and four different optimizers, Adam, Adamax, Nadam, and RMSProp, were utilized on a CNN model. Two publicly available datasets, named Indian Pines and WHU-Hi HongHu, were used in the experiments. According to the results, the CNN model using Adamax optimizer and Mish activation function had the best overall accuracies for the Indian Pines WHU-Hi HongHu dataset at 98.32% and 97.54%, respectively. [ABSTRACT FROM AUTHOR]

Streaming services are increasingly leveraging Artificial Intelligence (AI) technologies for improved content cataloging, user experiences in content discovery, and personalization. A significant challenge in this domain is the automated assignment of microgenres to movies. This study introduces and evaluates approaches based on clustering, topic modeling, and word embedding to address this task. The evaluation employs a preprocessed dataset containing movie-related data—title tags, synopses, genres, and reviews—alongside a predefined microgenre list. Comparisons of three activation functions (binary step, ramp, and sigmoid) gauge their effectiveness in augmenting microgenre tags. Results demonstrate the superiority of the word embedding approach over clustering and topic modeling in terms of mean accuracy. Even more, the word embedding approach stands as the sole fully automated solution. Analysis indicates that incorporating review-based tags introduces noise and undermines accuracy. Besides, the word embedding approach yields optimal outcomes using the sigmoid function, effectively doubling assigned tags while maintaining matching quality. This sheds light on the potential of word embedding methods within the movie domain. [ABSTRACT FROM AUTHOR]

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Unsupervised learning based on restricted Boltzmann machine or autoencoders has become an important research domain in the area of neural networks. In this paper mathematical expressions to adaptive learning step calculation for RBM with ReLU transfer function are proposed. As a result, we can automatically estimate the step size that minimizes the loss function of the neural network and correspondingly update the learning step in every iteration. We give a theoretical justification for the proposed adaptive learning rate approach, which is based on the steepest descent method. The proposed technique for adaptive learning rate estimation is compared with the existing constant step and Adam methods in terms of generalization ability and loss function. We demonstrate that the proposed approach provides better performance. [ABSTRACT FROM AUTHOR]

This study addresses the challenge of accurately estimating air pollution levels, which pose significant health, environmental, and economic risks. Variations in air quality across different regions, with urban and power plant areas typically experiencing higher pollution levels, highlight the need for effective monitoring methods beyond traditional sensor-based approaches. This study proposes a method to estimate air pollution levels from images using a Convolution Neural Network (CNN) model, aiming to overcome the limitations of traditional monitoring stations. The proposed method leverages the Resnet-152 architecture to accurately estimate Particulate Matter (PM2.5) concentrations, benefiting from its implicit and invariant distortion features tailored for particulate matter modeling. Hyperparameter tuning during image training and using max-pooling layers with three kernels and one stride helps mitigate overfitting issues. The application of max-group layers facilitates the extraction of relevant information from activation maps, enhancing estimation precision. The Resnet-152 architecture with fewer parameters and invariant distortion characteristics, accelerates particulate matter estimation. Experimental results demonstrate the effectiveness of the proposed method, with a root mean square error (RMSE) of 0.10 and a mean absolute percentage error (MAPE) of 19.38%, outperforming other models such as Inception-v3, VGG-19, and Googlenet, thus showcasing its potential for practical air pollution monitoring applications. [ABSTRACT FROM AUTHOR]

Precise and rapid detection of seatbelts is an essential research field for intelligent traffic management. In order to improve the detection precision of seatbelts and speed up algorithm inference velocity, a lightweight seatbelt detection algorithm is proposed. Firstly, by adding the G-ELAN module designed in this paper to the YOLOv7-tiny network, the optimization of construction and reduction of parameters are accomplished, and the ResNet is compressed with the channel pruning approach to decrease computational overheads. Then, the Mish activation function is utilized to replace the Leaky Relu in the neck to enhance the non-linear competence of the network. Finally, the triplet attention module is integrated into the model after pruning to make up for the underlying performance reduction caused by the previous stage and upgrade overall detection precision. The experimental results based on the self-built seatbelt dataset showed that, compared to the initial network, the Mean Average Precision (mAP) achieved by the proposed GM-YOLOv7 was improved by 3.8%, while the volume and the computation amount were lowered by 20% and 24.6%, respectively. Compared with YOLOv3, YOLOX, and YOLOv5, the mAP of GM-YOLOv7 increased by 22.4%, 4.6%, and 4.2%, respectively, and the number of computational operations decreased by 25%, 63%, and 38%, respectively. In addition, the accuracy of the improved RST-Net increased to 98.25%, while the parameter value was reduced by 48% compared to the basic model, effectively improving the detection performance and realizing a lightweight structure. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Pedestrian detection is crucial for various applications, including intelligent transportation and video surveillance systems. Although recent research has advanced pedestrian detection models like the YOLO series, they still face limitations in handling diverse pedestrian scales, leading to performance challenges. To address these issues, we propose HF-YOLO, an advanced pedestrian detection model. HF-YOLO tackles the complexities of pedestrian detection in complex scenes by addressing scale variations and occlusions among pedestrians. In the feature fusion stage, our algorithm leverages both shallow localization information and deep semantic information. This involves fusing P2 layer features and adding a high-resolution detection layer, significantly improving the detection of small-scale pedestrians and occluded instances. To enhance feature representation, HF-YOLO incorporates the HardSwish activation function, introducing more non-linear factors and strengthening the model’s ability to represent complex and discriminative features. Additionally, to address regression imbalance, a balance factor is introduced to the CIoU loss function. This modification effectively resolves the imbalance problem and enhances pedestrian localization accuracy. Experimental results demonstrate the effectiveness of our proposed algorithm. HF-YOLO achieves notable improvements, including a 3.52% increase in average precision, a 1.35% boost in accuracy, and a 4.83% enhancement in recall. Moreover, the algorithm maintains real-time performance with a detection time of 8.5ms, meeting the stringent requirements of real-time applications. [ABSTRACT FROM AUTHOR]

A garment image instance segmentation method based on improved YOLACT was proposed to solve the problem of low accuracy and speed of clothing image instance segmentation. Based on the YOLACT model, firstly, the depth separable convolution was used in the ResNetlOl network to replace the traditional convolution, reduce the amount of model calculation and model parameters, and accelerate the speed of the model. Then, the efficient channel attention module was introduced to optimize the output features after the protonet, capture the cross-channel interaction information of the clothing image, and strengthen the feature extraction ability of mask branches. Finally, the Leaky ReLU activation function was used in the training process to ensure that the weight information is updated in time, and to improve the model's ability to extract the negative feature information of the clothing image. The experimental results show that compared with the original model, the proposed method can effectively reduce the number of model parameters, and increase the accuracy and the speed. The speed increased by 4. 82 frame per second, and the average accuracy increased by 5. 4%. [ABSTRACT FROM AUTHOR]

Copyright of Automotive Digest is the property of Automotive Digest Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Economics & Administrative Sciences is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Irrigation & Drainage is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

A vehicle engine cooling system is of utmost importance to ensure that the engine operates in a safe temperature range. In most radiators that are used to cool an engine, water serves as a cooling fluid. The performance of a radiator in terms of heat transmission is significantly influenced by the incorporation of nanoparticles into the cooling water. Concentration and uniformity of nanoparticle distribution are the two major factors for the practical use of nanofluids. The shape and size of nanoparticles also have a great impact on the performance of heat transfer. Many researchers are investigating the impact of nanoparticles on heat transfer. This study aims to develop an artificial neural network (ANN) model for predicting the thermal conductivity of an ethylene glycol (EG)/water-based crystalline nanocellulose (CNC) nanofluid for cooling internal combustion engine. The implementation of an artificial neural network considering different activation functions in the hidden layer is made to find the best model for the cooling of an engine using the nanofluid. Accuracies of the model with different activation functions in artificial neural networks are analyzed for different nanofluid concentrations and temperatures. In artificial neural networks, Levenberg–Marquardt is an optimization approach used with activation functions, including Tansig and Logsig functions in the training phase. The findings of each training, testing, and validation phase are presented to demonstrate the network that provides the highest level of accuracy. The best result was obtained with Tansig, which has a correlation of 0.99903 and an error of 3.7959 ×10. It has also been noticed that the Logsig function can also be a good model due to its correlation of 0.99890 and an error of 4.9218 ×10. Thus our ANN with Tansig and Logsig functions demonstrates a high correlation between the actual output and the predicted output. Graphic Abstract [ABSTRACT FROM AUTHOR]

In Music Information Retrieval (MIR), a general goal is to recognize times of novelty within music recordings. This includes estimating structural boundaries through the detection of changes in harmony, tempo, or instrumentation and identifying onsets of note and sound events by capturing changes in the music signal’s energy or spectral content. These tasks leverage novelty functions, which are one-dimensional, time-dependent functions characterized by sharp local maxima that indicate significant musical and acoustical changes. From a given music recording, novelty functions can be derived using a variety of methods, ranging from traditional signal-processing techniques to modern data-driven approaches, where they are often termed “activation functions.” In this tutorial, we explore the concept of novelty functions and some of their essential properties. We discuss methods to enhance these functions and improve their distinctive peak-like structures. These improvements are crucial for simplifying the identification of specific musical events using post-processing methods, from basic peak picking to more sophisticated approaches like periodicity analysis. We also assess novelty functions through commonly used metrics such as precision, recall, and F-measure but with an emphasis on error tolerance. Aimed at Bachelor’s degree and beginning Master’s degree students with basic knowledge of signal processing and mathematics, this tutorial uses illustrative figures to clarify key concepts, thereby broadening its accessibility to a wider MIR audience and enriching their comprehension of this significant subject. Furthermore, Jupyter notebooks, including Python source code for the core algorithms and audio examples that allow for reproducing the tutorial’s figures, are provided at https://github.com/groupmm/edu_novfct.

Aiming at the problem of low detection accuracy or missing detection caused by dense vehicles and small scale of distant vehicles in the visual detection of urban road scenes, an improved YOLOv3-SPP algorithm was proposed to optimize the activation function and take DIOU-NMS Loss as the boundary frame loss function to enhance the expression ability of the network.In order to improve the feature extraction ability of the proposed algorithm for small targets and occluding targets, the void convolution module was introduced to increase the receptive field of the target.Based on the experimental results, the proposed algorithm improves the mAP by 1.79% when detecting vehicle targets, and also effectively reduce the missing phenomenon when detecting tight vehicle targets.

An unbalanced protocol recognition method based on the improved Residual U-Net was proposed to solve the challenge of network security posed by the increasing network attacks with the continuous development of the Internet.In the captured network traffic, a small proportion is constituted by malicious traffic, typically utilizing minority protocols.However, existing protocol recognition methods struggle to accurately identify these minority protocols when the class distribution of the protocol data is imbalanced.To address this issue, an unbalanced protocol recognition method was proposed, which utilized the improved Residual U-Net, incorporating a novel activation function and the Squeeze-and-Excitation Networks (SE-Net) to enhance the feature extraction capability.The loss function employed in the proposed model was the weighted Dice loss function.In cases where the recognition accuracies of the minority protocols were low, the loss function value would be high.Consequently, the optimization direction of the model would be dominated by the minority protocols, resulting in improved recognition accuracies for them.During the protocol recognition process, the network flow was extracted from the network traffic and preprocessed to convert it into a one-dimensional matrix.Subsequently, the protocol recognition model extracted the features of the protocol data, and the Softmax classifier predicted the protocol types.Experimental results demonstrate that the proposed protocol recognition model achieves more accurate recognition of the minority protocols compared to the comparison model, while also improving the recognition accuracies of the majority protocols.

VGG16 is a convolutional neural network model used for image recognition. It is unique in that it only has 16 weighted layers, rather than relying on a large number of hyperparameters. It is considered one of the best vision model architectures. However, several things need to be improved to increase the accuracy of image recognition. In this context, this work proposes and investigates two ensemble CNNs using transfer learning and compares them with state-of-the-art CNN architectures. This study compares the performance of (rectified linear unit) ReLU and sigmoid activation functions on CNN models for animal classification. To choose which model to use, we tested two state-of-the-art CNN architectures: the default VGG16 with the proposed method VGG16. A dataset consisting of 2,000 images of five different animals was used. The results show that ReLU achieves a higher classification accuracy than sigmoid. The model with ReLU in fully connected and convolutional layers achieved the highest precision of 97.56% in the test dataset. The research aims to find better activation functions and identify factors that influence model performance. The dataset consists of animal images collected from Kaggle, including cats, cows, elephants, horses, and sheep. It is divided into training sets and test sets (ratio 80:20). The CNN model has two convolution layers and two fully connected layers. ReLU and sigmoid activation functions with different learning rates are used. Evaluation metrics include accuracy, precision, recall, F1 score, and test cost. ReLU outperforms sigmoid in accuracy, precision, recall, and F1 score. This study emphasizes the importance of choosing the right activation function for better classification accuracy. ReLU is identified as effective in solving the vanish-gradient problem. These findings can guide future research to improve CNN models in animal classification.

In addressing the complex challenge of real-time and precise recognition of umpire signals in sporting events, we introduce ICCG-YOLO, a rapid and effective approach that builds upon the YOLO-v5 architecture. Our method innovatively incorporates Involution operations within the CSP components for superior spatial information modeling and channel-wise parameter sharing, significantly reducing parameter count while capturing extensive features through larger kernels. This stable approach between computational resource & performance and detection accuracy is further enhanced by integrating the CoordAttention block for precise localization and recognition of signals with minimal parameter addition. We also refine the model’s up-sampling process with the CARAFE (Content-Aware ReAssembly of FEatures) block, enabling content-driven feature enlargement that amplifies the receptive field without compromising the model’s compact stature. Complementing this, model compression is achieved using Ghost convolution, capitalizing on simple linear transformations for feature generation, paired with an improved activation function to activate all neural network neurons fully. This results in a multi-scale detection capability while maintaining a moderate depth and reducing overall model complexity. Our experiments on a custom dataset for umpire signal detection and the Chalearn dataset for general gesture recognition in diverse scenarios have successfully validated the high accuracy, rapid processing, and versatility of ICCG-YOLO.

Generative Adversarial Networks (GANs) have gained considerable attention owing to their impressive ability to generate high-quality, realistic images from a desired data distribution. This research introduces advancements in GANs by developing an improved activation function, a novel training strategy, and an adaptive rank decomposition method to compress the network. The proposed activation function, called Parametric Mish (PMish), automatically adjusts a trainable parameter to control the smoothness and shape of the activation function. Our method employs a Neural Architecture Search (NAS) to discover the optimal architecture for image generation while using the Maximum Mean Discrepancy (MMD) repulsive loss for adversarial training. The proposed novel training strategy improves performance by progressively increasing the upper bound of the bounded MMD-GAN repulsive loss. Finally, the proposed Adaptive Rank Decomposition (ARD) method reduces the complexity of the network with minimal impact on its generative performance, thus enabling efficient deployment on resource-limited platforms. The effectiveness of these advancements is rigorously tested on standard benchmark datasets such as CIFAR-10, CIFAR-100, STL-10, and CelebA, where significant improvements over existing techniques are demonstrated. The implementation code is available at: https://github.com/PrasannaPulakurthi/MMD-PMish-NAS

Machine learning algorithms based on deep neural networks have been widely used in many fields especially in computer vision, with impressive results. However, these models are vulnerable to different types of attacks like adversarial ones, which require attention to the model security and confidentiality. This study proposes a defense strategy to improve the insurance of white-box models while minimizing adversarial attacks against Fast Gradient Sign Method (FGSM)-based attacks and tackling the issue of adversarial training to improve their robustness. Mainly, we proposed a CNN model based on personalized activation functions using Beta function and its primitive. Then, the new resulted low degree polynomials are used to approximate the ReLU, Sigmoid and Tanh activation functions. Batch normalization was evoked to significantly improve the learning capacity of neural networks. The obtained results, using Mnist dataset prove the effectiveness of the proposed model.

The time-varying Lyapunov equation plays a central role in the stability analysis and control of dynamic systems across various fields. Traditional methods for dealing with time-varying Lyapunov equations often face challenges such as slow convergence rates and sensitivity to environmental noise. To address these challenges, this paper proposes a novel hybrid neural network model known as the dual integral structure ZNN model. The proposed model integrates an innovative arctangent activation function and designs corresponding arctangent exponential time-varying convergence parameters, known as the VPDIAZNN model, along with another model featuring linear exponential time-varying convergence parameters, referred to as the VPDIZNN model. Theoretical analysis proves the stability and robustness of the models under linear noise. Additionally, comparative experiments demonstrate that in bounded noise and linear noise environments, both the VPDIAZNN and VPDIZNN models can converge quickly, achieving accuracies of up to ${10}^{-6}$ , while the HTVPZNN and NTFZNN models fail to converge. When computing high-order matrices, our models achieve the highest precision and even reach up to ${10}^{-10}$ .

The dynamic matrix square root (DMSR) problem is a recurring nonlinear challenge in many engineering disciplines. Although the original zeroing neural network (OZNN) shows potential for handling such problems, it faces challenges in robustness and convergence. To address these limitations, we developed a simplified activation function, the simplified signal power activation function (SSPAF), which accelerates convergence. Building on this, we introduced a dual integral enhancement term, leading to the design of the rapid dual integral neural dynamic (R-DIND) model to further enhance convergence accuracy, speed, and robustness. Mathematically, the R-DIND model aligns more closely with calculus principles, showcasing unique advantages over existing ZNN models. Theoretical analysis and extensive experiments demonstrated that the R-DIND model has inherent structural advantages over the RNDAC model. Results showed that the R-DIND model improves residual precision from $10^{-2}$ to $10^{-6}$ under the same noise conditions. Furthermore, its robustness and applicability were confirmed when applied to higher-order and more complex matrix problems with harmonic noise.

It is essential to achieve the high-accuracy recognition of low probability of intercept (LPI) radar signals in modern electronic warfare. However, under low signal-to-noise ratio (SNR), the recognition accuracy of the LPI radar signals is relatively low. In this paper, a novel radar signal recognition method based on Convolutional Stacked Recurrent Deep Neural Network (CSRDNN) is proposed. Firstly, we design a Convolutional Neural Network (CNN) to expand the feature space of input time domain signals, the features extracted by CNN were then used as inputs of the Stacked Recurrent Neural Networks (SRNN) module. In the SRNN module, we sequentially stack GRU, LSTM, and BGRU, enabling the model to better handle the short-term and long-term dependence of signal features and effectively solve asynchronous problems in unidirectional RNN networks. Subsequently, a Fully Connected Deep Neural Network (FCDNN) was employed to accomplish the recognition task. In addition, we design a training algorithm composed of the Nesterov-Adaptive Moment Estimation (Nadam) algorithm and the CosineAnnealing Learning Rate (LR) adjustment strategy to improve the training efficiency of the model. The experimental results demonstrate that the proposed model has higher recognition accuracy at low SNR compared to other models, with an overall recognition accuracy of 92.96% at −4 dB.

Matrix inversion is a fundamental operation utilized across numerous disciplines such as mathematics, engineering, and control theory. The original zeroing neural network (OZNN) method has proven effective in tackling the challenge of time-varying matrix inversion (TVMI) under ideal conditions. The integration-enhanced zeroing neural network (IEZNN) is commonly used to handle TVMI issues in the presence of various types of noise. In this paper, we have enhanced the IEZNN model’s tolerance to noise by introducing a dual integral component, resulting in the dual noise tolerant zeroing neural network (DNTZNN) model. We have further improved this model by incorporating a positive odd activation function to create the triple noise tolerant zeroing neural network (TNTZNN). This advancement enables the TNTZNN to effectively solve TVMI problems despite various noise disturbances. Consequently, the TNTZNN model demonstrates excellent convergence and robustness even under noisy conditions. Furthermore, theoretical analysis grounded on the Lyapunov theorem validates the convergence and resilience of the TNTZNN model against diverse forms of noise. Computational simulations further substantiate the superior efficacy of the proposed TNTZNN model in resolving TVMI problems.

The prediction of drug properties plays a vital role in drug research. However, the drug property prediction accuracy of traditional methods is limited due to their inability to fully capture molecular structure and function information. As a result, the use of graph neural networks has attracted significant attention as an effective drug property prediction approach. Nevertheless, traditional graph neural networks still exhibit certain drawbacks in this regard, including their disregard of the interaction information between nodes and edges, the loss of local information during global pooling operations, and the absence of feature fusion mechanisms. This study proposes an enhanced graph neural network (GNN) model that incorporates an attention mechanism, multiscale pooling, an adaptive weight generator, and an activation function to predict drug properties. A comparative analysis with the conventional graph neural network model reveals significant improvements in terms of predicting the side effects of drugs on the heart and liver, with increases of 1%, 7%, and 13%. Furthermore, the enhanced graph neural network model exhibits good performance across the remaining two datasets. Empirical findings underscore the efficacy of the model in drug property prediction tasks, and it is characterized by enhanced predictive precision and robust performance outcomes.

The detection of pavement diseases is an important and basic link in the road maintenance process. Many methods based on deep learning have been applied. However, these methods are not accurate enough and cannot accurately identify defects in complex background with shadow occlusion and uneven lighting brightness. In order to overcome the shortcomings of previous detection methods, a complex background defect detection algorithm based on improved YOLOv7 is proposed. First, the K-means++ clustering algorithm is used for initial anchor box setting to obtain better anchor box parameters; then, the group spatial pyramid pooling module SPPCSPC_G is introduced to replace the original SPPCSPC module to improve the fusion speed of image features and thereby improve the detection accuracy; Finally, the GELU activation function is used as the activation function of the REPConv convolution module in the YOLOv7 model, which effectively reduces model overfitting and thereby improves model detection accuracy. The test results show that the average accuracy of the improved detection algorithm for disease detection increased from 65.4% to 72.3%, an increase of 6.9%, the amount of calculation and parameters decreased by 4% and 14.9% respectively, and the FPS reached 80, an increase of 17%, and no pavement defects are missed or wrongly detected. It is more suitable for real-time detection of defects in complex background. It can be seen that the improved YOLOv7 has better detection effect on complex background defects.

The application of machine learning in healthcare, financial, social media, and other sensitive sectors not only involves high accuracy but privacy as well. Due to the emergence of the Cloud as a computation and one-to-many access paradigm; training and classification/inference tasks have been outsourced to Cloud. However, its usage is limited due to legal and ethical constraints regarding privacy. In this work, we propose a privacy-preserving neural networks-based classification model based on Homomorphic Encryption (HE) where the user can send an encrypted instance to the cloud and receive an encrypted inference from it to preserve the user’s query privacy. In contrast to existing works, we demonstrate the realistic limitations of HE for privacy-preserving machine learning by changing its parameters for enhanced security and accuracy. We showcase scenarios where the choice of HE parameters impedes accurate classification and present an optimized setting for achieving reliable classification. We present several results to demonstrate its effectiveness using MNIST dataset with highly improved inference time for a query as compared to the state of the art.

The study of road traffic flow theory utilizes physics and applied mathematics to analyze relevant parameters and their relationships quanlitatively and quantitatively, in order to explore their dynamic changes. The fluid dynamics model used for traffic flow analysis is highly favored by scholars due to its solid mathematical foundation and good simulation results. However, existing models have two main shortcomings: firstly, existing research is mostly limited to non-viscoelastic fluid equation or incompressible non-Newtonian fluid equation, making it difficult to accurately describe the viscosity state and micro cluster properties of the actual traffic flow; secondly, the existing non-Newtonian fluid partial differential equations (PDEs) rely heavily on the finite element method (FEM) for solving, requiring higher computational cost, larger storage space, and more constraint conditions. Thus, in this paper, a traffic flow equation based on compressible non-Newtonian fluid has been constructed, and it has been solved by using physical-informed rational neural network (PIRNN) and noise heavy-ball acceleration gradient descent (NHAGD) to ensure learning and training speed and accuracy. Numerical results indicate that the proposed method can truly reflect the gradual change process in the viscosity of traffic flow, and has better solving performance than traditional FEM and physical-informed neural network (PINN) with activation functions; under the same conditions, the prediction error of the proposed method is also smaller than that of traditional traffic flow models.

This article presents a length-dependent deep neural network (LD-DNN) based channel modeling methodology to predict the frequency response of high-speed channels. The proposed method significantly enhances the model accuracy and design efficiency while considering the channel length dependence that was neglected in previous modeling approaches. We define the concept of the electrical length to model the length and frequency dependence, then further leverage the activation function to capture the multiple reflection effects to improve accuracy. Additionally, we model the insertion loss resonance induced by crosstalk that can seriously deteriorate signal integrity. As a result, by adopting the proposed model which can predict the S-parameters as a function of length, the need for performing additionally 3D electromagnetic simulations when adjusting the channel length can be eliminated. Various high-speed channel cases are tested to validate the accuracy of the proposed method. The modeling accuracy is less than 4% for different high-speed channel structures with run times of less than 1.4 second per design.

Accurate self-motion estimation is critical for various navigational tasks in mobile robotics. Optic flow provides a means to estimate self-motion using a camera sensor and is particularly valuable in GPS- and radio-denied environments. The present study investigates the influence of different activation functions—ReLU, leaky ReLU, GELU, and Mish—on the accuracy, robustness, and encoding properties of convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) trained to estimate self-motion from optic flow. Our results demonstrate that networks with ReLU and leaky ReLU activation functions not only achieved superior accuracy in self-motion estimation from novel optic flow patterns but also exhibited greater robustness under challenging conditions. The advantages offered by ReLU and leaky ReLU may stem from their ability to induce sparser representations than GELU and Mish do. Our work characterizes the encoding of optic flow in neural networks and highlights how the sparseness induced by ReLU may enhance robust and accurate self-motion estimation from optic flow.

Lightweight convolutional neural network (CNN) models have proven effective in recognizing common pest species, yet challenges remain in enhancing their nonlinear learning capacity and reducing overfitting. This study introduces a grouped dropout strategy and modifies the CNN architecture to improve the accuracy of multi-class insect recognition. Specifically, we optimized the base model by selecting appropriate optimizers, fine-tuning the dropout probability, and adjusting the learning rate decay strategy. Additionally, we replaced ReLU with PReLU and added BatchNorm layers after each Inception layer, enhancing the model’s nonlinear expression and training stability. Leveraging the Inception module’s branching structure and the adaptive grouping properties of the WeDIV clustering algorithm, we developed two grouped dropout models, the iGDnet-IP and GDnet-IP. Experimental results on a dataset containing 20 insect species (15 pests and five beneficial insects) demonstrated an increase in cross-validation accuracy from 84.68% to 92.12%, with notable improvements in the recognition rates for difficult-to-classify species, such as Parnara guttatus Bremer and Grey (PGBG) and Papilio xuthus Linnaeus (PXLL), increasing from 38% and 47% to 62% and 93%, respectively. Furthermore, these models showed significant accuracy advantages over standard dropout methods on test sets, with faster training times compared to four conventional CNN models, highlighting their suitability for mobile applications. Theoretical analyses of model gradients and Fisher information provide further insight into the grouped dropout strategy’s role in improving CNN interpretability for insect recognition tasks.

Background A neural network is a machine learning algorithm that can learn and make predictions by adjusting the strength of the connections between nodes. The sigmoid function is commonly used as an activation function in these nodes. This study explores the potential applicability of biological materials in the development of alternative activation functions. Methods Inspired by the fact that acetylcholine induces intestinal contractions that follow a sigmoid function, we used pharmacological data obtained from guinea pig ilea in a layered neural network for image classification tasks. Results and Conclusions We found that the intestinal data-based neural network with the same structure as a conventional three-layer perceptron achieved an impressive classification accuracy of 85.7% ± 0.6% based on the MNIST handwritten digit dataset (chance = 10%). Additionally, the neural network was trained to determine whether objects in photographs collected from the internet were digestible, achieving an accuracy of 88.5% ± 0.9% (chance = 50%). Our approach highlights the potential applicability of intestine data in neural computations based on pharmacological mechanisms.