Search

Accurate self-motion estimation is critical for various navigational tasks in mobile robotics. Optic flow provides a means to estimate self-motion using a camera sensor and is particularly valuable in GPS- and radio-denied environments. The present study investigates the influence of different activation functions—ReLU, leaky ReLU, GELU, and Mish—on the accuracy, robustness, and encoding properties of convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) trained to estimate self-motion from optic flow. Our results demonstrate that networks with ReLU and leaky ReLU activation functions not only achieved superior accuracy in self-motion estimation from novel optic flow patterns but also exhibited greater robustness under challenging conditions. The advantages offered by ReLU and leaky ReLU may stem from their ability to induce sparser representations than GELU and Mish do. Our work characterizes the encoding of optic flow in neural networks and highlights how the sparseness induced by ReLU may enhance robust and accurate self-motion estimation from optic flow. [ABSTRACT FROM AUTHOR]

Lightweight convolutional neural network (CNN) models have proven effective in recognizing common pest species, yet challenges remain in enhancing their nonlinear learning capacity and reducing overfitting. This study introduces a grouped dropout strategy and modifies the CNN architecture to improve the accuracy of multi-class insect recognition. Specifically, we optimized the base model by selecting appropriate optimizers, fine-tuning the dropout probability, and adjusting the learning rate decay strategy. Additionally, we replaced ReLU with PReLU and added BatchNorm layers after each Inception layer, enhancing the model's nonlinear expression and training stability. Leveraging the Inception module's branching structure and the adaptive grouping properties of the WeDIV clustering algorithm, we developed two grouped dropout models, the iGDnet-IP and GDnet-IP. Experimental results on a dataset containing 20 insect species (15 pests and five beneficial insects) demonstrated an increase in cross-validation accuracy from 84.68% to 92.12%, with notable improvements in the recognition rates for difficult-to-classify species, such as Parnara guttatus Bremer and Grey (PGBG) and Papilio xuthus Linnaeus (PXLL), increasing from 38% and 47% to 62% and 93%, respectively. Furthermore, these models showed significant accuracy advantages over standard dropout methods on test sets, with faster training times compared to four conventional CNN models, highlighting their suitability for mobile applications. Theoretical analyses of model gradients and Fisher information provide further insight into the grouped dropout strategy's role in improving CNN interpretability for insect recognition tasks. [ABSTRACT FROM AUTHOR]

To address the challenges of high experimental costs, complexity, and time consumption associated with pre-mixed combustible gas deflagration experiments under semi-open space obstacle conditions, a rapid temporal prediction method for flame propagation velocity based on Ranger-GRU neural networks is proposed. The deflagration experiment data are employed as the training dataset for the neural network, with the coefficient of determination (R2) and mean squared error (MSE) used as evaluation metrics to assess the predictive performance of the network. First, 108 sets of pre-mixed methane gas deflagration experiments were conducted, varying obstacle parameters to investigate methane deflagration mechanisms under different conditions. The experimental results demonstrate that obstacle-to-ignition source distance, obstacle shape, obstacle length, obstacle quantity, and thick and fine wire mesh obstacles all significantly influence flame propagation velocity. Subsequently, the GRU neural network was trained, and different activation functions (Sigmoid, Relu, PReLU) and optimizers (Lookahead, RAdam, Adam, Ranger) were incorporated into the backpropagation updating process of the network. The training results show that the Ranger-GRU neural network based on the PReLU activation function achieves the highest mean R2 value of 0.96 and the lowest mean MSE value of 7.16759. Therefore, the Ranger-GRU neural network with PReLU activation function can be a viable rapid prediction method for flame propagation velocity in pre-mixed methane gas deflagration experiments under semi-open space obstacle conditions. [ABSTRACT FROM AUTHOR]

Wheat spike count is one of the critical indicators for assessing the growth and yield of wheat. However, illumination variations, mutual occlusion, and background interference have greatly affected wheat spike detection. A lightweight detection method was proposed based on the YOLOv5s. Initially, the original YOLOv5s was improved by combing the additional small-scale detection layer and integrating the ECA (Efficient Channel Attention) attention mechanism into all C3 modules (YOLOv5s + 4 + ECAC3). After comparing GhostNet, ShuffleNetV2, and MobileNetV3, the GhostNet architecture was finally selected as the optimal lightweight model framework based on its superior performance in various evaluations. Subsequently, the incorporation of five different activation functions into the network led to the identification of the RReLU (Randomized Leaky ReLU) activation function as the most effective in augmenting the network's performance. Ultimately, the network's loss function of CIoU (Complete Intersection over Union) was optimized using the EIoU (Efficient Intersection over Union) loss function. Despite a minor reduction of 2.17% in accuracy for the refined YOLOv5s + 4 + ECAC3 + G + RR + E network when compared to the YOLOv5s + 4 + ECAC3, there was a marginal improvement of 0.77% over the original YOLOv5s. Furthermore, the parameter count was diminished by 32% and 28.2% relative to the YOLOv5s + 4 + ECAC3 and YOLOv5s, respectively. The model size was reduced by 28.0% and 20%, and the Giga Floating-point Operations Per Second (GFLOPs) were lowered by 33.2% and 9.5%, respectively, signifying a substantial improvement in the network's efficiency without significantly compromising accuracy. This study offers a methodological reference for the rapid and accurate detection of agricultural objects through the enhancement of a deep learning network. [ABSTRACT FROM AUTHOR]

Here we study the quantitative multivariate approximation of perturbed hyperbolic tangent-activated singular integral operators to the unit operator. The engaged neural network activation function is both parametrized and deformed, and the related kernel is a density function on R N . We exhibit uniform and L p , p ≥ 1 approximations via Jackson-type inequalities involving the first L p modulus of smoothness, 1 ≤ p ≤ ∞ . The differentiability of our multivariate functions is covered extensively in our approximations. We continue by detailing the global smoothness preservation results of our operators. We conclude the paper with the simultaneous approximation and the simultaneous global smoothness preservation by our multivariate perturbed activated singular integrals. [ABSTRACT FROM AUTHOR]

The physics-informed neural network (PINN) is an effective alternative method for solving differential equations that do not require grid partitioning, making it easy to implement. In this study, using automatic differentiation techniques, the PINN method is employed to solve differential equations by embedding prior physical information, such as boundary and initial conditions, into the loss function. The differential equation solution is obtained by minimizing the loss function. The PINN method is trained using the Adam algorithm, taking the differential equations of motion in structural dynamics as an example. The time sample set generated by the Sobol sequence is used as the input, while the displacement is considered the output. The initial conditions are incorporated into the loss function as penalty terms using automatic differentiation techniques. The effectiveness of the proposed method is validated through the numerical analysis of a two-degree-of-freedom system, a four-story frame structure, and a cantilever beam. The study also explores the impact of the input samples, the activation functions, the weight coefficients of the loss function, and the width and depth of the neural network on the PINN predictions. The results demonstrate that the PINN method effectively solves the differential equations of motion of damped systems. It is a general approach for solving differential equations of motion. [ABSTRACT FROM AUTHOR]

A novel morphing activation function is proposed, motivated by the wavelet theory and the use of wavelets as activation functions. Morphing refers to the gradual change of shape to mimic several apparently unrelated activation functions. The shape is controlled by the fractional order derivative, which is a trainable parameter to be optimized in the neural network learning process. Given the morphing activation function, and taking only integer-order derivatives, efficient piecewise polynomial versions of several existing activation functions are obtained. Experiments show that the performance of polynomial versions PolySigmoid, PolySoftplus, PolyGeLU, PolySwish, and PolyMish is similar or better than their counterparts Sigmoid, Softplus, GeLU, Swish, and Mish. Furthermore, it is possible to learn the best shape from the data by optimizing the fractional-order derivative with gradient descent algorithms, leading to the study of a more general formula based on fractional calculus to build and adapt activation functions with properties useful in machine learning. [ABSTRACT FROM AUTHOR]

End-to-end disparity estimation algorithms based on cost volume deployed in edge-end neural network accelerators have the problem of structural adaptation and need to ensure accuracy under the condition of adaptation operator. Therefore, this paper proposes a novel disparity calculation algorithm that uses low-rank approximation to approximately replace 3D convolution and transposed 3D convolution, WReLU to reduce data compression caused by the activation function, and unimodal cost volume filtering and a confidence estimation network to regularize cost volume. It alleviates the problem of disparity-matching cost distribution being far away from the true distribution and greatly reduces the computational complexity and number of parameters of the algorithm while improving accuracy. Experimental results show that compared with a typical disparity estimation network, the absolute error of the proposed algorithm is reduced by 38.3%, the three-pixel error is reduced to 1.41%, and the number of parameters is reduced by 67.3%. The calculation accuracy is better than that of other algorithms, it is easier to deploy, and it has strong structural adaptability and better practicability. [ABSTRACT FROM AUTHOR]

The problem of inverting dynamic complex matrices remains a central and intricate challenge that has garnered significant attention in scientific and mathematical research. The zeroing neural network (ZNN) has been a notable approach, utilizing time derivatives for real-time solutions in noiseless settings. However, real-world disturbances pose a significant challenge to a ZNN's convergence. We design an accelerated dual-integral structure zeroing neural network (ADISZNN), which can enhance convergence and restrict linear noise, particularly in complex domains. Based on the Lyapunov principle, theoretical analysis proves the convergence and robustness of ADISZNN. We have selectively integrated the SBPAF activation function, and through theoretical dissection and comparative experimental validation we have affirmed the efficacy and accuracy of our activation function selection strategy. After conducting numerous experiments, we discovered oscillations and improved the model accordingly, resulting in the ADISZNN-Stable model. This advanced model surpasses current models in both linear noisy and noise-free environments, delivering a more rapid and stable convergence, marking a significant leap forward in the field. [ABSTRACT FROM AUTHOR]

We propose a neural-network-based watermarking method that introduces the quantized activation function that approximates the quantization of JPEG compression. Many neural-network-based watermarking methods have been proposed. Conventional methods have acquired robustness against various attacks by introducing an attack simulation layer between the embedding network and the extraction network. The quantization process of JPEG compression is replaced by the noise addition process in the attack layer of conventional methods. In this paper, we propose a quantized activation function that can simulate the JPEG quantization standard as it is in order to improve the robustness against the JPEG compression. Our quantized activation function consists of several hyperbolic tangent functions and is applied as an activation function for neural networks. Our network was introduced in the attack layer of ReDMark proposed by Ahmadi et al. to compare it with their method. That is, the embedding and extraction networks had the same structure. We compared the usual JPEG compressed images and the images applying the quantized activation function. The results showed that a network with quantized activation functions can approximate JPEG compression with high accuracy. We also compared the bit error rate (BER) of estimated watermarks generated by our network with those generated by ReDMark. We found that our network was able to produce estimated watermarks with lower BERs than those of ReDMark. Therefore, our network outperformed the conventional method with respect to image quality and BER. [ABSTRACT FROM AUTHOR]

This article addresses the semantic segmentation of laparoscopic surgery images, placing special emphasis on the segmentation of structures with a smaller number of observations. As a result of this study, adjustment parameters are proposed for deep neural network architectures, enabling a robust segmentation of all structures in the surgical scene. The U-Net architecture with five encoder–decoders (U-Net5ed), SegNet-VGG19, and DeepLabv3+ employing different backbones are implemented. Three main experiments are conducted, working with Rectified Linear Unit (ReLU), Gaussian Error Linear Unit (GELU), and Swish activation functions. The applied loss functions include Cross Entropy (CE), Focal Loss (FL), Tversky Loss (TL), Dice Loss (DiL), Cross Entropy Dice Loss (CEDL), and Cross Entropy Tversky Loss (CETL). The performance of Stochastic Gradient Descent with momentum (SGDM) and Adaptive Moment Estimation (Adam) optimizers is compared. It is qualitatively and quantitatively confirmed that DeepLabv3+ and U-Net5ed architectures yield the best results. The DeepLabv3+ architecture with the ResNet-50 backbone, Swish activation function, and CETL loss function reports a Mean Accuracy (MAcc) of 0.976 and Mean Intersection over Union (MIoU) of 0.977. The semantic segmentation of structures with a smaller number of observations, such as the hepatic vein, cystic duct, Liver Ligament, and blood, verifies that the obtained results are very competitive and promising compared to the consulted literature. The proposed selected parameters were validated in the YOLOv9 architecture, which showed an improvement in semantic segmentation compared to the results obtained with the original architecture. [ABSTRACT FROM AUTHOR]

Precise and rapid detection of seatbelts is an essential research field for intelligent traffic management. In order to improve the detection precision of seatbelts and speed up algorithm inference velocity, a lightweight seatbelt detection algorithm is proposed. Firstly, by adding the G-ELAN module designed in this paper to the YOLOv7-tiny network, the optimization of construction and reduction of parameters are accomplished, and the ResNet is compressed with the channel pruning approach to decrease computational overheads. Then, the Mish activation function is utilized to replace the Leaky Relu in the neck to enhance the non-linear competence of the network. Finally, the triplet attention module is integrated into the model after pruning to make up for the underlying performance reduction caused by the previous stage and upgrade overall detection precision. The experimental results based on the self-built seatbelt dataset showed that, compared to the initial network, the Mean Average Precision (mAP) achieved by the proposed GM-YOLOv7 was improved by 3.8%, while the volume and the computation amount were lowered by 20% and 24.6%, respectively. Compared with YOLOv3, YOLOX, and YOLOv5, the mAP of GM-YOLOv7 increased by 22.4%, 4.6%, and 4.2%, respectively, and the number of computational operations decreased by 25%, 63%, and 38%, respectively. In addition, the accuracy of the improved RST-Net increased to 98.25%, while the parameter value was reduced by 48% compared to the basic model, effectively improving the detection performance and realizing a lightweight structure. [ABSTRACT FROM AUTHOR]

Semantic segmentation of remote sensing (RS) images is a pivotal branch in the realm of RS image processing, which plays a significant role in urban planning, building extraction, vegetation extraction, etc. With the continuous advancement of remote sensing technology, the spatial resolution of remote sensing images is progressively improving. This escalation in resolution gives rise to challenges like imbalanced class distributions among ground objects in RS images, the significant variations of ground object scales, as well as the presence of redundant information and noise interference. In this paper, we propose a multi-scale context extraction network, ASPP+-LANet, based on the LANet for semantic segmentation of high-resolution RS images. Firstly, we design an ASPP+ module, expanding upon the ASPP module by incorporating an additional feature extraction channel, redesigning the dilation rates, and introducing the Coordinate Attention (CA) mechanism so that it can effectively improve the segmentation performance of ground object targets at different scales. Secondly, we introduce the Funnel ReLU (FReLU) activation function for enhancing the segmentation effect of slender ground object targets and refining the segmentation edges. The experimental results show that our network model demonstrates superior segmentation performance on both Potsdam and Vaihingen datasets, outperforming other state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) have surfaced as a transformative approach in the domain of cryptography, introducing a novel paradigm where two neural networks, the generator (akin to Alice) and the discriminator (akin to Bob), are pitted against each other in a cryptographic setting. A third network, representing Eve, attempts to decipher the encrypted information. The efficacy of this encryption–decryption process is deeply intertwined with the choice of activation functions employed within these networks. This study conducted a comparative analysis of four widely used activation functions within a standardized GAN framework. Our recent explorations underscore the superior performance achieved when utilizing the Rectified Linear Unit (ReLU) in the hidden layers combined with the Sigmoid activation function in the output layer. The non-linear nature introduced by the ReLU provides a sophisticated encryption pattern, rendering the deciphering process for Eve intricate. Simultaneously, the Sigmoid function in the output layer guarantees that the encrypted and decrypted messages are confined within a consistent range, facilitating a straightforward comparison with original messages. The amalgamation of these activation functions not only bolsters the encryption strength but also ensures the fidelity of the decrypted messages. These findings not only shed light on the optimal design considerations for GAN-based cryptographic systems but also underscore the potential of investigating hybrid activation functions for enhanced system optimization. In our exploration of cryptographic strength and training efficiency using various activation functions, we discovered that the "ReLU and Sigmoid" combination significantly outperforms the others, demonstrating superior security and a markedly efficient mean training time of 16.51 s per 2000 steps. This highlights the enduring effectiveness of established methodologies in cryptographic applications. This paper elucidates the implications of these choices, advocating for their adoption in GAN-based cryptographic models, given the superior results they yield in ensuring security and accuracy. [ABSTRACT FROM AUTHOR]

The article presents the results of research into a method for representing complex data based on an overcomplete basis and l0/l1 norms. The proposed method is an extended modification of the neural-like MFNN (minimum fuel neural network) for the case of complex data. The influence of the choice of activation function on the performance of the method is analyzed. The results of the numerical simulation demonstrate the effectiveness of the proposed method for the case of sparse representation of complex data and can be used to determine the direction of arrival (DOA) for a uniform linear array (ULA). [ABSTRACT FROM AUTHOR]

Pavement defect detection technology stands as a pivotal component within intelligent driving systems, demanding heightened precision and rapid detection rates. Addressing the complexities arising from diverse defect types and intricate backgrounds in visual sensing, this study introduces an enhanced approach to augment the network structure and activation function within the foundational YOLOv5 algorithm. Initially, modifications to the YOLOv5′s architecture incorporate an adjustment to the Leaky ReLU activation function, thereby enhancing regression stability and accuracy. Subsequently, the integration of bi-level routing attention into the network's head layer optimizes the attention mechanism, notably improving overall efficiency. Additionally, the replacement of the YOLOv5 backbone layer's C3 module with the C3-TST module enhances initial convergence efficiency in target detection. Comparative analysis against the original YOLOv5s network reveals a 2% enhancement in map50 and a 1.8% improvement in F1, signifying an overall advancement in network performance. The initial convergence rate of the algorithm has been improved, and the accuracy and operational efficiency have also been greatly improved, especially on models with small-scale training sets. [ABSTRACT FROM AUTHOR]

In this review, we compiled convolutional neural network (CNN) methods which have the potential to automate the manual, costly and error-prone processing of medical images. We attempted to provide a thorough survey of improved architectures, popular frameworks, activation functions, ensemble techniques, hyperparameter optimizations, performance metrics, relevant datasets and data preprocessing strategies that can be used to design robust CNN models. We also used machine learning algorithms for the statistical modeling of the current literature to uncover latent topics, method gaps, prevalent themes and potential future advancements. The statistical modeling results indicate a temporal shift in favor of improved CNN designs, such as a shift from the use of a CNN architecture to a CNN-transformer hybrid. The insights from statistical modeling point that the surge of CNN practitioners into the medical imaging field, partly driven by the COVID-19 challenge, catalyzed the use of CNN methods for detecting and diagnosing pathological conditions. This phenomenon likely contributed to the sharp increase in the number of publications on the use of CNNs for medical imaging, both during and after the pandemic. Overall, the existing literature has certain gaps in scope with respect to the design and optimization of CNN architectures and methods specifically for medical imaging. Additionally, there is a lack of post hoc explainability of CNN models and slow progress in adopting CNNs for low-resource medical imaging. This review ends with a list of open research questions that have been identified through statistical modeling and recommendations that can potentially help set up more robust, improved and reproducible CNN experiments for medical imaging. [ABSTRACT FROM AUTHOR]

In this paper, our main aim is to present innovative fixed-point theorems that provide solutions to the fixed-circle problem with symmetric contractions. We accomplish this by employing operator enrichment techniques within the context of Banach spaces. Furthermore, we demonstrate the practical application of these theorems by showcasing their relevance to the rectified linear unit (ReLU) activation function. By exploring the connection between fixed points and activation functions, our work contributes to a deeper understanding of the behavior and properties of these fundamental mathematical concepts. [ABSTRACT FROM AUTHOR]

Groundnut, being a widely consumed oily seed with significant health benefits and appealing sensory profiles, is extensively cultivated in tropical regions worldwide. However, the yield is substantially impacted by the changing climate. Therefore, predicting stressed groundnut yield based on climatic factors is desirable. This research focuses on predicting groundnut yield based on several combinations of climatic factors using artificial neural networks and three training algorithms. The Levenberg–Marquardt, Bayesian Regularization, and Scaled Conjugate Gradient algorithms were evaluated for their performance using climatic factors such as minimum temperature, maximum temperature, and rainfall in different regions of Sri Lanka, considering the seasonal variations in groundnut yield. A three-layer neural network was employed, comprising a hidden layer. The hidden layer consisted of 10 neurons, and the log sigmoid functions were used as the activation function. The performance of these configurations was evaluated based on the mean squared error and Pearson correlation. Notable improvements were observed when using the Levenberg–Marquardt algorithm as the training algorithm and applying the natural logarithm transformation to the yield values. These improvements were evident through the higher Pearson correlation values for training (0.84), validation (1.00) and testing (1.00), and a lower mean squared error (2.2859 × 10−21) value. Due to the limited data, K-Fold cross-validation was utilized for optimization, with a K value of 5 utilized for the process. The application of the natural logarithm transformation to the yield values resulted in a lower mean squared error (0.3724) value. The results revealed that the Levenberg–Marquardt training algorithm performs better in capturing the relationships between the climatic factors and groundnut yield. This research provides valuable insights into the utilization of climatic factors for predicting groundnut yield, highlighting the effectiveness of the training algorithms and emphasizing the importance of carefully selecting and expanding the climatic factors in the modeling equation. [ABSTRACT FROM AUTHOR]

Medical image segmentation primarily utilizes a hybrid model consisting of a Convolutional Neural Network and sequential Transformers. The latter leverage multi-head self-attention mechanisms to achieve comprehensive global context modelling. However, despite their success in semantic segmentation, the feature extraction process is inefficient and demands more computational resources, which hinders the network's robustness. To address this issue, this study presents two innovative methods: PTransUNet (PT model) and C-PTransUNet (C-PT model). The C-PT module refines the Vision Transformer by substituting a sequential design with a parallel one. This boosts the feature extraction capabilities of Multi-Head Self-Attention via self-correlated feature attention and channel feature interaction, while also streamlining the Feed-Forward Network to lower computational demands. On the Synapse public dataset, the PT and C-PT models demonstrate improvements in DSC accuracy by 0.87% and 3.25%, respectively, in comparison with the baseline model. As for the parameter count and FLOPs, the PT model aligns with the baseline model. In contrast, the C-PT model shows a decrease in parameter count by 29% and FLOPs by 21.4% relative to the baseline model. The proposed segmentation models in this study exhibit benefits in both accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) possess remarkable fitting capabilities and play a crucial role in the field of computer vision. Super-resolution restoration is the process of converting low-resolution images into high-resolution ones, providing more detail and information. This is of paramount importance for monitoring and managing forest resources, enabling the surveillance of vegetation, wildlife, and potential disruptive factors in forest ecosystems. In this study, we propose an image super-resolution model based on Generative Adversarial Networks. We incorporate Multi-Scale Residual Blocks (MSRB) as the core feature extraction component to obtain image features at different scales, enhancing feature extraction capabilities. We introduce a novel attention mechanism, GAM Attention, which is added to the VGG network to capture more accurate feature dependencies in both spatial and channel domains. We also employ the adaptive activation function Meta ACONC and Ghost convolution to optimize training efficiency and reduce network parameters. Our model is trained on the DIV2K and LOVEDA datasets, and experimental results indicate improvements in evaluation metrics compared to SRGAN, with a PSNR increase of 0.709/2.213 dB, SSIM increase of 0.032/0.142, and LPIPS reduction of 0.03/0.013. The model performs on par with Real-ESRGAN but offers significantly improved speed. Our model efficiently restores single-frame remote sensing images of forests while achieving results comparable to state-of-the-art methods. It overcomes issues related to image distortion and texture details, producing forest remote sensing images that closely resemble high-resolution real images and align more closely with human perception. This research has significant implications on a global scale for ecological conservation, resource management, climate change research, risk management, and decision-making processes. [ABSTRACT FROM AUTHOR]

Software is widely used in various fields. There is no place where it is not used from the smallest part to the entire part. In particular, the tendency to rely on software is accelerating as the fields of artificial intelligence and big data become more widespread. Therefore, it is extremely important to evaluate the reliability of software because of the extensive damage that could occur if the software fails. Previously, software reliability models were developed based on mathematical and statistical grounds; however, immediate response was difficult. Therefore, in this study, a software reliability model was developed that depends on data using deep learning, and it was analyzed by replacing the activation function previously used in deep learning with the proposed software reliability model. Since the sigmoid function has a similar shape to the software reliability model, we utilized this to propose a deep learning software reliability model that replaces the activation function, the sigmoid function, with the software reliability function. Two datasets were compared and analyzed using 10 criteria, and the superiority of the proposed deep-learning software reliability model was proved. In addition, the results were compared by changing the parameters utilized in the proposed deep-learning software reliability model by −10%, −5%, 5%, and 10%, and it was found that the larger the parameters, the smaller the change. [ABSTRACT FROM AUTHOR]

This work studies the application of low-cost noise reduction algorithms for the data processing of distributed acoustic sensors (DAS). It presents an improvement of the previously described methodology using the activation function of neurons, which enhances the speed of data processing and the quality of event identification, as well as reducing spatial distortions. The possibility of using a cheaper radiation source in DAS setups is demonstrated. Optimal algorithms' combinations are proposed for different types of the events recorded. The criterion for evaluating the effectiveness of algorithm performance was an increase in the signal-to-noise ratio (SNR). The finest effect achieved with a combination of algorithms provided an increase in SNR of 10.8 dB. The obtained results can significantly expand the application scope of DAS. [ABSTRACT FROM AUTHOR]

Small target detection has been widely used in applications that are relevant to everyday life and have many real-time requirements, such as road patrols and security surveillance. Although object detection methods based on deep learning have achieved great success in recent years, they are not effective in small target detection. In order to solve the problem of low recognition rate caused by factors such as low resolution of UAV viewpoint images and little valid information, this paper proposes an improved algorithm based on the YOLOv5s model, called YOLOv5s-pp. First, to better suppress interference from complex backgrounds and negative samples in images, we add a CA attention module, which can better focus on task-specific important channels while weakening the influence of irrelevant channels. Secondly, we improve the forward propagation and generalisation of the network using the Meta-ACON activation function, which adaptively learns to adjust the degree of linearity or nonlinearity of the activation function based on the input data. Again, the SPD Conv module is incorporated into the network model to address the problems of reduced learning efficiency and loss of fine-grained information due to cross-layer convolution in the model. Finally, the detection head is improved by using smaller, smaller-target detection heads to reduce missed detections. We evaluated the algorithm on the VisDrone2019-DET and UAVDT datasets and compared it with other state-of-the-art algorithms. Compared to YOLOv5s, mAP@.5 improved by 7.4% and 6.5% on the VisDrone2019-DET and UAVDT datasets, respectively, and compared to YOLOv8s, mAP@.5 improved by 0.8% and 2.1%, respectively. For improving the performance of the UAV-side small target detection algorithm, it will help to enhance the reliability and safety of UAVs in critical missions such as military reconnaissance, road patrol and security surveillance. [ABSTRACT FROM AUTHOR]

Aiming at the environment of low illumination, high dust, and heavy water fog in coal mine driving face and the problems of occlusion, coincidence, and irregularity of bolt mesh laid on coal wall, a YOLOv7 bolt mesh-detection algorithm combining the image enhancement and convolutional block attention module is proposed. First, the image brightness is enhanced by a hyperbolic mapping transform-based image enhancement algorithm, and the image is defogged by a dark channel-based image defogging algorithm. Second, by introducing a convolutional block attention model in the YOLOv7 detection network, the significance of bolt mesh targets in the image is improved, and its feature expression ability in the detection network is enhanced. Meanwhile, the original activation function ReLU in the convolutional layer Conv of the YOLOv7 network is replaced by LeakyReLU so that the activation function has stronger nonlinear expression capability, which enhances the feature extraction performance of the network and thus improves the detection accuracy. Finally, the training and testing samples were prepared using the actual video of the drilling and bolting operation, and the proposed algorithm is compared with five classical target detection algorithms. The experimental results show that the proposed algorithm can be better applied to the low illumination, high dust environment, and irregular shape on the detection accuracy of coal mine roadway bolt mesh, and the average detection accuracy of the image can reach 95.4% with an average detection time of 0.0392 s. [ABSTRACT FROM AUTHOR]

As the banana industry develops, the demand for intelligent banana crown cutting is increasing. To achieve efficient crown cutting of bananas, accurate segmentation of the banana crown is crucial for the operation of a banana crown cutting device. In order to address the existing challenges, this paper proposed a method for segmentation of banana crown based on improved DeepLabv3+. This method replaces the backbone network of the classical DeepLabv3+ model with MobilenetV2, reducing the number of parameters and training time, thereby achieving model lightweightness and enhancing model speed. Additionally, the Atrous Spatial Pyramid Pooling (ASPP) module is enhanced by incorporating the Shuffle Attention Mechanism and replacing the activation function with Meta-ACONC. This enhancement results in the creation of a new feature extraction module, called Banana-ASPP, which effectively handles high-level features. Furthermore, Multi-scale Channel Attention Module (MS-CAM) is introduced to the Decoder to improve the integration of features from multiple semantics and scales. According to experimental data, the proposed method has a Mean Intersection over Union (MIoU) of 85.75%, a Mean Pixel Accuracy (MPA) of 91.41%, parameters of 5.881 M and model speed of 61.05 f/s. Compared to the classical DeepLabv3+ network, the proposed model exhibits an improvement of 1.94% in MIoU and 1.21% in MPA, while reducing the number of parameters by 89.25% and increasing the model speed by 47.07 f/s. The proposed method enhanced banana crown segmentation accuracy while maintaining model lightweightness and speed. It also provided robust technical support for relevant parameters calculation of banana crown and control of banana crown cutting equipment. [ABSTRACT FROM AUTHOR]

In order to solve the problem of the small degree of variability between the features of ginseng grading classes and the resulting need for heavy reliance on professionals, this study established a ginseng dataset containing 5116 images with three classes in different contexts and proposed a ginseng-grading model based on an improved ConvNeXt framework. Firstly, a Channel Shuffle module was embedded in the backbone network after down-sampling to fully fuse the channel features and improve the model's grading accuracy. The model's characterization ability enriched the feature space of the convolutional block and further improved the model's accuracy. Finally, the original activation function, GELU, was replaced with the PreLU activation function to increase the nonlinear variability of the neural network model and improve the model's accuracy and efficiency. The experimental results show that the method demonstrated accuracy improvements of 2.46% and 4.32%, respectively, compared with the current advanced networks, Vision Transformer and Swim Transformer. Furthermore, the accuracy, precision, recall, and specificity of ginseng classification reached values of 94.44%, 91.58%, 91.04%, and 95.82%, respectively, and the loss rate was reduced to 0.24. A comparison with expert appraisal results showed high consistency, thus verifying our model's accuracy and reliability in ginseng quality assessment and its ability to provide technical support for intelligent ginseng quality grading. [ABSTRACT FROM AUTHOR]

With the flourishing development of wireless communication, further challenges will be introduced by the future demands of emerging applications. However, in the face of more complex communication scenarios, favorable decoding results may not be yielded by conventional channel decoding schemes based on mathematical models. The remarkable contributions of deep neural networks (DNNs) in various fields have garnered widespread recognition, which has ignited our enthusiasm for their application in wireless communication systems. Therefore, a reliable DNN-based decoding scheme designed for wireless communication systems is proposed. This scheme comprises efficient local decoding using linear and nonlinear operations. To be specific, linear operations are carried out on the edges connecting neurons, while nonlinear operations are performed on each neuron. After forward propagation through the DNN, the loss value is estimated based on the output, and backward propagation is employed to update the weights and biases. This process is performed iteratively until a near-optimal message sequence is recovered. Various factors within the DNN are considered in the simulation and the potential impacts of each factor are analyzed. Simulation results indicate that our proposed DNN-based decoding scheme is superior to the conventional hard decision. [ABSTRACT FROM AUTHOR]

This paper introduces a new spike activation function (SPKAF) or spike membership function for fuzzy adaptive neurons (FAN), developed for decoding spatiotemporal information with spikes, optimizing digital signal processing. A solution with the adaptive network-based fuzzy inference system (ANFIS) method is proposed and compared with that of the FAN-SPKAF model, obtaining very precise simulation results. Stability analysis of systems models is presented. An application to voice recognition using solfeggio syllables in Spanish is performed experimentally, comparing the methods of FAN-step activation function (STEPAF)-SPKAF, Augmented Spiking Neuron Model, and Augmented FAN-STEPAF-SPKAF, achieving very good results. [ABSTRACT FROM AUTHOR]

Detecting and classifying vehicles as objects from images and videos is challenging in appearance-based representation, yet plays a significant role in the substantial real-time applications of Intelligent Transportation Systems (ITSs). The rapid development of Deep Learning (DL) has resulted in the computer-vision community demanding efficient, robust, and outstanding services to be built in various fields. This paper covers a wide range of vehicle detection and classification approaches and the application of these in estimating traffic density, real-time targets, toll management and other areas using DL architectures. Moreover, the paper also presents a detailed analysis of DL techniques, benchmark datasets, and preliminaries. A survey of some vital detection and classification applications, namely, vehicle detection and classification and performance, is conducted, with a detailed investigation of the challenges faced. The paper also addresses the promising technological advancements of the last few years. [ABSTRACT FROM AUTHOR]

The burning of straw is a very destructive process that threatens people's livelihoods and property and causes irreparable environmental damage. It is therefore essential to detect and control the burning of straw. In this study, we analyzed Sentinel-2 data to select the best separation bands based on the response characteristics of clouds, smoke, water bodies, and background (vegetation and bare soil) to the different bands. The selected bands were added to the red, green, and blue bands (RGB) as training sample data. The band that featured the highest detection accuracy, RGB_Band6, was finally selected, having an accuracy of 82.90%. The existing object detection model cannot directly handle multi-band images. This study modified the input layer structure based on the YOLOv5s model to build an object detection network suitable for multi-band remote sensing images. The Squeeze-and-Excitation (SE) network attention mechanism was introduced based on the YOLOv5s model so that the delicate features of smoke were enhanced, and the Convolution + Batch normalization + Leaky ReLU (CBL) module was replaced with the Convolution + Batch normalization + Mish (CBM) module. The accuracy of the model was improved to 75.63%, which was 1.81% better than before. We also discussed the effect of spatial resolution on model detection and where accuracies of 84.18%, 73.13%, and 45.05% for images of 60-, 20-, and 10-m resolution, respectively, were realized. The experimental results demonstrated that the accuracy of the model only sometimes improved with increasing spatial resolution. This study provides a technical reference for the monitoring of straw burning, which is vital for both the control of straw burning and ways to improve ambient air quality. [ABSTRACT FROM AUTHOR]

Among various network compression methods, network quantization has developed rapidly due to its superior compression performance. However, trivial activation quantization schemes limit the compression performance of network quantization. Most conventional activation quantization methods directly utilize the rectified activation functions to quantize models, yet their unbounded outputs generally yield drastic accuracy degradation. To tackle this problem, we propose a comprehensive activation quantization technique namely Bilateral Clipping Parametric Rectified Linear Unit (BCPReLU) as a generalized version of all rectified activation functions, which limits the quantization range more flexibly during training. Specifically, trainable slopes and thresholds are introduced for both positive and negative inputs to find more flexible quantization scales. We theoretically demonstrate that BCPReLU has approximately the same expressive power as the corresponding unbounded version and establish its convergence in low-bit quantization networks. Extensive experiments on a variety of datasets and network architectures demonstrate the effectiveness of our trainable clipping activation function. [ABSTRACT FROM AUTHOR]

To address the problems of poor recognition effect, low detection accuracy, many model parameters and computation, complex network structure, and unfavorable portability to embedded devices in traditional tennis ball detection algorithms, this study proposes a lightweight tennis ball detection algorithm, YOLOv5s-Z, based on the YOLOv5s algorithm and Robomater EP. The main work is as follows: firstly, the lightweight network G-Backbone and G-Neck network layers are constructed to reduce the number of parameters and computation of the network structure. Secondly, convolutional coordinate attention is incorporated into the G-Backbone to embed location information into channel attention, which enables the network to obtain location information of a larger area through multiple convolutions and enhances the expression ability of mobile network learning features. In addition, the Concat module in the original feature fusion is modified into a weighted bi-directional feature pyramid W-BiFPN with settable learning weights to improve the feature fusion capability and achieve efficient weighted feature fusion and bi-directional cross-scale connectivity. Finally, the Loss function EIOU Loss is introduced to split the influence factor of the aspect ratio and calculate the length and width of the target frame and anchor frame, respectively, combined with Focal-EIOU Loss to solve the problem of imbalance between complex and easy samples. Meta-ACON's activation function is introduced to achieve an adaptive selection of whether to activate neurons and improve the detection accuracy. The experimental results show that compared with the YOLOv5s algorithm, the YOLOv5s-Z algorithm reduces the number of parameters and computation by 42% and 44%, respectively, reduces the model size by 39%, and improves the mean accuracy by 2%, verifying the effectiveness of the improved algorithm and the lightweight of the model, adapting to Robomaster EP, and meeting the deployment requirements of embedded devices for the detection and identification of tennis balls. [ABSTRACT FROM AUTHOR]

Due to the tremendous volume taken by the 3D point-cloud models, knowing how to achieve the balance between a high compression ratio, a low distortion rate, and computing cost in point-cloud compression is a significant issue in the field of virtual reality (VR). Convolutional neural networks have been used in numerous point-cloud compression research approaches during the past few years in an effort to progress the research state. In this work, we have evaluated the effects of different network parameters, including neural network depth, stride, and activation function on point-cloud compression, resulting in an optimized convolutional neural network for compression. We first have analyzed earlier research on point-cloud compression based on convolutional neural networks before designing our own convolutional neural network. Then, we have modified our model parameters using the experimental data to further enhance the effect of point-cloud compression. Based on the experimental results, we have found that the neural network with the 4 layers and 2 strides parameter configuration using the Sigmoid activation function outperforms the default configuration by 208% in terms of the compression-distortion rate. The experimental results show that our findings are effective and universal and make a great contribution to the research of point-cloud compression using convolutional neural networks. [ABSTRACT FROM AUTHOR]

A deep learning approach for the efficient electromagnetic analysis of an on-chip inductor with dummy metal fillings (DMFs) is proposed. By comparing different activation functions and loss functions, a deep neural network for DMF modeling is built using a smooth maximum unit activation function and log-cosh loss function. The parasitic capacitive effect of DMFs is quickly and accurately extracted though this model, and the effective permittivity can be obtained subsequently. An on-chip inductor containing DMFs with different filling densities is analyzed using this proposed method and compared with the electromagnetic simulation of entire structures. The results validate the accuracy and efficiency of this proposed method. [ABSTRACT FROM AUTHOR]

In the academic world, ginseng (Panax ginseng C. A. Meyer) has received much attention as the most representative element of Chinese medicine. To address the lack of traditional algorithms in the identification of ginseng appearance quality and further improve the manual identification on ginseng, we propose a grading method of ginseng appearance quality based on deep learning, taking advantage of the benefits of deep learning in the image identification. Firstly, we substituted LeakyReLU for the conventional activation function ReLU to enhance the predictive power of the model. Secondly, we added an ECA module to the residual block, which allowed attention to be focused on the input object to capture more precise and detailed features. Thirdly, we used the focal loss function to solve the problem of an imbalanced dataset. Then, the self-constructed dataset was processed with data enhancement and divided into four different classes of ginseng. The dataset was trained on a model with transfer learning to finally obtain the best model applicable to the identification of ginseng appearance quality. The experiments showed that, compared with the classical convolutional neural network models VGG16, GoogLeNet, ResNet50 and Densenet121, the proposed model reported the best performance, its accuracy in the test set was as high as 97.39%, and the loss value was 0.035. This method can efficiently classify the appearance quality of ginseng, and has a significant value in the field of ginseng appearance quality identification. [ABSTRACT FROM AUTHOR]

This research aims to study and assess state-of-the-art physics-informed neural networks (PINNs) from different researchers' perspectives. The PRISMA framework was used for a systematic literature review, and 120 research articles from the computational sciences and engineering domain were specifically classified through a well-defined keyword search in Scopus and Web of Science databases. Through bibliometric analyses, we have identified journal sources with the most publications, authors with high citations, and countries with many publications on PINNs. Some newly improved techniques developed to enhance PINN performance and reduce high training costs and slowness, among other limitations, have been highlighted. Different approaches have been introduced to overcome the limitations of PINNs. In this review, we categorized the newly proposed PINN methods into Extended PINNs, Hybrid PINNs, and Minimized Loss techniques. Various potential future research directions are outlined based on the limitations of the proposed solutions. [ABSTRACT FROM AUTHOR]

While non-linear activation functions play vital roles in artificial neural networks, it is generally unclear how the non-linearity can improve the quality of function approximations. In this paper, we present a theoretical framework to rigorously analyze the performance gain of using non-linear activation functions for a class of residual neural networks (ResNets). In particular, we show that when the input features for the ResNet are uniformly chosen and orthogonal to each other, using non-linear activation functions to generate the ResNet output averagely outperforms using linear activation functions, and the performance gain can be explicitly computed. Moreover, we show that when the activation functions are chosen as polynomials with the degree much less than the dimension of the input features, the optimal activation functions can be precisely expressed in the form of Hermite polynomials. This demonstrates the role of Hermite polynomials in function approximations of ResNets. [ABSTRACT FROM AUTHOR]

The performance of the activation function in convolutional neural networks is directly related to the model's image classification accuracy. The rectified linear unit (ReLU) activation function has been extensively used in image classification models but has significant shortcomings, including low classification accuracy. The performance of a series of parametric activation functions has made parameter addition a popular research avenue for improving the performance of activation functions in recent years, and excellent progress has been achieved. Existing parametric activation functions often focus on assigning a different slope to the negative part of the activation function and still involve the negative value alone in the activation function calculation, without considering the impact of linking the negative value to the positive value on the performance of the activation function. As a result, this work proposes a novel parametric right-shift activation function, the adaptive offset activation function (AOAF). By inserting an adaptive parameter (the mean value of the input feature tensor) and two custom ReLU parameters, the negative parameters previously driven to zero by ReLU can be turned into positive parameters with lower weight and participate in CNN feature extraction. We compared the performance of the suggested activation function to the performance of a selection of typical activation functions using four distinct public datasets. Compared with ReLU, the average classification accuracy of our proposed activation function improved by 3.82%, 0.6%, 1.02%, and 4.8% for the four datasets, respectively. [ABSTRACT FROM AUTHOR]

The Sunway supercomputers have recently attracted considerable attention to execute neural networks. Meanwhile, activation functions help extend the applicability of neural networks to nonlinear models by introducing nonlinear factors. Despite the numerous activation function-supported AI frameworks, only PyTorch and TensorFlow were ported to the Sunway platforms. Although these libraries can meet the minimum functional requirements to deploy a neural network on the Sunway machines, there still exist some drawbacks including the limited number of usable functions and unsatisfactory performances remaining unresolved. Therefore, two activation function algorithms with different computing accuracies were developed in this study, and an efficient implementation scheme was designed using the single instruction/multiple data extension and multiply–add instructions of the platform. Finally, an efficient library-swAFL-composed of 48 function interfaces was designed and implemented on the Sunway platforms. Experimental results indicate that swAFL outperformed PyTorch and TensorFlow by 19.5 and 23 times, respectively, on average. [ABSTRACT FROM AUTHOR]

This paper aims at finding a fixed-time solution to the Sylvester equation by using a gradient neural network (GNN). To reach this goal, a modified sign-bi-power (msbp) function is presented and applied on a linear GNN as an activation function. Accordingly, a fixed-time convergent GNN (FTC-GNN) model is developed for solving the Sylvester equation. The upper bound of the convergence time of such an FTC-GNN model can be predetermined if parameters are given regardless of the initial conditions. This point is corroborated by a detailed theoretical analysis. In addition, the convergence time is also estimated utilizing the Lyapunov stability theory. Two examples are then simulated to demonstrate the validation of the theoretical analysis, as well as the superior convergence performance of the presented FTC-GNN model as compared to the existing GNN models. [ABSTRACT FROM AUTHOR]

In this paper, a combined approach of Principal Component Analysis (PCA)-based Extreme Learning Machine (ELM) for boiler output forecasting in a thermal power plant is presented. The input used for this prediction model is taken from the boiler unit of the Yermarus Thermal Power Station (YTPS), India. Calculation of the accurate electrical output of a boiler in an operating system requires the knowledge of hundreds of operating parameters. The dimensionality of the input dataset is reduced by applying principal component analysis using IBM@SPSS Software. In the process of principal component analysis, a dataset of 232 parameters is standardized into 16 principal components. The total dataset collected is divided into training and testing datasets. The extreme learning machine is designed for various activation functions and the number of neurons. Sigmoid and hyperbolic tangent activation functions are studied here. Its generalization performance is examined in terms of the Mean Square Error (MSE), Mean Absolute Error (MAE), Root Mean Square (RMSE), and Mean Absolute Percentage Error (MAPE). ELM and PCA–ELM are compared. In both the ELM and PCA–ELM models, when the extreme learning machine was designed with a sigmoid activation function with 100 nodes in the hidden layer, RMSE was 5.026 and 4.730, respectively. Therefore, the developed combined approach of PCA–ELM proved as a promising technique in forecasting with reduced errors and reduced time. [ABSTRACT FROM AUTHOR]

The activation function represents a crucial component in the design of a convolutional neural network (CNN). It enables the efficient extraction of multiple features from visual patterns, and introduces systemic non-linearity to data processing. This paper proposes a novel and insightful activation method termed FPLUS, which exploits mathematical power function with polar signs in form. It is enlightened by common inverse operations while endowed with an intuitive meaning of bionics. The formulation is derived theoretically under conditions of some prior knowledge and anticipative properties. Subsequently, its feasibility is verified through a series of experiments using typical benchmark datasets. The results indicate that our approach bears superior competitiveness among numerous activation functions, as well as compatible stability across many CNN architectures. Furthermore, we extend the function presented to a more generalized type called PFPLUS with two parameters that can be fixed or learnable, so as to augment its expressive capacity. The outcomes of identical tests serve to validate this improvement. Therefore, we believe the work in this paper holds a certain value in enriching the family of activation units. [ABSTRACT FROM AUTHOR]

Controlling straw burning is important for ensuring the ambient air quality and for sustainable agriculture. Detecting burning straw is vital for managing and controlling straw burning. Existing methods for detecting straw combustion mainly look for combustion products, especially smoke. In this study, the improved You Only Look Once version 5 (YOLOv5s) algorithm was used to detect smoke in Sentinel-2 images captured by remote sensing. Although the original YOLOv5s model had a faster detection speed, its detection accuracy was poor. Thus, a convolutional block attention module was added to the original model. In addition, in order to speed up the convergence of the model, this study replaced the leaky Rectified Linear Unit (leaky ReLU) activation function with the Mish activation function. The accuracy of the improved model was approximately 4% higher for the same detection speed. The improved YOLOv5s had a higher detection accuracy and speed compared to common target detection algorithms, such as RetinaNet, mask Region-Based Convolutional Neural Network (R-CNN), Single-Shot Multibox Detector (SSD), and faster R-CNN. The improved YOLOv5s analyzed an image in 2 ms. In addition, mAP50 exceeded 94%, demonstrating that with this study's improved method, smoke can be quickly and accurately identified. This work may serve as a reference for improving smoke detection, and for the effective management and control of straw burning. [ABSTRACT FROM AUTHOR]

Artificial ANNs (ANNs) are relatively new computational tools used in the development of intelligent systems, some of which are inspired by biological ANNs, and have found widespread application in the solving of a variety of complex real-world problems. It boasts enticing features as well as remarkable data processing capabilities. In this paper, a comprehensive overview of the backpropagation algorithm for digital image denoising was discussed. Then, we presented a probabilistic analysis of how different algorithms address this challenge, arguing that small ANNs can denoise small-scale texture patterns almost as effectively as their larger equivalents. The results also show that self-similarity and ANNs are complementary paradigms for patch denoising, as demonstrated by an algorithm that effectively complements BM3D with small ANNs, surpassing BM3D at a low cost. Here, one of the most significant advantages of this learning technique is that, once taught, digital images may be recovered without prior knowledge of the degradation model (noise/blurring) that caused the digital image to become distorted. [ABSTRACT FROM AUTHOR]

Over the past few decades, convolutional neural networks (CNNs) have achieved outstanding results in addressing a broad scope of computer vision problems. Despite these improvements, fully convolutional Siamese neural networks (FCSNN) still hardly adapt to complex scenes, such as appearance change, scale change, similar objects interference, etc. The present study focuses on an enhanced FCSNN based on convolutional block hyperparameters optimization, a new activation function (ModReLU) and Gaussian low pass filter. The optimization of hyperparameters is an important task, as it has a crucial ascendancy on the tracking process performance, especially when it comes to the initialization of weights and bias. They have to work efficiently with the following activation function layer. Inadequate initialization can result in vanishing or exploding gradients. In the first method, we propose an optimization strategy for initializing weights and bias in the convolutional block to ameliorate the learning of features so that each neuron learns as much as possible. Next, the activation function normalizes the output. We implement the convolutional block hyperparameters optimization by setting the convolutional weights initialization to constant, the bias initialization to zero and the Leaky ReLU activation function at the output. In the second method, we propose a new activation, ModReLU, in the activation layer of CNN. Additionally, we also introduce a Gaussian low pass filter to minimize image noise and improve the structures of images at distinct scales. Moreover, we add a pixel-domain-based color adjustment implementation to enhance the capacity of the proposed strategies. The proposed implementations handle better rotation, moving, occlusion and appearance change problems and improve tracking speed. Our experimental results clearly show a significant improvement in the overall performance compared to the original SiamFC tracker. The first proposed technique of this work surpasses the original fully convolutional Siamese networks (SiamFC) on the VOT 2016 dataset with an increase of 15.42% in precision, 16.79% in AUPC and 15.93% in IOU compared to the original SiamFC. Our second proposed technique also reveals remarkable advances over the original SiamFC with 18.07% precision increment, 17.01% AUPC improvement and an increase of 15.87% in IOU. We evaluate our methods on the Visual Object Tracking (VOT) Challenge 2016 dataset, and they both outperform the original SiamFC tracker performance and many other top performers. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) is progressing rapidly, and in this trend, edge AI has been researched intensively. However, much less work has been performed around the security of edge AI. Machine learning models are a mass of intellectual property, and an optimized network is very valuable. Trained machine learning models need to be black boxes as well because they may give away information about the training data to the outside world. As selecting the appropriate activation functions to enable fast training of accurate deep neural networks is an active area of research, it is important to conceal the information of the activation functions used in a neural network architecture as well. There has been research on the use of physical attacks such as the side-channel attack (SCA) in areas other than cryptography. The SCA is highly effective against edge artificial intelligence due to its property of the device computing close to the user. We studied a previously proposed method to retrieve the activation functions of a black box neural network implemented on an edge device by using simple electromagnetic analysis (SEMA) and improved the signal processing procedure for further noisy measurements. The SEMA attack identifies activation functions by directly observing distinctive electromagnetic (EM) traces that correspond to the operations in the activation function. This method requires few executions and inputs and also has little implementation dependency on the activation functions. We distinguished eight similar activation functions with EM measurements and examined the versatility and limits of this attack. In this work, the machine learning architecture is a multilayer perceptron, evaluated on an Arduino Uno. [ABSTRACT FROM AUTHOR]

Deep learning models are based on a combination of neural network architectures, optimization parameters and activation functions. All of them provide exponential combinations whose computational fitness is difficult to pinpoint. The intricate resemblance of the microscopic features that are found in bone surface modifications make their differentiation challenging, and determining a baseline combination of optimizers and activation functions for modeling seems necessary for computational economy. Here, we experiment with combinations of the most resolutive activation functions (relu, swish, and mish) and the most efficient optimizers (stochastic gradient descent (SGD) and Adam) for bone surface modification analysis. We show that despite a wide variability of outcomes, a baseline of relu–SGD is advised for raw bone surface modification data. For imbalanced samples, augmented datasets generated through generative adversarial networks are implemented, resulting in balanced accuracy and an inherent bias regarding mark replication. In summary, although baseline procedures are advised, these do not prevent to overcome Wolpert's "no free lunch" theorem and extend it beyond model architectures. [ABSTRACT FROM AUTHOR]

The convolutional neural network (CNN) has been widely used in image recognition field due to its good performance. This paper proposes a facial expression recognition method based on the CNN model. Regarding the complexity of the hierarchic structure of the CNN model, the activation function is its core, because the nonlinear ability of the activation function really makes the deep neural network have authentic artificial intelligence. Among common activation functions, the ReLu function is one of the best of them, but it also has some shortcomings. Since the derivative of the ReLu function is always zero when the input value is negative, it is likely to appear as the phenomenon of neuronal necrosis. In order to solve the above problem, the influence of the activation function in the CNN model is studied in this paper. According to the design principle of the activation function in CNN model, a new piecewise activation function is proposed. Five common activation functions (i.e., sigmoid, tanh, ReLu, leaky ReLus and softplus–ReLu, plus the new activation function) have been analysed and compared in facial expression recognition tasks based on the Keras framework. The Experimental results on two public facial expression databases (i.e., JAFFE and FER2013) show that the convolutional neural network based on the improved activation function has a better performance than most-of-the-art activation functions. [ABSTRACT FROM AUTHOR]

Featured Application: This work can be used in engineering and information applications. Multi-class classification is a very important technique in engineering applications, e.g., mechanical systems, mechanics and design innovations, applied materials in nanotechnologies, etc. A large amount of research is done for single-label classification where objects are associated with a single category. However, in many application domains, an object can belong to two or more categories, and multi-label classification is needed. Traditionally, statistical methods were used; recently, machine learning techniques, in particular neural networks, have been proposed to solve the multi-class classification problem. In this paper, we develop radial basis function (RBF)-based neural network schemes for single-label and multi-label classification, respectively. The number of hidden nodes and the parameters involved with the basis functions are determined automatically by applying an iterative self-constructing clustering algorithm to the given training dataset, and biases and weights are derived optimally by least squares. Dimensionality reduction techniques are adopted and integrated to help reduce the overfitting problem associated with the RBF networks. Experimental results from benchmark datasets are presented to show the effectiveness of the proposed schemes. [ABSTRACT FROM AUTHOR]