Search

In this article, the reference and probability matching priors for Cronbach's alpha will be derived. The performance of these two priors will be compared to that of the well-known Jeffreys prior and a divergence prior. Cronbach's alpha is a measure used to assess the reliability of a set of test items. A simulation study will be considered to compare the performance of the priors, where the coverage rates, average interval lengths, and standard deviations of the interval lengths will be computed. A second simulation study will be considered where the mean relative error will be compared for the various priors using three different loss functions. The following loss functions will be considered, Squared error loss, Absolute error loss, and Linex loss. An illustrative example will also be considered. Throughout the article, the random effects approach will be used. [ABSTRACT FROM AUTHOR]

Pansharpening is a traditional image fusion problem where the reference image (or ground truth) is not accessible. Machine-learning-based algorithms designed for this task require an extensive optimization phase of network parameters, which must be performed using unsupervised learning techniques. The learning phase can either rely on a companion problem where ground truth is available, such as by reproducing the task at a lower scale or using a pretext task, or it can use a reference-free cost function. This study focuses on the latter approach, where performance depends not only on the accuracy of the quality measure but also on the mathematical properties of these measures, which may introduce challenges related to computational complexity and optimization. The evaluation of the most recognized no-reference image quality measures led to the proposal of a novel criterion, the Regression-based QNR (RQNR), which has not been previously used. To mitigate computational challenges, an approximate version of the relevant indices was employed, simplifying the optimization of the cost functions. The effectiveness of the proposed cost functions was validated through the reduced-resolution assessment protocol applied to a public dataset (PairMax) containing images of diverse regions of the Earth's surface. [ABSTRACT FROM AUTHOR]

Memory has been the subject of scientific study for nearly 150 years. Because a broad range of studies have been done, we can now assess how effective memory is for a range of materials, from simple nonsense syllables to complex materials such as novels. Moreover, we can assess memory effectiveness for a variety of durations, anywhere from a few seconds up to decades later. Our aim here is to assess a range of factors that contribute to the patterns of retention and forgetting under various circumstances. This was done by taking a meta-analytic approach that assesses performance across a broad assortment of studies. Specifically, we assessed memory across 256 papers, involving 916 data sets (e.g., experiments and conditions). The results revealed that exponential-power, logarithmic, and linear functions best captured the widest range of data compared with power and hyperbolic-power functions. Given previous research on this topic, it was surprising that the power function was not the best-fitting function most often. Contrary to what would be expected, a substantial amount of data also revealed either stable memory over time or improvement. These findings can be used to improve our ability to model and predict the amount of information retained in memory. In addition, this analysis of a large set of memory data provides a foundation for expanding behavioral and neuroimaging research to better target areas of study that can inform the effectiveness of memory. [ABSTRACT FROM AUTHOR]

Loss functions are crucial in training generative adversarial networks (GANs) and shaping the resulting outputs. These functions, specifically designed for GANs, optimize generator and discriminator networks together but in opposite directions. GAN models, which typically handle large datasets, have been successful in the field of deep learning. However, exploring the factors that influence the success of GAN models developed for limited data problems is an important area of research. In this study, we conducted a comprehensive investigation into the loss functions commonly used in GAN literature, such as binary cross entropy (BCE), Wasserstein generative adversarial network (WGAN), least squares generative adversarial network (LSGAN), and hinge loss. Our research focused on examining the impact of these loss functions on improving output quality and ensuring training convergence in single-image GANs. Specifically, we evaluated the performance of a single-image GAN model, SinGAN, using these loss functions in terms of image quality and diversity. Our experimental results demonstrated that loss functions successfully produce high-quality, diverse images from a single training image. Additionally, we found that the WGAN-GP and LSGAN-GP loss functions are more effective for single-image GAN models.

This paper presents a procedure for and evaluation of using a semantic similarity metric as a loss function for neural source code summarization. Code summarization is the task of writing natural language descriptions of source code. Neural code summarization refers to automated techniques for generating these descriptions using neural networks. Almost all current approaches involve neural networks as either standalone models or as part of a pretrained large language models, for example, GPT, Codex, and LLaMA. Yet almost all also use a categorical cross‐entropy (CCE) loss function for network optimization. Two problems with CCE are that (1) it computes loss over each word prediction one‐at‐a‐time, rather than evaluating a whole sentence, and (2) it requires a perfect prediction, leaving no room for partial credit for synonyms. In this paper, we extend our previous work on semantic similarity metrics to show a procedure for using semantic similarity as a loss function to alleviate this problem, and we evaluate this procedure in several settings in both metrics‐driven and human studies. In essence, we propose to use a semantic similarity metric to calculate loss over the whole output sentence prediction per training batch, rather than just loss for each word. We also propose to combine our loss with CCE for each word, which streamlines the training process compared to baselines. We evaluate our approach over several baselines and report improvement in the vast majority of conditions. [ABSTRACT FROM AUTHOR]

The detection of distribution box doors on construction sites is particularly important in site safety management, but the size and posture of distribution boxes vary in different scenarios, and there are still challenges. This article proposes an improved YOLOv8n construction site distribution box door status detection and recognition method. Firstly, Global Attention Mechanism is introduced to reduce information dispersion and enhance global interaction representation, preserving the correlation between spatial and channel information to strengthen the network's feature extraction capability during the detection process. Secondly, to tackle the problem of class imbalance in construction site distribution box door state detection, the Focal_EIoU detection box loss function is used to replace the CIoU loss function, optimizing the model's ability to learn from difficult samples.Lastly,the proposed method is evaluated on a dataset of distribution boxes with different shapes and sizes collected from various construction scenes. Experimental results demonstrate that the improved YOLOv8n algorithm achieves an average precision (mAP) of 82.1% at a speed of 66.7 frames per second, outperforming other classical object detection networks and the original network. This improved method provides an efficient and accurate solution for practical detection tasks in smart chemical sites, especially in enhancing feature extraction and processing difficult sample cases, which has made significant progress. [ABSTRACT FROM AUTHOR]

Relative belief inferences are shown to arise as Bayes rules or limiting Bayes rules. These inferences are invariant under reparameterizations and possess a number of optimal properties. In particular, relative belief inferences are based on a direct measure of statistical evidence. [ABSTRACT FROM AUTHOR]

The potential of neuromorphic (NM) solutions often lies in their low-SWaP (Size, Weight, and Power) capabilities, which often drive their application to domains that could benefit from this. Nevertheless, spiking neural networks (SNNs), with their inherent time-based nature, present an attractive alternative also for areas where data features are present in the time dimension, such as time series forecasting. Time series data, characterized by seasonality and trends, can benefit from the unique processing capabilities of SNNs, which offer a novel approach for this type of task. Additionally, time series data can serve as a benchmark for evaluating SNN performance, providing a valuable alternative to traditional datasets. However, the challenge lies in the real-valued nature of time series data, which is not inherently suited for SNN processing. In this work, we propose a novel spike-encoding mechanism and two loss functions to address this challenge. Our encoding system, inspired by NM event-based sensors, converts the derivative of a signal into spikes, enhancing interoperability with the NM technology and also making the data suitable for SNN processing. Our loss functions then optimize the learning of subsequent spikes by the SNN. We train a simple SNN using SLAYER as a learning rule and conduct experiments using two electricity load forecasting datasets. Our results demonstrate that SNNs can effectively learn from encoded data, and our proposed DecodingLoss function consistently outperforms SLAYER's SpikeTime loss function. This underscores the potential of SNNs for time series forecasting and sets the stage for further research in this promising area of research. [ABSTRACT FROM AUTHOR]

In recent years, machine learning has become a frequently used method for statistical estimation. Random forest regression, decision tree regression, support vector regression and polynomial regression are commonly used supervised machine learning methods. The most commonly used loss function in gradient descent during the optimization phase of these methods is the quadratic loss function, which estimates model parameters by minimizing the cost. The selection of an appropriate loss function is crucial for method selection. There are several loss functions in the literature, such as absolute loss, logarithmic loss and squared error loss. In this study, we propose the use of an inverted normal loss function, which is a finite loss function, to gain a new perspective on minimizing cost and measuring performance in machine learning regression problems. We assert that this loss function provides more accurate estimations of cost minimisation as compared to the quadratic loss function, which is an infinite loss function. This article presents a new approach based on the inverted normal loss function for optimization in regression and performance metrics in machine learning. The procedure and its advantages are illustrated using a simulation study. [ABSTRACT FROM AUTHOR]

NLP or Natural Language Processing is a dramatically developing field. In the "new ordinary" forced by COVID-19, a huge extent of instructive material, news, and conversations occur through computerized media stages. This gives more text information accessible to work on! Initially, straightforward RNNS (Recurrent Neural Networks) were utilized for preparing text classification. Yet, as of late, there have been many new examination distributions that give better cutting-edge outcomes. One such is BERT. In this paper, I’ll put down my presentation of the BERT transformer utilizing different measurements. [ABSTRACT FROM AUTHOR]

Here, a machine-learning method based on a kinetically informed neural network (NN) is introduced. The proposed method is designed to analyze a time series of difference electron-density maps from a time-resolved X-ray crystallographic experiment. The method is named KINNTREX (kinetics-informed NN for time-resolved X-ray crystallography). To validate KINNTREX, multiple realistic scenarios were simulated with increasing levels of complexity. For the simulations, time-resolved X-ray data were generated that mimic data collected from the photocycle of the photoactive yellow protein. KINNTREX only requires the number of intermediates and approximate relaxation times (both obtained from a singular valued decomposition) and does not require an assumption of a candidate mechanism. It successfully predicts a consistent chemical kinetic mechanism, together with difference electron-density maps of the intermediates that appear during the reaction. These features make KINNTREX attractive for tackling a wide range of biomolecular questions. In addition, the versatility of KINNTREX can inspire more NN-based applications to time-resolved data from biological macromolecules obtained by other methods.

Underwater images suffer from severe color attenuation and contrast reduction due to the poor and complex lighting conditions in the water. Most mainstream methods employing deep learning typically require extensive underwater paired training data, resulting in complex network structures, long training time, and high computational cost. To address this issue, a novel ZeroReference Parameter Estimation Network (Zero-UAE) model is proposed in this paper for the adaptive enhancement of underwater images. Based on the principle of light attenuation curves, an underwater adaptive curve model is designed to eliminate uneven underwater illumination and color bias. A lightweight parameter estimation network is designed to estimate dynamic parameters of underwater adaptive curve models. A tailored set of non-reference loss functions are developed for underwater scenarios to fine-tune underwater images, enhancing the network's generalization capabilities. These functions implicitly control the learning preferences of the network and effectively solve the problems of color bias and uneven illumination in underwater images without additional datasets. The proposed method examined on three widely used real-world underwater image enhancement datasets. Experimental results demonstrate that our method performs adaptive enhancement on underwater images. Meanwhile, the proposed method yields competitive performance compared with state-of-the-art other methods. Moreover, the Zero-UAE model requires only 17K parameters, minimizing the hardware requirements for underwater detection tasks. What'more, the adaptive enhancement capability of the Zero-UAE model offers a new solution for processing images under extreme underwater conditions, thus contributing to the advancement of underwater autonomous monitoring and ocean exploration technologies. [ABSTRACT FROM AUTHOR]

Heavily imbalanced datasets are common in lesion segmentation. Specifically, the lesions usually comprise less than 5% of the whole image volume when dealing with brain MRI. A common solution when training with a limited dataset is the use of specific loss functions that rebalance the effect of background and foreground voxels. These approaches are usually evaluated running a single cross-validation split without taking into account other possible random aspects that might affect the true improvement of the final metric (i.e., random weight initialisation or random shuffling). Furthermore, the evolution of the effect of the loss on the heavily imbalanced class is usually not analysed during the training phase. In this work, we present an analysis of different common loss metrics during training on public datasets dealing with brain lesion segmentation in heavy imbalanced datasets. In order to limit the effect of hyperparameter tuning and architecture, we chose a 3D Unet architecture due to its ability to provide good performance on different segmentation applications. We evaluated this framework on two public datasets and we observed that weighted losses have a similar performance on average, even though heavily weighting the gradient of the foreground class gives better performance in terms of true positive segmentation. [ABSTRACT FROM AUTHOR]

In this study, we present the Bayesian estimates of the unknown parameters of the Topp-Leone Lindley distribution using the maximum likelihood and Bayesian methods. In this study, the Bayes theorem was adopted for obtaining the posterior distribution of the shape parameter and scale parameter of the Topp-Leone Lindley distribution assuming the Jeffreys' (non-informative) prior for the shape parameter and the Gamma (conjugate) prior for the scale parameter under three different loss functions namely: Square Error Loss Function, Linear Exponential Loss Function and Generalized Entropy Loss Function. The posterior distribution derived for both parameters are not solvable analytically, it requires a numerical approximation techniques to obtain the solution. The Lindley approximation techniques was adopted to obtain the parameters of interest. The loss function were used to derive the estimates of both parameters with an assumption that the both parameters are unknown and independent. To ascertain the accuracy of these estimators, the proposed Bayesian estimators under different loss functions are compared with the corresponding maximum likelihood estimator using a Monte Carlo simulation on the performance of these estimators according to the mean square error and BIAS based on simulated samples simulated from the Topp-Leone Lindley distribution.. It was also observed for any fixed value of the parameters, as sample size increases, the mean square errors of the Bayesian Estimates and maximum likelihood estimates decrease. Also, the maximum likelihood estimates and Bayesian estimates converge to the same value as the sample gets larger except for Generalized Entropy Loss Function. [ABSTRACT FROM AUTHOR]

Bayesian nonparametric mixture models are widely used to cluster observations. However, one major drawback of the approach is that the estimated partition often presents unbalanced clusters' frequencies with only a few dominating clusters and a large number of sparsely-populated ones. This feature translates into results that are often uninterpretable unless we accept to ignore a relevant number of observations and clusters. Interpreting the posterior distribution as penalized likelihood, we show how the unbalance can be explained as a direct consequence of the cost functions involved in estimating the partition. In light of our findings, we propose a novel Bayesian estimator of the clustering configuration. The proposed estimator is equivalent to a post-processing procedure that reduces the number of sparsely-populated clusters and enhances interpretability. The procedure takes the form of entropy-regularization of the Bayesian estimate. While being computationally convenient with respect to alternative strategies, it is also theoretically justified as a correction to the Bayesian loss function used for point estimation and, as such, can be applied to any posterior distribution of clusters, regardless of the specific model used. [ABSTRACT FROM AUTHOR]

Reconstructing low-dose CT imaging deals with handling the inherent noise within the data, which makes it a complex mathematical problem known as an ill-posed inverse problem. Recent attention has shifted towards deep learning-based techniques in CT image reconstruction. However, these approaches encounter limitations due to extensive data requirements for training and validation. We propose an unsupervised CT reconstruction technique that leverages the power of Deep convolutional neural networks (Deep CNNs), demonstrating that a randomly initialized neural network can serve as a prior. We have proposed a completely unsupervised deep learning technique called Loss-construct unsupervised network (LUNA) adjustment for low-dose CT image reconstruction. Our approach combines the traditional reconstruction technique, i.e., simultaneous algebraic reconstruction technique (SART), and integrates the image prior i.e., weighted total variation (WTV) regularization within the Deep CNN model. The overall reconstruction process is optimized using the alternating direction method of multipliers (ADMM) framework, that balances the neural network’s internal representation with the observed data, yielding a more consistent and accurate final image. The proposed method uses various loss functions to update the Deep CNN. The optimal update of the network depends on the various loss functions. Different loss functions, including perceptual loss, SSIM loss, WL2 loss, WTV loss, and sinogram loss, are used to guide the overall reconstruction. This approach effectively handles the constraints of data limitation of deep learning-based techniques, offering a robust and unsupervised solution for low-dose CT image reconstruction.

Anomaly detection is an important application in factory environments. The sounds emitted by a manufacturing machine during runtime can be indicative of either normal performance or of mechanical failure. Traditionally, cosine losses are frequently utilized in anomalous sound detection (ASD) algorithms. In this work, we evaluate cosine losses within the context and scope of Deep Metric Learning (DML), and we investigate various ways to modify and potentially boost their performance. The impact of each modification on the performance of ASD systems is studied under extensive experimental settings and ablations utilizing both the DCASE 2022 and DCASE 2023 machine ASD datasets. Additionally, under the framework of DML, we develop a key new insight into the inner workings of cosine losses, and we verify it with an ablation.

This research work introduces two novel loss functions, pattern-loss (POL) and label similarity-based instance modeling (LSIM), for improving the performance of multi-label classification using artificial neural network-based techniques. These loss functions incorporate additional optimization constraints based on the distribution of multi-label class patterns and the similarity of data instances. By integrating these patterns during the network training process, the trained model is tuned to align with the existing patterns in the training data. The proposed approach decomposes the loss function into two components: the cross entropy loss and the pattern loss derived from the distribution of class-label patterns. Experimental evaluations were conducted on eight standard datasets, comparing the proposed methods with three existing techniques.The results demonstrate the effectiveness of the proposed approach, with POL and LSIM consistently achieving superior accuracy performance compared to the benchmark methods.

Manually checking for defects on industrial parts such as steel surfaces is ineffective, error-prone, and can damage a company’s reputation. Current automated methods often lack accuracy or real-time detection capabilities. Early detection allows for timely corrective action, such as removing defective parts or adjusting production parameters which can streamline the manufacturing process and improve brand reputation, customer satisfaction, and compliance with industry standards. This paper presents MoL-YOLOv7 (MobileNet integrated with attention and loss function to You Only Look Once version 7), a deep learning model for accurate and real-time detection of steel defects. MoL-YOLOv7 modifies the YOLOv7 model by inserting a MobileNet block that reduces computational complexity while maintaining accuracy, allowing for faster detection. Adding the SimAm (Simple Parameter free Attention module) attention mechanism to the MobileNet block refines feature representations for complex tasks such as steel defect detection. Finally, replacing loss functions with EIoU (Effective IoU), WIoU (Wise-IoU), and SIoU (Scylla-IoU) improves the localization accuracy and addresses the class imbalance in the data. The modified model achieves high accuracy and real-time detection, enabling a streamlined defect detection process. Experimental results show that the modified model involving different loss functions used in this work achieves high accuracy, i.e. 0.5% to 3.5% higher than the original model YOLOv7. The superiority and validity of our modified model are demonstrated by comparison with other attention mechanisms and loss functions integrated into YOLOv7, and also on different texture datasets, putting forward a modified method to detect surface defects on steel strips in daily operations.

Classification applications in order to obtain the desired information from satellite images are one of the increasing areas of study. Remote sensing satellite images are very difficult to obtain, but nevertheless, they can be used in many different areas. In this context, satellite images have become an important data source for land use and land cover analysis. In this study, the public EuroSAT dataset, which is compatible with land cover and land use classification, has been used. There are a total of 27000 images in this dataset, with ten different class labels, such as industry, forest, and lake, and approximately 3000 images in each. In order to perform a more precise and efficient classification study, the images have first been subjected to data augmentation using various techniques and have been divided into a training, test, and validation set. In terms of classification, six different convolutional neural network (CNN) architectures and two different vision transformer architectures have been used. Three experiments have been carried out for the proposed system, and the selected models have been trained with three different loss functions, and the results have been evaluated. The first of the loss functions used is cross-entropy, which is one of the most well-known and basic functions; the second is label smoothing cross-entropy; and the third is a new custom loss function that has been improved and consists of a special combination of both loss functions. The eight proposed models have been trained for all three loss functions; performance evaluations have been made by analyzing metrics such as accuracy, precision, and recall; and comparisons have been realized. After the proposed models have been evaluated in terms of these metrics, a Fuzzy Cognitive Map (FCM) has been created to integrate the model outputs to obtain the final classification result. For this construction, the accuracy of each model has been used in the neighborhood matrix of FCM. The accuracy of the proposed FCM method has been calculated as 99.97%. When the results of the proposed different models have been examined, it has been clearly seen that the new loss function improved in this study has been more successful than the other two loss functions in all models. In addition, the simulation results of FCM-based integration indicate that our approach has very high accuracy.

Enhancing the crop yield predictability in intercropping systems is important for optimizing agricultural productivity. However, accurately predicting yield in such systems is quite challenging due to complex interactions between crops. This study introduces an advanced methodology using integrated loss functions within an optimally driven Feedback Neural Network (FNN) approach to improve yield prediction in a pea-cucumber intercropping systems. Traditional models relying only on Mean Square Error (MSE) loss function often unable to capture the complexity of models, leading to suboptimal performance. To address this limitation, the advanced loss functions are introduced like Dynamic Margin Loss (DML), Risk-Adjusted Loss (RAL), Quantile Loss (QL), and Hybrid Agronomic Efficiency Loss (HAEL) along with three optimizers such as Adaptive Momentum (Adam), Root Mean Square Propagation (RMSprop), and Adaptive delta (Adadelta). These loss functions incorporate risk, uncertainty, and agronomic efficiency into the model training process, enhances predictive capabilities and robustness. This proposed framework is able to capture the complexity of yield prediction by incorporating agricultural factors. While Gradient Boost Machines (GBM) and Long Short Term Memory (LSTM) have some potential, they are not able to capture these dynamics. The sensitivity and weight analysis also focuses that HAEL targets important agronomic factors such as nitrogen uptake and residue biomass, which provide a holistic view of yield prediction. The proposed approach improves the predictive performance compared to traditional models and helps to identify the importance of features, which makes it an effective tool for decision making in sustainable agriculture. Selecting appropriate loss functions is essential to improve the accuracy and robustness of crops yield prediction models. Thus, study provides a strong foundation for enhancing yield prediction in intricate intercropping systems, which all significantly enhance the advancement of precision agriculture.

This paper proposes a fabric defect detection algorithm based on the SA-Pix2pix network and transfer learning to address the issue of insufficient accuracy in detecting complex pattern fabric defects in scenarios with limited sample data. Its primary contribution lies in treating defects as disruptions to the fabric's texture. It leverages a generative adversarial network to reconstruct defective images, restoring them to images of normal fabric texture. Subsequently, the reconstituted images are subjected to dissimilarity calculations against defective images, leading to image segmentation for the purpose of defect detection. This approach addresses the issues of poor defect image reconstruction accuracy due to the limited ability of remote dependency modeling within the generator's convolutional neural network. It also tackles deficiencies in the generative adversarial network's loss function in handling image details. To enhance the structure and loss function of the generative adversarial network, it introduces self-attention mechanisms, L1 loss, and an improved structural loss, thus mitigating the problems of low defect image reconstruction accuracy and insufficient image detail handling by the network. To counteract the issue of declining model training accuracy in the face of sparse complex fabric defect samples, a channel-wise domain transfer learning approach is introduced. This approach constrains the training of the target network through feature distribution, thereby overcoming the problem of target network overfitting caused by limited sample data. The study employs three methods to experimentally compare and investigate five distinct complex pattern fabric defects. The results demonstrate that, when compared to two other defect detection methods, the approach advocated in this paper exhibits superior detection accuracy in scenarios with limited sample data. [ABSTRACT FROM AUTHOR]

This work is devoted to improving the quality of segmentation of images of various scientific papers and legal acts by neural network models by training them using modified loss functions that take into account special features of images of the appropriate subject domain. The analysis of existing loss functions is carried out, and new functions are proposed that work both with the coordinates of bounding boxes and use information about the pixels of the input image. To assess the quality, a neural network segmentation model with modified loss functions is trained, and a theoretical assessment is carried out using a simulation experiment showing the convergence rate and segmentation error. As a result of the study, rapidly converging loss functions are created that improve the quality of document image segmentation using additional information about the input data. [ABSTRACT FROM AUTHOR]

Object detection remains a pivotal aspect of remote sensing image analysis, and recent strides in Earth observation technology coupled with convolutional neural networks (CNNs) have propelled the field forward. Despite advancements, challenges persist, especially in detecting objects across diverse scales and pinpointing small-sized targets. This paper introduces YOLO-SE, a novel YOLOv8-based network that innovatively addresses these challenges. First, the introduction of a lightweight convolution SEConv in lieu of standard convolutions reduces the network's parameter count, thereby expediting the detection process. To tackle multi-scale object detection, the paper proposes the SEF module, an enhancement based on SEConv. Second, an ingenious Efficient Multi-Scale Attention (EMA) mechanism is integrated into the network, forming the SPPFE module. This addition augments the network's feature extraction capabilities, adeptly handling challenges in multi-scale object detection. Furthermore, a dedicated prediction head for tiny object detection is incorporated, and the original detection head is replaced by a transformer prediction head. To address adverse gradients stemming from low-quality instances in the target detection training dataset, the paper introduces the Wise-IoU bounding box loss function. YOLO-SE showcases remarkable performance, achieving an average precision at IoU threshold 0.5 (AP50) of 86.5% on the optical remote sensing dataset SIMD. This represents a noteworthy 2.1% improvement over YOLOv8 and YOLO-SE outperforms the state-of-the-art model by 0.91%. In further validation, experiments on the NWPU VHR-10 dataset demonstrated YOLO-SE's superiority with an accuracy of 94.9%, surpassing that of YOLOv8 by 2.6%. The proposed advancements position YOLO-SE as a compelling solution in the realm of deep learning-based remote sensing image object detection. [ABSTRACT FROM AUTHOR]

Deep representation learning has gained significant momentum in advancing text-dependent speaker verification (TD-SV) systems. When designing deep neural networks (DNN) for extracting bottleneck (BN) features, the key considerations include training targets, activation functions, and loss functions. In this paper, we systematically study the impact of these choices on the performance of TD-SV. For training targets, we consider speaker identity, time-contrastive learning (TCL), and auto-regressive prediction coding, with the first being supervised and the last two being self-supervised. Furthermore, we study a range of loss functions when speaker identity is used as the training target. With regard to activation functions, we study the widely used sigmoid function, rectified linear unit (ReLU), and Gaussian error linear unit (GELU). We experimentally show that GELU is able to reduce the error rates of TD-SV significantly compared to sigmoid, irrespective of the training target. Among the three training targets, TCL performs the best. Among the various loss functions, cross-entropy, joint-softmax, and focal loss functions outperform the others. Finally, the score-level fusion of different systems is also able to reduce the error rates. To evaluate the representation learning methods, experiments are conducted on the RedDots 2016 challenge database consisting of short utterances for TD-SV systems based on classic Gaussian mixture model-universal background model (GMM-UBM) and i-vector methods.

The potential of neuromorphic (NM) solutions often lies in their low-SWaP (Size, Weight, and Power) capabilities, which often drive their application to domains that could benefit from this. Nevertheless, spiking neural networks (SNNs), with their inherent time-based nature, present an attractive alternative also for areas where data features are present in the time dimension, such as time series forecasting. Time series data, characterized by seasonality and trends, can benefit from the unique processing capabilities of SNNs, which offer a novel approach for this type of task. Additionally, time series data can serve as a benchmark for evaluating SNN performance, providing a valuable alternative to traditional datasets. However, the challenge lies in the real-valued nature of time series data, which is not inherently suited for SNN processing. In this work, we propose a novel spike-encoding mechanism and two loss functions to address this challenge. Our encoding system, inspired by NM event-based sensors, converts the derivative of a signal into spikes, enhancing interoperability with the NM technology and also making the data suitable for SNN processing. Our loss functions then optimize the learning of subsequent spikes by the SNN. We train a simple SNN using SLAYER as a learning rule and conduct experiments using two electricity load forecasting datasets. Our results demonstrate that SNNs can effectively learn from encoded data, and our proposed DecodingLoss function consistently outperforms SLAYER’s SpikeTime loss function. This underscores the potential of SNNs for time series forecasting and sets the stage for further research in this promising area of research.

In recent years, deep learning methods have shown significant potential in soil moisture modeling. However, a prominent limitation of deep learning approaches has been the absence of physical mechanisms. To address this challenge, this study introduces two novel loss functions designed around physical mechanisms to guide deep learning models in capturing physical information within the data. These two loss functions are crafted to leverage the monotonic relationships between surface water variables and shallow soil moisture as well as deep soil water. Based on these physically-guided loss functions, two physically-guided Long Short-Term Memory (LSTM) networks, denoted as PHY-LSTM and PHYs-LSTM, are proposed. These networks are trained on the global ERA5-Land dataset, and the results indicate a notable performance improvement over traditional LSTM models. When used for global soil moisture forecasting for the upcoming day, PHY-LSTM and PHYs-LSTM models exhibit closely comparable results. In comparison to conventional data-driven LSTM models, both models display a substantial enhancement in various evaluation metrics. Specifically, PHYs-LSTM exhibits improvements in several key performance indicators: an increase of 13.6% in Kling-Gupta Efficiency (KGE), a 20.7% increase in Coefficient of Determination (R2), an 8.2% reduction in Root Mean Square Error (RMSE), and a 4.4% increase in correlation coefficient (R). PHY-LSTM also demonstrates improvements, with a 14.8% increase in KGE, a 19.6% increase in R2, an 8.2% reduction in RMSE, and a 4.4% increase in R. Additionally, both models exhibit enhanced physical consistency over a wide geographical area. Experimental results strongly emphasize that the incorporation of physical mechanisms can significantly bolster the predictive capabilities of data-driven soil moisture models.

In this study, efficient Bayes estimators of sensitive proportion are proposed. It is documented that indirect reports increase variances of the estimates. To counteract this increase in variances we divided the total sample size, n = n1+n2, such that n1 individuals record direct responses and n2 individuals record indirect responses. The decision that a group of individuals should report indirect or direct responses would be based on distinct known factors. Bayes estimates and subsequent posterior risks are calculated taking into account different prior distributions, loss functions and a generalized randomized response technique. The impact of design parameters and the number of responses obtained using direct and indirect questioning techniques on the relative efficiencies are investigated. Graphical and numerical results indicate that the proposed estimators are better than the existing. [ABSTRACT FROM AUTHOR]

Robust loss functions are crucial for training models with strong generalization capacity in the presence of noisy labels. The commonly used Cross Entropy (CE) loss function tends to overfit noisy labels, while symmetric losses that are robust to label noise are limited by their symmetry conditions. We conduct an analysis of the gradient of CE and identify the main difficulty posed by label noise: the imbalance of gradient norm among samples. Inspired by long-tail learning, we propose a gradient prior (GP)-based logit adjustment method to mitigate the impact of label noise. This method makes full use of the gradient of samples to adjust the logit, enabling DNNs to effectively ignore noisy samples and instead focus more on learning hard samples. Experiments on benchmark datasets demonstrate that our method significantly improves the performance of CE and outperforms existing methods, especially in the case of symmetric noise. Experiments on the object detection dataset Pascal VOC further verify the plug-and-play and effective robustness of our method. [ABSTRACT FROM AUTHOR]

In 2007, Cheng and colleagues published their influential review wherein they analyzed the literature on spatial cue interaction during navigation through a Bayesian lens, and concluded that models of optimal cue integration often applied in psychophysical studies could explain cue interaction during navigation. Since then, numerous empirical investigations have been conducted to assess the degree to which human navigators are optimal when integrating multiple spatial cues during a variety of navigation-related tasks. In the current review, we discuss the literature on human cue integration during navigation that has been published since Cheng et al.'s original review. Evidence from most studies demonstrate optimal navigation behavior when humans are presented with multiple spatial cues. However, applications of optimal cue integration models vary in their underlying assumptions (e.g., uninformative priors and decision rules). Furthermore, cue integration behavior depends in part on the nature of the cues being integrated and the navigational task (e.g., homing versus non-home goal localization). We discuss the implications of these models and suggest directions for future research. [ABSTRACT FROM AUTHOR]

This research aims to develop a multireliability inference method for stress–strength variables based on progressive first failure and an inverse Lomax distribution. This research examines the difficulties associated with estimating the stress–strength reliability function, R, when X, Y, and Z are drawn from three different Inverse Lomax distributions. On the basis of progressive first‐failure censored samples, reliability estimators for multi‐stress–strength Inverse Lomax distributions are estimated using the maximum likelihood, maximum product of spacing, and Bayesian estimation methods. The Bayes estimate of R is obtained using the MCMC method for a symmetric loss function. Monte Carlo simulations and real‐data applications are used to assess and compare the performance of the various suggested estimators. [ABSTRACT FROM AUTHOR]

In multiple linear regression models, the explanatory variables should be uncorrelated within each other but this assumption is violated in most of the cases. Generally, ordinary least square (OLS) estimator produces larger variances when explanatory variables are highly multicollinear. So, in this paper, we propose some new ridge parameters under Bayesian perspective relative to different loss functions, using Tierney and Kadane (T-K) approximation technique to overcome the effect of multicollinearity. We conduct the simulation study to compare the performance of the proposed estimators with OLS estimator and ordinary ridge estimator with some available best ridge parameters using mean squared error as the performance evaluation criterion. A real application is also consider to show the superiority of proposed estimators against competitive estimators. Based on the results of simulation and real application, we conclude that Bayesian ridge parameter estimated under general entropy loss function is better as compared to the OLS estimator and ordinary ridge estimator, when explanatory variables are small. This statement is also true for larger explanatory variables with small sample size. While for larger sample sizes and explanatory variables, the ordinary ridge estimator with best ridge parameter gives the better performance as others. [ABSTRACT FROM AUTHOR]

Deep representation learning has gained significant momentum in advancing text-dependent speaker verification (TD-SV) systems. When designing deep neural networks (DNN) for extracting bottleneck (BN) features, the key considerations include training targets, activation functions, and loss functions. In this paper, we systematically study the impact of these choices on the performance of TD-SV. For training targets, we consider speaker identity, time-contrastive learning (TCL), and auto-regressive prediction coding, with the first being supervised and the last two being self-supervised. Furthermore, we study a range of loss functions when speaker identity is used as the training target. With regard to activation functions, we study the widely used sigmoid function, rectified linear unit (ReLU), and Gaussian error linear unit (GELU). We experimentally show that GELU is able to reduce the error rates of TD-SV significantly compared to sigmoid, irrespective of the training target. Among the three training targets, TCL performs the best. Among the various loss functions, cross-entropy, joint-softmax, and focal loss functions outperform the others. Finally, the score-level fusion of different systems is also able to reduce the error rates. To evaluate the representation learning methods, experiments are conducted on the RedDots 2016 challenge database consisting of short utterances for TD-SV systems based on classic Gaussian mixture model-universal background model (GMM-UBM) and i-vector methods. [ABSTRACT FROM AUTHOR]

Statistical data analysis and machine learning heavily rely on error measures for regression, classification, and forecasting. Bregman divergence (BD ) is a widely used family of error measures, but it is not robust to outlying observations or high leverage points in large- and high-dimensional datasets. In this paper, we propose a new family of robust Bregman divergences called "robust- BD " that are less sensitive to data outliers. We explore their suitability for sparse large-dimensional regression models with incompletely specified response variable distributions and propose a new estimate called the "penalized robust- BD estimate" that achieves the same oracle property as ordinary non-robust penalized least-squares and penalized-likelihood estimates. We conduct extensive numerical experiments to evaluate the performance of the proposed penalized robust- BD estimate and compare it with classical approaches, and show that our proposed method improves on existing approaches. Finally, we analyze a real dataset to illustrate the practicality of our proposed method. Our findings suggest that the proposed method can be a useful tool for robust statistical data analysis and machine learning in the presence of outliers and large-dimensional data. [ABSTRACT FROM AUTHOR]

This paper considers properties of half-normal distribution using informative priors under the Bayesian criterion. It employs the squared root inverted gamma, Chi-square and Rayleigh distributions as the prior distribution to construct the Posterior distributions of the respective distributional parameters. Hyperparameters are elicited via prior predictive distribution. The properties of posterior distribution are studied, and their graphs are presented using a real data set. A comprehensive simulation scheme is conducted using informative priors. Bayes estimates are obtained using the loss functions (squared error loss function, modified loss function, quadratic loss function and Degroot loss function). Statistical inferences interval estimates and Bayesian hypothesis testing are presented to demonstrate the usefulness of the study. [ABSTRACT FROM AUTHOR]

The objective of this study is to supply an overview of research work based on video-based networks and tiny object identification. The identification of tiny items and video objects, as well as research on current technologies, are discussed first. The detection, loss function, and optimization techniques are classified and described in the form of a comparison table. These comparison tables are designed to help you identify differences in research utility, accuracy, and calculations. Finally, it highlights some future trends in video and small object detection (people, cars, animals, etc.), loss functions, and optimization techniques for solving new problems.

Deep learning-based models generalize better to unknown data samples after being guided “where to look” by incorporating human perception into training strategies. We made an observation that the entropy of the model’s salience trained in that way is lower when compared to salience entropy computed for models training without human perceptual intelligence. The research problem addressed by this paper is whether lowering the entropy of model’s class activation map helps in further increasing the performance, on top of the performance increase we observe for human saliency-based model’s training. In this paper we propose and evaluate four new entropy-based loss functions controlling the model’s focus, covering the full range of the level of such control, from none to its “aggresive” minimization. We show, using a problem of synthetic face detection, that improving the model’s focus, through lowering entropy by the proposed loss components, leads to models that perform better in an open-set scenario (in which the test samples are synthesized by unknown generative models): the obtained average Area Under the ROC curve (AUROC) ranges from 0.72 to 0.78, compared to AUROC = 0.64 observed for a state-of-the-art human-salience-only-based control of the model’s focus. We also show that optimal performance is obtained when the model’s loss function blends three aspects: regular classification performance, low-entropy of the model’s focus, and closeness of the model’s focus to human saliency. The major conclusion from this work is that maximization of the model’s focus is an important regularizer allowing the models to generalize better in an open set scenario. Future work directions include methods of blending classification-, human salience-, and model’s salience entropy-based loss components to achieve optimal performance in other domains than the synthetic face detection.

Injecting prior knowledge into the learning process of a neural architecture is one of the main challenges currently faced by the artificial intelligence community, which also motivated the emergence of neural-symbolic models. One of the main advantages of these approaches is their capacity to learn competitive solutions with a significant reduction of the amount of supervised data. In this regard, a commonly adopted solution consists of representing the prior knowledge via first-order logic formulas, then relaxing the formulas into a set of differentiable constraints by using a t-norm fuzzy logic. This paper shows that this relaxation, together with the choice of the penalty terms enforcing the constraint satisfaction, can be unambiguously determined by the selection of a t-norm generator, providing numerical simplification properties and a tighter integration between the logic knowledge and the learning objective. When restricted to supervised learning, the presented theoretical framework provides a straight derivation of the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. However, the proposed learning formulation extends the advantages of the cross-entropy loss to the general knowledge that can be represented by neural-symbolic methods. In addition, the presented methodology allows the development of novel classes of loss functions, which are shown in the experimental results to lead to faster convergence rates than the approaches previously proposed in the literature. [ABSTRACT FROM AUTHOR]

This paper deals with the analysis of the occurrence of instantaneous failures in Lindley distribution using progressive type-II censored samples. The test items that fail at a time are called instantaneous failures. Such failures are naturally experienced in life testing experiments, clinical trials, weather predictions, geographic information systems, athlete performance analysis, and many other real fields. These occurrences may be due to the inferior quality of a product or service, faulty construction, or alignment of events/objects, or due to no response to the treatments. Such failures usually discard the assumption of a single-mode distribution and hence the usual method of modeling and inference procedures may not be accurate in practice. To tackle this problem, one must use a non-standard mixture of degenerate distribution degenerated at zero and a standard distribution of a continuous or discrete variable. In this paper, we have considered a non-standard mixture model with continuous Lindley failure distribution for positive components under a progressive type-II censoring scheme. We obtained the maximum likelihood estimators of the proposed distribution and its asymptotic, bootstrap-p (boot-p), and bootstrap-t (boot-t) confidence intervals are derived. The Bayes estimators for the proposed distribution parameters, reliability function, and hazard rate function with their highest posterior density credible intervals using informative priors and non-informative priors under different loss functions are obtained. The performances of different estimators are studied using the MCMC simulation technique. Two real-life data sets have been analyzed for illustration purposes. [ABSTRACT FROM AUTHOR]

A probabilistic seismic loss assessment of RC high-rise (RCHR) buildings designed according to Eurocode 8 and located in the Southern Euro-Mediterranean zone is presented herein. The loss assessment methodology is based on a comprehensive simulation approach which takes into account ground motion (GM) uncertainty, and the random effects in seismic demand, as well as in predicting the damage states (DSs). The methodology is implemented on three RCHR buildings of 20-story, 30-story and 40-story with a core wall structural system. The loss functions described by a cumulative lognormal probability distribution are obtained for two intensity levels for a large set of simulations (NLTHAs) based on 60 GM records with a wide range of magnitude (M), distance to source (R) and different site soil conditions (SS). The losses expressed in percent of building replacement cost for RCHR buildings are obtained. In the estimation of losses, both structural (S) and nonstructural (NS) damage for four DSs are considered. The effect of different GM characteristics (M, R and SS) on the obtained losses are investigated. Finally, the estimated performance of the RCHR buildings are checked to ensure that they fulfill limit state requirements according to Eurocode 8. [ABSTRACT FROM AUTHOR]

In this study, we propose a cost-sensitive learning approach applied on neural networks to deal with classification under imbalanced domains. Our approach is able to automatically learn robust features for both frequent and rare classes by automatically assigning misclassification penalties to each class based the frequency of occurrence of that class. This approach is investigated in the context of shallow networks (multi-layer perceptrons) and deep networks (convolutional neural networks). Moreover, it offers not only a better convergence but also a faster convergence since it can boost optimization by increasing weight gradients which are getting small due to their fitting to the frequent classes. Extensive experiments were carried out on one- and two-dimensional datasets. Running experiments using several loss functions showed the efficacity of our approach on loss functions which do not have probability estimates. Additionally, our approach achieved a good performance compared to common undersampling and oversampling methods as well as models based on generative adversarial networks. [ABSTRACT FROM AUTHOR]

Traditional detect and count strategy cannot well handle the extremely crowded footage in computer vision- based counting task. In recent years, deep learning approaches have been widely explored to tackle this challenge. By regressing visual features to density map, the total crowd number can be predicted while avoids the detection of their actual positions. Efforts of improving performance distribute at various phases of the detecting pipeline, such as optimizing feature extraction and eliminating deviation of regressed density map etc. In this article, we conduct a thorough review on the most representative and state-of-the-art techniques. They are systematically categorized into three topics: the evolving of front-end network, the handling of unbalanced density map prediction, and the selection of loss function. After evaluating most significant techniques, innovations of the state-of-the-art are inspected in detail to analyze specific reasons of achieving high performances. As conclusion, possible directions of enhancement are discussed to provide insights of future research. [ABSTRACT FROM AUTHOR]

Recently, generative adversarial networks (GANs) have progressed enormously, which makes them able to learn complex data distributions in particular faces. More and more efficient GAN architectures have been designed and proposed to learn the different variations of faces, such as cross pose, age, expression, and style. These GAN-based approaches need to be reviewed, discussed, and categorized in terms of architectures, applications, and metrics. Several reviews that focus on the use and advances of GAN in general have been proposed. However, to the best of our knowledge, the GAN models applied to the face, which we call facial GANs, have never been addressed. In this article, we review facial GANs and their different applications. We mainly focus on architectures, problems, and performance evaluation with respect to each application and used datasets. More precisely, we review the progress of architectures and discuss the contributions and limits of each. Then, we expose the encountered problems of facial GANs and propose solutions to handle them. Additionally, as GAN evaluation has become a notable current defiance, we investigate the state-ofthe- art quantitative and qualitative evaluation metrics and their applications. We conclude this work with a discussion on the face generation challenges and propose open research issues. [ABSTRACT FROM AUTHOR]