Search

Recently, deep learning models have achieved great success in computer vision applications, relying on large-scale class-balanced datasets. However, imbalanced class distributions still limit the wide applicability of these models due to degradation in performance. To solve this problem, in this paper, we concentrate on the study of cross entropy which mostly ignores output scores on incorrect classes. This work discovers that neutralizing predicted probabilities on incorrect classes improves the prediction accuracy for imbalanced image classification. This paper proposes a simple but effective loss named complement cross entropy based on this finding. The proposed loss makes the ground truth class overwhelm the other classes in terms of softmax probability, by neutralizing probabilities of incorrect classes, without additional training procedures. Along with it, this loss facilitates the models to learn key information especially from samples on minority classes. It ensures more accurate and robust classification results on imbalanced distributions. Extensive experiments on imbalanced datasets demonstrate the effectiveness of the proposed method., 8 pages, Accepted to Pattern Recognition Letters (PRL), August 2021

Simulation-based hull form optimization is a typical HEB (high-dimensional, expensive computationally, black-box) problem. Conventional optimization algorithms easily fall into the “curse of dimensionality” when dealing with HEB problems. A recently proposed Cross-Entropy (CE) optimization algorithm is an advanced stochastic optimization algorithm based on a probability model, which has the potential to deal with high-dimensional optimization problems. Currently, the CE algorithm is still in the theoretical research stage and rarely applied to actual engineering optimization. One reason is that the Monte Carlo (MC) method is used to estimate the high-dimensional integrals in parameter update, leading to a large sample size. This paper proposes an improved CE algorithm based on quasi-Monte Carlo (QMC) estimation using high-dimensional truncated Sobol subsequence, referred to as the QMC-CE algorithm. The optimization performance of the proposed algorithm is better than that of the original CE algorithm. With a set of identical control parameters, the tests on six standard test functions and a hull form optimization problem show that the proposed algorithm not only has faster convergence but can also apply to complex simulation optimization problems.

Background and study aims Colonoscopy completion reduces post-colonoscopy colorectal cancer. As a result, there have been attempts at implementing artificial intelligence to automate the detection of the appendiceal orifice (AO) for quality assurance. However, the utilization of these algorithms has not been demonstrated in suboptimal conditions, including variable bowel preparation. We present an automated computer-assisted method using a deep convolutional neural network to detect the AO irrespective of bowel preparation. Methods A total of 13,222 images (6,663 AO and 1,322 non-AO) were extracted from 35 colonoscopy videos recorded between 2015 and 2018. The images were labelled with Boston Bowel Preparation Scale scores. A total of 11,900 images were used for training/validation and 1,322 for testing. We developed a convolutional neural network (CNN) with a DenseNet architecture pre-trained on ImageNet as a feature extractor on our data and trained a classifier uniquely tailored for identification of AO and non-AO images using binary cross entropy loss. Results The deep convolutional neural network was able to correctly classify the AO and non-AO images with an accuracy of 94 %. The area under the receiver operating curve of this neural network was 0.98. The sensitivity, specificity, positive predictive value, and negative predictive value of the algorithm were 0.96, 0.92, 0.92 and 0.96, respectively. AO detection was > 95 % regardless of BBPS scores, while non-AO detection improved from BBPS 1 score (83.95 %) to BBPS 3 score (98.28 %). Conclusions A deep convolutional neural network was created demonstrating excellent discrimination between AO from non-AO images despite variable bowel preparation. This algorithm will require further testing to ascertain its effectiveness in real-time colonoscopy.

The cross-entropy (CE) method can accelerate power system reliability evaluation effectively. An importance sampling (IS) based iterative parameter updating algorithm is usually used in CE optimization. However, the efficiency in capturing the concerned samples for parameter updating may be unsatisfactory, especially for a system with intrinsic nature of rare failure events. Besides, the stopping criterion of this iterative algorithm is usually determined subjectively, incurring insufficient or excessive parameter updating and a resultant higher computational burden of the entire simulation. To address the above two problems, a CE method based on subset simulation and minimum computational burden criterion is proposed. In CE optimization, the subset simulation combined with M-H sampling is adopted as an alternative to IS to effectively improve the efficiency of capturing desired samples for parameter updating at each iteration. Moreover, a novel quantitative stopping criterion is presented in such a way that the computational burden for the entire simulation is estimated and compared after each parameter updating iteration, and the optimal parameters corresponding to the minimum computational burden can be determined. The computational performance of the proposed method is validated by comparing the proposed approach with existing methods under several numerical tests including a realistic power system.

A model and training method for observational context classification in CCTV sewer inspection vide frames was developed and researched. The object of research is the process of detection of temporal-spatial context during CCTV sewer inspections. The subjects of the research are machine learning model and training method for classification analysis of CCTV video sequences under the limited and imbalanced training dataset constraint. Objective. Stated research goal is to develop an efficient context classifier model and training algorithm for CCTV sewer inspection video frames under the constraint of the limited and imbalanced labeled training set. Methods. The four-stage training algorithm of the classifier is proposed. The first stage involves training with soft triplet loss and regularisation component which penalises the network’s binary output code rounding error. The next stage is needed to determine the binary code for each class according to the principles of error-correcting output codes with accounting for intra- and interclass relationship. The resulting reference vector for each class is then used as a sample label for the future training with Joint Binary Cross Entropy Loss. The last machine learning stage is related to decision rule parameter optimization according to the information criteria to determine the boundaries of deviation of binary representation of observations for each class from the corresponding reference vector. A 2D convolutional frame feature extractor combined with the temporal network for inter-frame dependency analysis is considered. Variants with 1D Dilated Regular Convolutional Network, 1D Dilated Causal Convolutional Network, LSTM Network, GRU Network are considered. Model efficiency comparison is made on the basis of micro averaged F1 score calculated on the test dataset. Results. Results obtained on the dataset provided by Ace Pipe Cleaning, Inc confirm the suitability of the model and method for practical use, the resulting accuracy equals 92%. Comparison of the training outcome with the proposed method against the conventional methods indicated a 4% advantage in micro averaged F1 score. Further analysis of the confusion matrix had shown that the most significant increase in accuracy in comparison with the conventional methods is achieved for complex classes which combine both camera orientation and the sewer pipe construction features. Conclusions. The scientific novelty of the work lies in the new models and methods of classification analysis of the temporalspatial context when automating CCTV sewer inspections under imbalanced and limited training dataset conditions. Training results obtained with the proposed method were compared with the results obtained with the conventional method. The proposed method showed 4% advantage in micro averaged F1 score. It had been empirically proven that the use of the regular convolutional temporal network architecture is the most efficient in utilizing inter-frame dependencies. Resulting accuracy is suitable for practical use, as the additional error correction can be made by using the odometer data. Розроблено та досліджено модель та метод навчання для класифікації контекстів спостереження на кадрах відеоінспекції стічних труб. Об’єктом дослідження є процес виявлення просторово-часового контексту під час інспекцій стічних труб. Предметом дослідження є модель та метод машинного навчання для класифікаційного аналізу кадрів відеоінспекції в умовах обмеженого та незбалансованого набору розмічених навчальних даних. Мета дослідження – розроблення ефективних моделі і методу машинного навчання для класифікаційного аналізу контексту відеокадрів інспекції стічних труб в умовах обмеженого обсягу та незбалансованості розміченого навчального набору даних. Методи дослідження. Запропоновано чотирьох етапний алгоритм навчання класифікатора. Перший етап полягає у навчанні з нормалізованою триплетною функцією втрат і регуляризуючою складовою, яка штрафує за помилку округлення вихідного сигналу до двійкового подання. Наступний етап полягає у визначенні двійкового коду для кожного класу для реалізації кодів, що виправляють помилки, але з урахуванням внутрішньокласових та міжкласових відношень. Отриманий еталонний двійковий вектор для кожного класу потім використовується як цільова мітка під час наступного етапу навчання з бінарною крос-ентропійною функцією втрат. Останній етап машинного навчання пов’язаний з оптимізацією параметрів правила прийняття рішень за інформаційним критерієм для визначення допустимих меж відхилення двійкового подання спостережень кожного класу від відповідного еталонного вектора. Розглядається 2D згортковий екстрактор ознак у поєднанні з темпоральною мережею для аналізу міжкадрових залежностей. Розглядаються варіанти з 1D згорткова мережа з дірявими регулярними згортками, 1D згорткова мережа з дірвими причинно-наслідковими згортками, рекурентна мережа LSTM та рекурентна мережа GRU. Порівняння ефективності моделей проводиться на основі мікро усередненої F1-міри, обчисленої на тестовому наборі даних. Результати. Результати, отримані за набором даних, наданим Ace Pipe Cleaning, Inc, підтверджують придатність моделі та методу для практичного використання, оскільки отримана точність дорівнює 92%. Порівняння результатів навчання із запропонованим методом та традиційним методом показало перевагу на 4% за мікро-усередним значенням F1-міри. Подальший аналіз матриці помилок показав, що найбільш суттєве підвищення точності порівняно зі традиційними методами досягається для складних класів, які поєднують як орієнтацію камери, так і особливості конструкції стічної труби. Висновки. Наукова новизна роботи полягає у нових моделях та методах класифікаційного аналізу просторово-часового контексту для автоматизації відеоінспекції стічних труб в умовах обмеженого обсягу та незбалансованості розмічених навчальних даних. Результати навчання, отримані за пропонованим методом, порівнюються з результатами, отриманими за допомогою традиційного методу класифікаційного аналізу зображень. Запропонований метод продемонстрував перевагу на 4% за мікро-усередненим значенням F1-міри. Емпірично було доведено, що темпоральна мережа на основі 1D згорткової мережі з дірявими регулярними згортками є найбільш ефективною для аналізу міжкадрових залежностей. Отримана точність забезпечує придатність отриманих моделей для практичного використання, оскільки додаткове виправлення помилок можна реалізувати на основі даних одометра.

In order to improve the classification performance of the model for different kinds of arrhythmias, a parallel classification model of arrhythmia based on DenseNet-BiLSTM is researched and proposed. Firstly, the model adopts a parallel structure. After wavelet denoising and heartbeat segmentation of ECG signals, this model can simultaneously capture the waveform features of small-scale heartbeat and large-scale heartbeat containing RR interval; Then, based on deep learning, Densely connected convolutional network (DenseNet) is applied to improve the model's ability to extract local features of ECG signals, and bidirectional long short-term memory network (BiLSTM) is introduced to improve the performance of the model in extracting time series features of ECG signals; Finally, weighted cross entropy loss function is used to alleviate the class imbalance of arrhythmia, and Softmax function is applied to achieve 4 classifications of arrhythmia. Experiments based on MIT-BIH arrhythmia database show that under the intra-patient paradigm, training time for each epoch, overall accuracy, F 1 and specificity are 42 s, 99.44%, 95.89% and 99.32%, respectively; Under the inter-patient paradigm, training time for each epoch, overall accuracy, F 1 and specificity are 23 s, 92.37%, 63.49% and 94.51%, respectively. Compared with other classification models, the model proposed in this paper has a good classification effect and is expected to be used in clinical auxiliary diagnosis.

A common problem in pixelwise classification or semantic segmentation is class imbalance, which tends to reduce the classification accuracy of minority-class regions. An effective way to address this is to tune the loss function, particularly when Cross Entropy (CE), is used for classification. Although several CE variants have been reported in previous studies to address this problem, for example, Weighted Cross Entropy (WCE), Dual Cross Entropy (DCE), and Focal Loss (FL), each has their own limitations, such as introducing a vanishing gradient, penalizing negative classes inversely, or a sub-optimal loss weighting between classes. This limits their ability to improve classification accuracy or reduces their ease of use. Focal Loss has proven to be effective at balancing loss by increasing the loss on hard-to-classify classes. However, it tends to produce a vanishing gradient during backpropagation. To address these limitations, a Dual Focal Loss (DFL) function is proposed to improve the classification accuracy of the unbalanced classes in a dataset. The proposed loss function modifies the loss scaling method of FL to be effective against a vanishing gradient. In addition, inspired by DCE, a regularization term has also been added to DFL to constrain the negative class labels to further reduce the vanishing gradient effect and increase the loss on hard-to-classify classes. Experimental results show that DFL has better training performance, and provides greater accuracy compared to CE, WCE, FL and DCE in every test run conducted over a variety of different network models and datasets.

Active learning is an effective solution to interactively select a limited number of informative examples and use them to train a learning algorithm that can achieve its optimal performance for specific tasks. It is suitable for medical image applications in which unlabeled data are abundant but manual annotation could be very time-consuming and expensive. However, designing an effective active learning strategy for informative example selection is a challenging task, due to the intrinsic presence of noise in medical images, the large number of images, and the variety of imaging modalities. In this study, a novel low-rank modeling-based multi-label active learning (LRMMAL) method is developed to address these challenges and select informative examples for training a classifier to achieve the optimal performance. The proposed method independently quantifies image noise and integrates it with other measures to guide a pool-based sampling process to determine the most informative examples for training a classifier. In addition, an automatic adaptive cross entropy-based parameter determination scheme is proposed for further optimizing the example sampling strategy. Experimental results on varied medical image datasets and comparisons with other state-of-the-art multi-label active learning methods illustrate the superior performance of the proposed method.

This study proposes an intelligent hybridization between the multi-level cross-entropy optimizer (MCEO) and moth-flame optimization (MFO) algorithms to keep a good compromise between exploration and exploitation. The proposed hybrid multi-level cross-entropy-based moth-flame (MCMF) algorithm uses MCEO as a global search engine during the first phase of the optimization process that allows for fast approximation of the global best position (BP). The boundaries of the search space are then adaptively confined within the effective region around the current BP by applying the proposed search space boundaries confining factor (SSBCF). The modified MFO with two different moth generation patterns is then employed as a local search engine to simultaneously probe for new and proper BPs within the confined and overall search space. This prevents MCMF from becoming trapped in local optima while maintaining a balance between exploration and exploitation. The hybridization between both algorithms allows MCMF to accelerate throughout the early steps of the search process using the high exploration ability of MCEO, whereas, in the later stages of optimization, promising solutions will possess a high probability to be exploited using the higher exploitation power of MFO in the confined space. The competence of the MCMF is compared with other well-known state-of-the-art algorithms on 15 unconstrained benchmark functions and 5 constrained engineering design problems having a wide range of dimensions and varied complexities. The statistical results on the benchmark functions and the solved engineering examples verify that the proposed algorithm can provide very competitive and promising results. The results demonstrate the comprehensive superiority of the hybrid MCMF compared to both MCEO and MFO in terms of fewer function calls, high escaping ability from local optima, and fast convergence speed.

Over the last years, utilizing deep learning for the analysis of survival data has become attractive to many researchers. This has led to the advent of numerous network architectures for the prediction of possibly censored time-to-event variables. Unlike networks for cross-sectional data (used e.g., in classification), deep survival networks require the specification of a suitably defined loss function that incorporates typical characteristics of survival data such as censoring and time-dependent features. Here, we provide an in-depth analysis of the cross-entropy loss function, which is a popular loss function for training deep survival networks. For each time point $t$ t , the cross-entropy loss is defined in terms of a binary outcome with levels “event at or before $t$ t ” and “event after $t$ t ”. Using both theoretical and empirical approaches, we show that this definition may result in a high prediction error and a heavy bias in the predicted survival probabilities. To overcome this problem, we analyze an alternative loss function that is derived from the negative log-likelihood function of a discrete time-to-event model. We show that replacing the cross-entropy loss by the negative log-likelihood loss results in much better calibrated prediction rules and also in an improved discriminatory power, as measured by the concordance index.

Recently, convolutional neural networks have improved the results of salient object detection by a significant margin. Most existing methods focus on aggregating multi-level features but pay little attention to the differences between their spatial resolution. Besides, the widely used binary cross entropy loss treats all pixels equally and neglects the fact that pixels near the salient boundaries are prone to being misclassified. To solve these problems, we propose a novel network named BPFINet to aggregate low-level detail features, high-level semantic information, and global information progressively by using the U-shape Feature Integration Modules (UFIMs). Moreover, a U-shape Self-Refinement Module (USRM) is proposed to generate multi-scale representation from the intra-layer features and fuse them progressively to generate features robust to scale variation of salient objects. Besides, a Channel Compression Module (CCM) is designed to reduce the channel number of certain features and enhance the features by leveraging channel-wise attention. Furthermore, an integrated loss is introduced to highlight pixels near the salient boundaries and solve the problem caused by the imbalance of foreground and background regions. Experimental results on six benchmark datasets prove that our BPFINet is competitive compared with 16 other state-of-the-art methods. The source code will be publicly available at https://github.com/clelouch/BPFINet .

International audience; This paper proposes a new framework for the operational reliability evaluation of integrated electric power-gas networks (IEPGNs). First, a novel approach for modeling the failure modes of natural gas pipelines is presented. This approach utilizes the concept of virtual nodes and employs a gas release rate model to consider the pinhole, hole, and rupture failure modes of pipelines. Thereafter, a four-state Markov model for natural gas-fired generators (NGFGs) with dual-fuel capabilities is proposed. The area risk method is extended to include the proposed reliability models, and the partial reliability indices of the area risk method are evaluated using a non-sequential Monte Carlo simulation (NSMCS). A nonlinear optimization model is also proposed to calculate electric and gas load curtailments for each system state in NSMCS. This model is linearized to obtain a mixed-integer linear programming (MILP) model for reducing the computational burden. The computational performance of NSMCS is further improved by adopting cross entropy (CE)-based importance sampling (IS). Finally, the efficacy of the proposed framework is demonstrated on three test systems. Case studies validate the importance of considering the proposed reliability models of IEPGNs for operational reliability evaluation. The impacts of operational strategies on the operational reliability indices are also demonstrated.

Caching is a promising technique in the next generation cellular network, reducing backhaul traffic while improving network efficiency. In this letter, we introduce a cache-enabled small base station cooperative transmission scheme while considering the macro base station (MBS) silence and mutual information accumulation (MIA). A tractable analytical framework for characterizing the successful transmission probability (STP) of the introduced scheme is described. The STP is maximized by optimizing the caching placement policy based on a constrained cross entropy optimization algorithm. Simulation results demonstrate that the proposed SBS cooperative transmission with MBS silence and MIA achieves higher STP than other compared schemes.

Machine learning models for classification and segmentation typically use loss functions based on binary or categorical cross entropy (BCE/CCE). Recent work has pointed out that cross entropy is not always optimal for maximizing accuracy of classification models. Instead, power error loss has been suggested as a generalization of BCE employing a power error exponent parameter q that can be adapted to output error distributions. In this paper we apply the power error loss to classification and segmentation using Convolutional Neural Network and a variant of DeepLabV3+. In line with previous findings, classification and segmentation performance can be improved significantly by using an appropriate fixed q. While previous works have found improvements mainly for q>1, here we report also cases where optimal q is, publishedVersion

Background: Research has shown that essential proteins play important roles in the development and survival of organisms. Because of the high costs of traditional biological experiments, several computational prediction methods based on known protein-protein interactions (PPIs) have been recently proposed to detect essential proteins. Objective: Here, a novel prediction model called IoMCD is proposed to identify essential proteins by combining known PPIs with a variety of biological information about proteins, including gene expression data and homologous information of proteins. Methods: Compared to the traditional state-of-the-art prediction models, IoMCD involves two kinds of weights that are obtained, respectively, by extracting topological features of proteins from the original known protein–protein interaction (PPI) networks and calculating the Pearson correlation coefficients (PCCs) between the gene expression data of proteins. Based on these two kinds of weights and adopting a cross-entropy method, a unique weight is assigned to each protein. Subsequently, the homologous information of proteins is used to calculate an initial score for each protein. Finally, based on the unique weights and initial score of proteins, an iterative method is designed to measure the essentialities of proteins. Results: Intensive experiments were performed, and simulation results showed that the prediction accuracy of IoMCD, based on the dataset downloaded from the DIP and Gavin databases, was 92.16% and 89.71%, respectively, in the top 1% of the predicted essential proteins. Conclusion: Both simulation results demonstrated that IoMCD can achieve excellent prediction accuracy and could be an effective method for essential protein prediction.

The study shows that there are two main problems that affect the performance of fuzzy time series (FTS) models, namely the selection of the universe of discourse and the determination of the fuzzy degree of the memberships. However, the selection of the appropriate universe of discourse along with the degree of memberships is simultaneously associated with the multiobjective optimization problem (MOOP). Therefore, in this study, an improved version of the quantum optimization algorithm (QOA) has been proposed to find their optimal solutions. This improved QOA is called fast forward quantum optimization algorithm (FFQOA). Finally, by integrating FFQOA with the FTS modelling approach, a hybrid model called fuzzy-quantum time series forecasting model (FQTSFM) is designed. The main objective of FFQOA in FQTSFM is to select the Pareto-optimal front (POF) from all optimal non-dominated Pareto solutions by using the archive and grid concept during simulation. The main advantages of the proposed FQTSFM are that it converges very fast compared to the existing hybridized based FTS models and is able to evolve one-step ahead forecasted results. The FQTSFM is tested with three different datasets, namely daily average temperatures of Taipei, Taiwan Futures Exchange (TAIFEX) index and Taiwan Stock Exchange Corporation (TSEC) weighted index. The performance of the FQTSFM is compared with various established well-known FTS and non-FTS models in terms of different statistical metrics. In the case of daily average temperatures, TAIFEX index and TSEC weighted index datasets, the FQTSFM achieves correlation coefficients of 0.9988, 0.9978 and 0.9768, respectively. For daily average temperatures, TAIFEX index and TSEC weighted index datasets, the Theil’s U statistic values of the FQTSFM are 0.0022, 0.0022 and 0.0054, respectively. The FQTSFM has mean squared errors of 0.3959, 22.3965 and 83.6793 for the daily average temperatures, the TAIFEX index and the TSEC-weighted index datasets, respectively. In the case of daily average temperatures, TAIFEX index and TSEC weighted index datasets, the FQTSFM shows cross entropy values of 3.3498, 189.5018 and 708.0296, respectively. These metrics, including other empirical analyses, confirm that the forecasted accuracy of the proposed FQTSFM exceeds that of well-known time series models.

This brief proposes a scheme to detect and classify across four different classes of white blood cells (WBC) based on their morphological features by ensemble learning applied over multiple deep neural networks (DNN). Contrary to the DNN based WBC classification schemes, this method makes use of a novel least entropy combiner (LEC) network that effectively combines the individual classifier decisions in order to minimize the cross entropy cost. In this brief, two pretrained DNNs, namely DenseNet121 and ResNet50 are used along with a custom-designed convolutional neural network (CNN) to individually produce the class confidence scores which are fed to the proposed LEC network for the ensemble learning purpose. A dataset comprising of around 12, 500 images of four different WBC types, namely Eosinophil , Lymphocyte , Monocyte , and Neutrophil are considered for training and testing the proposed network. The proposed model yields superior performance than the individual networks. Our results demonstrate that the proposed model achieves an overall test accuracy of 96.67%, with precision and sensitivity scores of 96.73%, and 96.62% respectively and the Matthews correlation coefficient (MCC) and kappa scores being 0.9334 and 0.9550, respectively. The overall model is termed as LeukoX (X: Extended).