Search

Temperature prediction is an important and significant step for monitoring global warming and the environment to save and protect human lives. The climatology parameters such as temperature, pressure, and wind speed are time-series data and are well predicted with data driven models. However, data-driven models have certain constraints, due to which these models are unable to predict the missing values and erroneous data caused by factors like sensor failure and natural disasters. In order to solve this issue, an efficient hybrid model, i.e., attention-based bidirectional long short term memory temporal convolution network (ABTCN) architecture is proposed. ABTCN uses k-nearest neighbor (KNN) imputation method for handling the missing data. A bidirectional long short term memory (Bi-LSTM) network with self-attention mechanism and temporal convolutional network (TCN) model that aids in the extraction of features from complex data and prediction of long data sequence. The performance of the proposed model is evaluated in comparison to various state-of-the-art deep learning models using error metrics such as MAE, MSE, RMSE, and R 2 score. It is observed that our proposed model is superior over other models with high accuracy., (© 2023. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

The rapid growth of the global population has placed immense pressure on agriculture to enhance food production while addressing environmental and socioeconomic challenges such as biodiversity loss, water scarcity, and climate variability. Addressing these challenges requires adopting modern techniques and advancing agricultural research. Although some techniques, such as machine learning and deep learning, are increasingly used in agriculture, progress is constrained by the lack of large labelled datasets. This constraint arises because collecting data is often time-consuming, labour-intensive, and requires expert knowledge for data annotation. To mitigate data limitations, transfer learning (TL) offers a viable solution by allowing pre-trained models to be adapted for agricultural applications. Many researchers have demonstrated TL's potential to advance agriculture. Despite its importance, there is a lack of a comprehensive review, which could be essential to guide researchers in this field. Given the significance and the lack of a review paper, this paper provides a review dedicated to TL in agriculture, offering three main contributions. First, we provide an in-depth background study on TL and its applications in agriculture. Second, we offer a comprehensive examination of TL-based agricultural applications, covering pre-trained models, dataset sources, input image types, implementation platforms, and TL approaches. Third, based on an exploration of the existing studies, we identify the challenges faced when applying TL in agriculture. Finally, to address the identified challenges, we recommend suggestions for future research directions. [ABSTRACT FROM AUTHOR]

The off-target activities within the CRISPR-Cas9 system remains a formidable barrier to its broader application and development. Recent advancements have highlighted the potential of deep learning models in predicting these off-target effects, yet they encounter significant hurdles including imbalances within datasets and the intricacies associated with encoding schemes and model architectures. To surmount these challenges, our study innovatively introduces an Efficiency and Specificity-Based (ESB) class rebalancing strategy, specifically devised for datasets featuring mismatches-only off-target instances, marking a pioneering approach in this realm. Furthermore, through a meticulous evaluation of various One-hot encoding schemes alongside numerous hybrid neural network models, we discern that encoding and models of moderate complexity ideally balance performance and efficiency. On this foundation, we advance a novel hybrid model, the CRISPR-MCA, which capitalizes on multi-feature extraction to enhance predictive accuracy. The empirical results affirm that the ESB class rebalancing strategy surpasses five conventional methods in addressing extreme dataset imbalances, demonstrating superior efficacy and broader applicability across diverse models. Notably, the CRISPR-MCA model excels in off-target effect prediction across four distinct mismatches-only datasets and significantly outperforms contemporary state-of-the-art models in datasets comprising both mismatches and indels. In summation, the CRISPR-MCA model, coupled with the ESB rebalancing strategy, offers profound insights and a robust framework for future explorations in this field. Author summary: In the field of gene editing, the application of deep learning technologies holds significant promise for predicting off-target effects in the CRISPR-Cas9 system. Nevertheless, one of the primary challenges encountered is the extreme imbalance among classes within the off-target datasets, which severely hampers the predictive accuracy for certain classes. Furthermore, as an array of sequence encoding methods continue to evolve, there has been a corresponding increase in model complexity. Addressing these issues, we introduce a novel Efficiency and Specificity-Based (ESB) class rebalancing strategy designed to mitigate the impact of class imbalance. Additionally, we assess the influence of six encoding schemes and four distinct architectural approaches on the prediction performance, employing four benchmark datasets for validation. Building upon these insights, we have developed a new hybrid model, termed CRISPR-MCA. Our experimental results demonstrate that the ESB strategy significantly surpasses the performance of existing baseline methods across multiple models. Moreover, the CRISPR-MCA model exhibits robust performance on two distinct types of datasets, affirming its effectiveness in enhancing the accuracy of deep learning predictions for off-target activities. [ABSTRACT FROM AUTHOR]

Performing data augmentation in medical named entity recognition (NER) is crucial due to the unique challenges posed by this field. Medical data is characterized by high acquisition costs, specialized terminology, imbalanced distributions, and limited training resources. These factors make achieving high performance in medical NER particularly difficult. Data augmentation methods help to mitigate these issues by generating additional training samples, thus balancing data distribution, enriching the training dataset, and improving model generalization. This paper proposes two data augmentation methods—Contextual Random Replacement based on Word2Vec Augmentation (CRR) and Targeted Entity Random Replacement Augmentation (TER)—aimed at addressing the scarcity and imbalance of data in the medical domain. When combined with a deep learning-based Chinese NER model, these methods can significantly enhance performance and recognition accuracy under limited resources. Experimental results demonstrate that both augmentation methods effectively improve the recognition capability of medical named entities. Specifically, the BERT-BiLSTM-CRF model achieved the highest F1 score of 83.587%, representing a 1.49% increase over the baseline model. This validates the importance and effectiveness of data augmentation in medical NER. [ABSTRACT FROM AUTHOR]

In ecology and evolutionary biology, the synthesis and modelling of data from published literature are commonly used to generate insights and test theories across systems. However, the tasks of searching, screening, and extracting data from literature are often arduous. Researchers may manually process hundreds to thousands of articles for systematic reviews, meta-analyses, and compiling synthetic datasets. As relevant articles expand to tens or hundreds of thousands, computer-based approaches can increase the efficiency, transparency and reproducibility of literature-based research. Methods available for text mining are rapidly changing owing to developments in machine learning-based language models. We review the growing landscape of approaches, mapping them onto three broad paradigms (frequency-based approaches, traditional Natural Language Processing and deep learning-based language models). This serves as an entry point to learn foundational and cutting-edge concepts, vocabularies, and methods to foster integration of these tools into ecological and evolutionary research. We cover approaches for modelling ecological texts, generating training data, developing custom models and interacting with large language models and discuss challenges and possible solutions to implementing these methods in ecology and evolution. [ABSTRACT FROM AUTHOR]

The growth of social networks has provided a platform for individuals with prejudiced views, allowing them to spread hate speech and target others based on their gender, ethnicity, religion, or sexual orientation. While positive interactions within diverse communities can considerably enhance confidence, it is critical to recognize that negative comments can hurt people's reputations and well-being. This emergence emphasizes the need for more diligent monitoring and robust policies on these platforms to protect individuals from such discriminatory and harmful behavior. Hate speech is often characterized as an intentional act of aggression directed at a specific group, typically meant to harm or marginalize them based on certain aspects of their identity. Most of the research related to hate speech has been conducted in resource-aware languages like English, Spanish, and French. However, low-resource European languages, such as Irish, Norwegian, Portuguese, Polish, Slovak, and many South Asian, present challenges due to limited linguistic resources, making information extraction labor-intensive. In this study, we present deep neural networks with FastText word embeddings using regularization methods for multi-class hate speech detection in the Norwegian language, along with the implementation of multilingual transformer-based models with hyperparameter tuning and generative configuration. FastText outperformed other deep learning models when stacked with Bidirectional LSTM and GRU, resulting in the FAST-RNN model. In the concluding phase, we compare our results with the state-of-the-art and perform interpretability modeling using Local Interpretable Model-Agnostic Explanations to achieve a more comprehensive understanding of the model's decision-making mechanisms. [ABSTRACT FROM AUTHOR]

Network-based intrusion detection (N-IDS) is an essential system inside an organization in a smart healthcare enterprise system to prevent the system and its networks from network attacks. A survey of the literature shows that in recent days deep learning approaches are employed successfully for N-IDS using network connections. However, finding the right features from a network connection is a daunting task. This work proposes a multidimensional attention-based deep learning approach for N-IDS that extracts the optimal features for intrusion detection using network payload. The proposed approach includes an embedding that transforms every word in the payload into a 100-dimensional feature vector representation and embedding follows deep learning layers such as a convolutional neural network (CNN) and long short-term memory (LSTM) with attention to extracting optimal features for attack classification. Next, the features of CNN and LSTM layers are concatenated and passed into fully connected layers for intrusion detection. The proposed approach showed 99% accuracy on the KISTI enterprise network payload dataset. In addition, the proposed approach showed 98% accuracy and 99% accuracy on network-based datasets such as KDDCup-99, CICIDS-2017, and WSN-DS and UNSW-NB15 respectively. The good experimental results on various network-based datasets suggest that the proposed N-IDS in smart healthcare enterprise systems is robust and generalizable to detect attacks from different network environments. The proposed approach performed better in all the experiments than the other deep learning-based methods. The model showed a 5% accuracy performance improvement compared to the existing study using the KISTI dataset. In addition, the proposed model has shown similar performances on the other intrusion datasets. The proposed approach serves as a network monitoring tool for efficient and accurate detection of attacks inside an organization on a healthcare enterprise network system. [ABSTRACT FROM AUTHOR]

Evidences are important proofs to support judicial trials. Automatically extracting evidences from judgement documents can be used to assess the trial quality and support "Intelligent Court". Current evidence extraction is primarily depended on sequence labelling models. Despite their success, they can only assign a label to a token, which is difficult to recognize nested evidence entities in judgment documents, where a token may belong to several evidences at the same time. In this paper, we present a novel evidence extraction architecture called ATT-MRC, in which extracting evidence entities is formalized as a question answer problem, where all evidence spans are screened out as possible correct answers. Furthermore, to address the data imbalance problem in the judgement documents, we revised the loss function and combined it with a data enhancement technique. Experimental results demonstrate that our model has better performance than related works in evidence extraction. [ABSTRACT FROM AUTHOR]

Human–robot walking with prosthetic legs and exoskeletons, especially over complex terrains, such as stairs, remains a significant challenge. Egocentric vision has the unique potential to detect the walking environment prior to physical interactions, which can improve transitions to and from stairs. This motivated us to develop the StairNet initiative to support the development of new deep learning models for visual perception of real-world stair environments. In this study, we present a comprehensive overview of the StairNet initiative and key research to date. First, we summarize the development of our large-scale data set with over 515,000 manually labeled images. We then provide a summary and detailed comparison of the performances achieved with different algorithms (i.e., 2D and 3D CNN, hybrid CNN and LSTM, and ViT networks), training methods (i.e., supervised learning with and without temporal data, and semi-supervised learning with unlabeled images), and deployment methods (i.e., mobile and embedded computing), using the StairNet data set. Finally, we discuss the challenges and future directions. To date, our StairNet models have consistently achieved high classification accuracy (i.e., up to 98.8%) with different designs, offering trade-offs between model accuracy and size. When deployed on mobile devices with GPU and NPU accelerators, our deep learning models achieved inference speeds up to 2.8 ms. In comparison, when deployed on our custom-designed CPU-powered smart glasses, our models yielded slower inference speeds of 1.5 s, presenting a trade-off between human-centered design and performance. Overall, the results of numerous experiments presented herein provide consistent evidence that StairNet can be an effective platform to develop and study new deep learning models for visual perception of human–robot walking environments, with an emphasis on stair recognition. This research aims to support the development of next-generation vision-based control systems for robotic prosthetic legs, exoskeletons, and other mobility assistive technologies. [ABSTRACT FROM AUTHOR]

In the world of technology, the electronic and technical development of the fields of communication and the internet has increased, which has caused a renaissance in the virtual world. This development has greatly impacted virtual communities for the ease and speed of communication and information transfer through social media platforms, making these platforms likable and easy to use. The social network faces major challenges due to its extensive use. As a result, many people have become involved in cybercrimes. There are accounts on the internet that are malicious. Platforms for social networking online, such as Facebook and Twitter, allow all users to freely generate and consume massive volumes of material regardless of their traits. While individuals and businesses utilize this information to gain a competitive edge, spam or phony users create important data. According to estimates, 1 in 200 posts on social media contain spam, and 1 in 21 tweets contain spam. The problem was centered around the accuracy of detecting false news and correcting it or preventing its dissemination before it spread in the network. A new method is given based on improving the false news detection system; the level of improvement was significant in the preprocessing stage where Glove is used, which is an unsupervised learning algorithm developed by researchers at Stanford University aiming to generate word embeddings by aggregating global word co-occurrence matrices from a given corpus. The basic idea behind the GloVe word embedding is to derive the relationship between the words from statistics. The proposed method contains deep learning algorithms of convolutional neural network (CNN), deep neural network (DNN), and long short-term memory (LSTM). The RNN with GloVe in the preprocessing stage using the Curpos fake news dataset to enhance the system, due to the sequential processes and classification, has the highest accuracy of 98.974%. [ABSTRACT FROM AUTHOR]

To perform semantic image segmentation using deep learning models, a significant quantity of data and meticulous manual annotation is necessary (Mani in: Research anthology on improving medical imaging techniques for analysis and intervention. IGI Global, pp. 107–125, 2023), and the process consumes a lot of resources, including time and money. To resolve such issues, we introduce a unique label propagation method (Qin et al. in IEEE/CAA J Autom Sinica 10(5):1192–1208, 2023) that utilizes cycle consistency across time to propagate labels over longer time horizons with higher accuracy. Additionally, we acknowledge that dense pixel annotation is a noisy process (Das et al. in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 5978–5987, 2023), whether performed manually or automatically. To address this, we present a principled approach that accounts for label uncertainty when training with labels from multiple noisy labeling processes. We introduce two new approaches; Warp-Refine Propagation and Uncertainty-Aware Training, for improving label propagation and handling noisy labels, respectively, and support the process with quantitative and qualitative evaluations and theoretical justification. Our contributions are validated on the Cityscapes and ApolloScape datasets, where we achieve encouraging results. In later endeavors, the aim should be to expand such approaches to include other noisy augmentation processes like image-based rendering methods (Laraqui et al. in Int J Comput Aid Eng Technol 18(5):141–151, 2023), thanks to the noisy label learning approach. [ABSTRACT FROM AUTHOR]

Over recent decades, machine learning, an integral subfield of artificial intelligence, has revolutionized diverse sectors, enabling data-driven decisions with minimal human intervention. In particular, the field of educational assessment emerges as a promising area for machine learning applications, where students can be classified and diagnosed using their performance data. The objectives of Diagnostic Classification Models (DCMs), which provide a suite of methods for diagnosing students' cognitive states in relation to the mastery of necessary cognitive attributes for solving problems in a test, can be effectively addressed through machine learning techniques. However, the challenge lies in the latent nature of cognitive status, which makes it difficult to obtain labels for the training dataset. Consequently, the application of machine learning methods to DCMs often assumes smaller training sets with labels derived either from theoretical considerations or human experts. In this study, the authors propose a supervised diagnostic classification model with data augmentation (SDCM-DA). This method is designed to utilize the augmented data using a data generation model constructed by leveraging the probability of correct responses for each attribute mastery pattern derived from the expert-labeled dataset. To explore the benefits of data augmentation, a simulation study is carried out, contrasting it with classification methods that rely solely on the expert-labeled dataset for training. The findings reveal that utilizing data augmentation with the estimated probabilities of correct responses substantially enhances classification accuracy. This holds true even when the augmentation originates from a small labeled sample with occasional labeling errors, and when the tests contain lower-quality items that may inaccurately measure students' true cognitive status. Moreover, the study demonstrates that leveraging augmented data for learning can enable the successful classification of students, thereby eliminating the necessity for specifying an underlying response model. [ABSTRACT FROM AUTHOR]

Objectives: Despite nearly universal prenatal ultrasound screening programs, congenital heart defects (CHD) are still missed, which may result in severe morbidity or even death. Deep machine learning (DL) can automate image recognition from ultrasound. The main aim of this study was to assess the performance of a previously developed DL model, trained on images from a tertiary center, using fetal ultrasound images obtained during the second‐trimester standard anomaly scan in a low‐risk population. A secondary aim was to compare initial screening diagnosis, which made use of live imaging at the point‐of‐care, with diagnosis by clinicians evaluating only stored images. Methods: All pregnancies with isolated severe CHD in the Northwestern region of The Netherlands between 2015 and 2016 with available stored images were evaluated, as well as a sample of normal fetuses' examinations from the same region and time period. We compared the accuracy of the initial clinical diagnosis (made in real time with access to live imaging) with that of the model (which had only stored imaging available) and with the performance of three blinded human experts who had access only to the stored images (like the model). We analyzed performance according to ultrasound study characteristics, such as duration and quality (scored independently by investigators), number of stored images and availability of screening views. Results: A total of 42 normal fetuses and 66 cases of isolated CHD at birth were analyzed. Of the abnormal cases, 31 were missed and 35 were detected at the time of the clinical anatomy scan (sensitivity, 53%). Model sensitivity and specificity were 91% and 78%, respectively. Blinded human experts (n = 3) achieved mean ± SD sensitivity and specificity of 55 ± 10% (range, 47–67%) and 71 ± 13% (range, 57–83%), respectively. There was a statistically significant difference in model correctness according to expert‐graded image quality (P = 0.03). The abnormal cases included 19 lesions that the model had not encountered during its training; the model's performance in these cases (16/19 correct) was not statistically significantly different from that for previously encountered lesions (P = 0.41). Conclusions: A previously trained DL algorithm had higher sensitivity than initial clinical assessment in detecting CHD in a cohort in which over 50% of CHD cases were initially missed clinically. Notably, the DL algorithm performed well on community‐acquired images in a low‐risk population, including lesions to which it had not been exposed previously. Furthermore, when both the model and blinded human experts had access to only stored images and not the full range of images available to a clinician during a live scan, the model outperformed the human experts. Together, these findings support the proposition that use of DL models can improve prenatal detection of CHD. © 2023 International Society of Ultrasound in Obstetrics and Gynecology. [ABSTRACT FROM AUTHOR]

Transformers have demonstrated outstanding performance in many applications of deep learning. When applied to time series data, transformers require effective position encoding to capture the ordering of the time series data. The efficacy of position encoding in time series analysis is not well-studied and remains controversial, e.g., whether it is better to inject absolute position encoding or relative position encoding, or a combination of them. In order to clarify this, we first review existing absolute and relative position encoding methods when applied in time series classification. We then proposed a new absolute position encoding method dedicated to time series data called time Absolute Position Encoding (tAPE). Our new method incorporates the series length and input embedding dimension in absolute position encoding. Additionally, we propose computationally Efficient implementation of Relative Position Encoding (eRPE) to improve generalisability for time series. We then propose a novel multivariate time series classification model combining tAPE/eRPE and convolution-based input encoding named ConvTran to improve the position and data embedding of time series data. The proposed absolute and relative position encoding methods are simple and efficient. They can be easily integrated into transformer blocks and used for downstream tasks such as forecasting, extrinsic regression, and anomaly detection. Extensive experiments on 32 multivariate time-series datasets show that our model is significantly more accurate than state-of-the-art convolution and transformer-based models. Code and models are open-sourced at https://github.com/Navidfoumani/ConvTran. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive review of deep learning applications in the video game industry, focusing on how these techniques can be utilized in game development, experience, and operation. As relying on computation techniques, the game world can be viewed as an integration of various complex data. This examines the use of deep learning in processing various types of game data. The paper classifies the game data into asset data, interaction data, and player data, according to their utilization in game development, experience, and operation, respectively. Specifically, this paper discusses deep learning applications in generating asset data such as object images, 3D scenes, avatar models, and facial animations; enhancing interaction data through improved text-based conversations and decision-making behaviors; and analyzing player data for cheat detection and match-making purposes. Although this review may not cover all existing applications of deep learning, it aims to provide a thorough presentation of the current state of deep learning in the gaming industry and its potential to revolutionize game production by reducing costs and improving the overall player experience. [ABSTRACT FROM AUTHOR]

Automatic text generation is the generation of natural language text by machines. Enabling machines to generate readable and coherent text is one of the most vital yet challenging tasks. Traditionally, text generation has been implemented either by using production rules of a predefined grammar or performing statistical analysis of existing human-written texts to predict sequences of words. Recently a paradigm change has emerged in text generation, induced by technological advancements, including deep learning methods and pre-trained transformers. However, many open challenges in text generation need to be addressed, including the generation of fluent, coherent, diverse, controllable, and consistent human-like text. This survey aims to provide a comprehensive overview of current advancements in automated text generation and introduce the topic to researchers by offering pointers and synthesis to pertinent studies. This paper studied the relevant twelve years of articles from 2011 onwards in the field of text generation and observed a total of 146 prime studies relevant to the objective of this survey that has been thoroughly reviewed and discussed. It covers core text generation applications, including text summarization, question–answer generation, story generation, machine translation, dialogue response generation, paraphrase generation, and image/video captioning. The most commonly used datasets for text generation and existing tools with their application domain have also been mentioned. Various text decoding and optimization methods have been provided with their strengths and weaknesses. For evaluating the effectiveness of the generated text, automatic evaluation metrices have been discussed. Finally, the article discusses the main challenges and notable future directions in the field of automated text generation for potential researchers. [ABSTRACT FROM AUTHOR]

Reading comprehension involves the process of reading and understanding textual information in order to answer questions related to it. It finds practical applications in various domains such as domain-specific FAQs, search engines, and dialog systems. Resource-rich languages like English, Japanese, Chinese, and most European languages benefit from the availability of numerous datasets and resources, enabling the development of machine reading comprehension (MRC) systems. However, building MRC systems for low-resource languages (LRL) with limited datasets, such as Vietnamese, Urdu, Bengali, and Hindi, poses significant challenges. To address this issue, this study utilizes quantitative analysis to conduct a systematic literature review (SLR) with the aim of comprehending the recent global shift in MRC research from high-resource languages (HRL) to low-resource languages. Notably, existing literature reviews on MRC lack comprehensive studies that compare techniques specifically designed for rich and low-resource languages. Hence, this study provides a comprehensive overview of the MRC research landscape in low-resource languages, offering valuable insights and a list of suggestions to enhance LRL–MRC research. [ABSTRACT FROM AUTHOR]

Unloading activities directly impact the progress of earthmoving and filling projects, and a reliable multistep-ahead unloading duration prediction could help optimize equipment scheduling and improve operational efficiency. However, unloading prediction is greatly challenging owing to the complex uncertainties and nonlinearities implied in the unloading process, as well as the difficulty of modeling long-term temporal dependencies. Thus, this study devises a new fuzzy sequence-to-sequence network for unloading time forecasting. First, a quantization error-improved information granulation method is exploited to establish the fuzzy partition function. The global and localized distribution characteristics of unloading time are utilized to adaptively optimize the number and distribution of nonuniform fuzzy intervals. Then, periodic and recent branches were developed to model the variation of unloading time in multiple temporal granularities. In each branch, based on the encoder–decoder structure, the underlying gated recurrent units learn the sequence features collaborating with the attention mechanism to capture implicit long-term dependencies, mitigating the effects of error accumulation in multistep forecasting. Finally, the temporal information of different granularities is fused to form the final prediction. We evaluate the proposed model using an unloading dataset from a heavy infrastructure project in southwest China. We conducted geo-fencing and unloading operation analysis to extract the unloading information of different construction areas. The experimental results show that our method can generate high-quality multistep predictions for unloading duration, and exhibits superior performance compared with baseline models. The novel approach has great potential to support earthwork management in complex environments. [ABSTRACT FROM AUTHOR]

In the recent improvement in deep learning approaches for realistic image generation and translation, Generative Adversarial Networks (GANs) delivered favorable results. GAN generates novel samples that look indistinguishable from authentic images. This paper proposes a novel generative network for thermal-to-visible image translation. Thermal to Visible synthesis is challenging due to the non-availability of accurate semantic and textural information in thermal images. The thermal sensors acquire the thermal face images by capturing the object's luminance with fewer details about the actual facial information. However, it is advantageous for low-light and night-time vision, where image information cannot be captured in a complex environment by an RGB camera. We design a new Attention-guided Cyclic Generative Adversarial Network for Thermal to Visible Face transformation (TVA-GAN) by integrating a new attention network. We utilize attention guidance with a recurrent block with an Inception module to simplify the learning space toward the optimum solution. The proposed TVA-GAN is trained and evaluated for thermal to visible face synthesis over three benchmark datasets, including the WHU-IIP, Tufts Face Thermal2RGB, and CVBL-CHILD datasets. The proposed TVA-GAN results show promising improvement in face synthesis compared to the state-of-the-art GAN methods. For the proposed TVA-GAN, code is available at: https://github.com/GANGREEK/TVA-GAN. [ABSTRACT FROM AUTHOR]

Survival analysis has become one of the paramount procedures in the modeling of time-to-event data. When we encounter complex survival problems, the traditional approach remains limited in accounting for the complex correlational structure between the covariates and the outcome due to the strong assumptions that limit the inference and prediction ability of the resulting models. Several studies exist on the deep learning approach to survival modeling; moreover, the application for the case of complex survival problems still needs to be improved. In addition, the existing models need to address the data structure's complexity fully and are subject to noise and redundant information. In this study, we design a deep learning technique (CmpXRnnSurv_AE) that obliterates the limitations imposed by traditional approaches and addresses the above issues to jointly predict the risk-specific probabilities and survival function for recurrent events with competing risks. We introduce the component termed Risks Information Weights (RIW) as an attention mechanism to compute the weighted cumulative incidence function (WCIF) and an external auto-encoder (ExternalAE) as a feature selector to extract complex characteristics among the set of covariates responsible for the cause-specific events. We train our model using synthetic and real data sets and employ the appropriate metrics for complex survival models for evaluation. As benchmarks, we selected both traditional, and machine learning models and our model demonstrates better performance across all datasets. [ABSTRACT FROM AUTHOR]

Relation extraction is an important topic in information extraction, as it is used to create large-scale knowledge graphs for a variety of downstream applications. Its goal is to find and extract semantic links between entity pairs in natural language sentences. Deep learning has substantially advanced neural relation extraction, allowing for the autonomous learning of semantic features. We offer an effective Chinese relation extraction model that uses bidirectional LSTM (Bi-LSTM) and an attention mechanism to extract crucial semantic information from phrases without relying on domain knowledge from lexical resources or language systems in this study. The attention mechanism included into the Bi-LSTM network allows for automatic focus on key words. Two benchmark datasets were used to create and test our models: Chinese SanWen and FinRE. The experimental results show that the SanWen dataset model outperforms the FinRE dataset model, with area under the receiver operating characteristic curve values of 0.70 and 0.50, respectively. The models trained on the SanWen and FinRE datasets achieve values of 0.44 and 0.19, respectively, for the area under the precision-recall curve. In addition, the results of repeated modeling experiments indicated that our proposed method was robust and reproducible. [ABSTRACT FROM AUTHOR]

The development of Neural Machine Translation (NMT) systems has attained prominent position in language translation tasks. However, it faces huge challenges in translating the new words and out-of-vocabularies. This problem is identified as a major drawback of conventional NMT systems in language translation results more copied outputs. In addition to that, it places the risks in understanding multilingual language structures and word relationships. In this paper, we propose novel deep stacked GRU algorithm based NMT System to address the aforementioned challenges and handles multilingual sentences based translation tasks efficiently. We aimed to develop the proposed model for translating the spoken sentences into sign words. The generated sign words (glosses) are mapped with sign gesture images to automate the sign gesture video generation process using deep generative models. The proposed hybrid NMT model has been evaluated qualitatively and quantitatively using different benchmark sign language datasets. The improved BLEU Score shows the outperformance of our model compared with earlier approaches. We also evaluated the proposed model using our self created Indian sign language corpus (ISL-CSLTR). The final result shows the achievement of greater translation results with minimal processing cost. [ABSTRACT FROM AUTHOR]

Phishing, in which social engineering techniques such as emails and instant messaging are employed and malicious links are disguised as normal URLs to steal sensitive information, is currently a major threat to networks worldwide. Phishing detection systems generally adopt feature engineering as one of the most important approaches to detect or even prevent phishing attacks. However, the accuracy of feature engineering systems is heavily dependent on the prior knowledge of features. In addition, extracting comprehensive features from different dimensions for high detection accuracy is time-consuming. To address these issues, this paper proposes a lightweight model that combines convolutional neural network (CNN), bi-directional long short-term memory (BiLSTM), and the attention mechanism for phishing detection. The proposed model, called the char-convolutional and BiLSTM with attention mechanism (CCBLA) model, employs deep learning to automatically extract features from target URLs and uses the attention mechanism to weight the importance of the selected features under different roles during phishing detection. The results of experiments conducted on two datasets with different scales show that CCBLA is accurate in phishing attack detection with minimal time consumption. [ABSTRACT FROM AUTHOR]

Objective: To develop a deep learning algorithm for anatomy recognition in thoracoscopic video frames from robot-assisted minimally invasive esophagectomy (RAMIE) procedures using deep learning. Background: RAMIE is a complex operation with substantial perioperative morbidity and a considerable learning curve. Automatic anatomy recognition may improve surgical orientation and recognition of anatomical structures and might contribute to reducing morbidity or learning curves. Studies regarding anatomy recognition in complex surgical procedures are currently lacking. Methods: Eighty-three videos of consecutive RAMIE procedures between 2018 and 2022 were retrospectively collected at University Medical Center Utrecht. A surgical PhD candidate and an expert surgeon annotated the azygos vein and vena cava, aorta, and right lung on 1050 thoracoscopic frames. 850 frames were used for training of a convolutional neural network (CNN) to segment the anatomical structures. The remaining 200 frames of the dataset were used for testing the CNN. The Dice and 95% Hausdorff distance (95HD) were calculated to assess algorithm accuracy. Results: The median Dice of the algorithm was 0.79 (IQR = 0.20) for segmentation of the azygos vein and/or vena cava. A median Dice coefficient of 0.74 (IQR = 0.86) and 0.89 (IQR = 0.30) were obtained for segmentation of the aorta and lung, respectively. Inference time was 0.026 s (39 Hz). The prediction of the deep learning algorithm was compared with the expert surgeon annotations, showing an accuracy measured in median Dice of 0.70 (IQR = 0.19), 0.88 (IQR = 0.07), and 0.90 (0.10) for the vena cava and/or azygos vein, aorta, and lung, respectively. Conclusion: This study shows that deep learning-based semantic segmentation has potential for anatomy recognition in RAMIE video frames. The inference time of the algorithm facilitated real-time anatomy recognition. Clinical applicability should be assessed in prospective clinical studies. [ABSTRACT FROM AUTHOR]

Spectral efficiency (SE) optimization in massive multiple input multiple output (MIMO) antenna cognitive systems is a challenge originated from the coexistence restrictions. Traditional power allocation can optimize the SE; however, involving deep learning can meet real-time and fairness processing requirements. In unfair allocation problem, all power is possibly assigned to one or few antennas of a particular user. In this paper, we build a mathematical optimization model considering the fairness problem such that SE is optimized for all users. To implement the model, we propose an attention-based convolutional neural network (Att-CNN), where h 0 and h k (i.e., cross-interference and direct channels) attention mechanisms are used to improve the SE. The convolutional neural network is applied to decrease the floating point operations (FLOPs) and number of network parameters. We conducted experiments from these aspects: Fair antenna power allocation, power allocation performance and computational performance. Heat maps with different interference thresholds show the fair allocation for all users. Analyses of SE validate the superiority of the Att-CNN compared with the equal power allocation and fully connected neural network (FNN) schemes. The analyses of the FLOPs and number of parameters show the superiority of the Att-CNN over the FNN. [ABSTRACT FROM AUTHOR]

EEG-based emotion recognition is not only an important branch in the field of affective computing, but is also an indispensable task for harmonious human–computer interaction. Recently, many deep learning emotion recognition algorithms have achieved good results, but most of them have been based on convolutional and recurrent neural networks, resulting in complex model design, poor modeling of long-distance dependency, and the inability to parallelize computations. Here, we proposed a novel bi-hemispheric asymmetric attention network (Bi-AAN) combining a transformer structure with the asymmetric property of the brain's emotional response. In this way, we modeled the difference of bi-hemispheric attention, and mined the long-term dependency between EEG sequences, which exacts more discriminative emotional representations. First, the differential entropy (DE) features of each frequency band were calculated using the DE-embedding block, and the spatial information between the electrode positions was extracted using positional encoding. Then, a bi-headed attention mechanism was employed to capture the intra-attention of frequency bands in each hemisphere and the attentional differences between the bi-hemispheric frequency bands. After carring out experiments both in DEAP and DREAMER datasets, we found that the proposed Bi-AAN achieved superior recognition performance as compared to state-of-the-art EEG emotion recognition methods. [ABSTRACT FROM AUTHOR]

Recently, with the rapid developments of the Internet and social networks, there have been tremendous increase in the amount of complex-structured text resources. These information explosions require extensive studies as well as more advanced methods in order to better understand and effectively model/learn these high-dimensional/structure-complicated textual datasets. Moving along with the recent progresses in deep learning and textual representation learning approaches, many researchers in this domain have been attracted by utilizing different deep neural architectures for learning essential features from texts. These novel neural architectures must enable to handle complex textual feature engineering. Moreover, it also has to be able to extract deeper semantic and structural information from textual resources. Recently, there are several integrations between advanced deep learning architectures, such as recurrent neural networks (RNNs), sequence-to-sequence (seq2seq) and transformers in text classification have been proposed. These hybrid deep neural architectures have shed light on how computers can comprehensively process sequential information from texts to fine-tune for leveraging the performance of multiple tasks in natural language processing, including classification. However, most of recent RNN-based techniques still suffer from several limitations. These limitations are mainly related to the capability of capturing the global long-range dependent as well syntactical structures of the given text corpus. There are some recent studies have shown that a combination of graph-based text representation and graph neural network (GNN) approaches can cope with these challenges. In this survey works, we mainly focus on discussing about recent state-of-the-art studies which are mainly dedicated on the text graph representation learning through GNN, named as TG-GNN. In addition, beside the TG-GNN based models' features and capability discussions, we also mentioned about the pros/cons. Extensive comparative studies of TG-GNN based techniques in benchmark datasets for text classification problem are also provided in this survey. Finally, we highlight existing challenges as well as identify perspectives which might be useful for future improvements in this research direction. [ABSTRACT FROM AUTHOR]

Inferring the topology of a network from network dynamics is a significant problem with both theoretical research significance and practical value. This paper considers how to reconstruct the network topology according to the continuous-time data on the network. Inspired by the generative adversarial network(GAN), we design a deep learning framework based on network continuous-time data. The framework predicts the edge connection probability between network nodes by learning the correlation between network node state vectors. To verify the accuracy and adaptability of our method, we conducted extensive experiments on scale-free networks and small-world networks at different network scales using three different dynamics: heat diffusion dynamics, mutualistic interaction dynamics, and gene regulation dynamics. Experimental results show that our method significantly outperforms the other five traditional correlation indices, which demonstrates that our method can reconstruct the topology of different scale networks well under different network dynamics. [ABSTRACT FROM AUTHOR]

Deep learning technology has been used in the medical field to produce devices for clinical practice. Deep learning methods in cytology offer the potential to enhance cancer screening while also providing quantitative, objective, and highly reproducible testing. However, constructing high-accuracy deep learning models necessitates a significant amount of manually labeled data, which takes time. To address this issue, we used the Noisy Student Training technique to create a binary classification deep learning model for cervical cytology screening, which reduces the quantity of labeled data necessary. We used 140 whole-slide images from liquid-based cytology specimens, 50 of which were low-grade squamous intraepithelial lesions, 50 were high-grade squamous intraepithelial lesions, and 40 were negative samples. We extracted 56,996 images from the slides and then used them to train and test the model. We trained the EfficientNet using 2,600 manually labeled images to generate additional pseudo labels for the unlabeled data and then self-trained it within a student-teacher framework. Based on the presence or absence of abnormal cells, the created model was used to classify the images as normal or abnormal. The Grad-CAM approach was used to visualize the image components that contributed to the classification. The model achieved an area under the curve of 0.908, accuracy of 0.873, and F1-score of 0.833 with our test data. We also explored the optimal confidence threshold score and optimal augmentation approaches for low-magnification images. Our model efficiently classified normal and abnormal images at low magnification with high reliability, making it a promising screening tool for cervical cytology. [ABSTRACT FROM AUTHOR]

Since the COVID-19 pandemic, healthcare services, particularly remote and automated healthcare consultations, have gained increased attention. Medical bots, which provide medical advice and support, are becoming increasingly popular. They offer numerous benefits, including 24/7 access to medical counseling, reduced appointment wait times by providing quick answers to common questions or concerns, and cost savings associated with fewer visits or tests required for diagnosis and treatment plans. The success of medical bots depends on the quality of their learning, which in turn depends on the appropriate corpus within the domain of interest. Arabic is one of the most commonly used languages for sharing users' internet content. However, implementing medical bots in Arabic faces several challenges, including the language's morphological composition, the diversity of dialects, and the need for an appropriate and large enough corpus in the medical domain. To address this gap, this paper introduces the largest Arabic Healthcare Q &A dataset, called MAQA, consisting of over 430,000 questions distributed across 20 medical specializations. Furthermore, this paper adopts three deep learning models, namely LSTM, Bi-LSTM, and Transformers, for experimenting and benchmarking the proposed corpus MAQA. The experimental results demonstrate that the recent Transformer model outperforms the traditional deep learning models, achieving an average cosine similarity of 80.81% and a BLeU score of 58%. [ABSTRACT FROM AUTHOR]

Emotion analysis of social media data is essential for understanding a person's mindset and judgment on a particular event. The emotion detection framework needs sophisticated deep learning models such as Long short-term memory networks and convolutional neural networks (CNN) for their development. These models did not give adequate emotional information in targeted emotion analysis. This paper proposes a new deep learning-based emotion detection framework using an optimized, stacked, bidirectional Gated Recurrent Unit with sematic attention (SRBi-GRU-SA). A multichannel word embedding model is proposed for representing the text document by combining the emotional information with the semantic information obtained from a natural language model (BERT), Term-frequency inverse gravity moment (TF-IGM) based term weighted word embedding model with trigrams. The proposed SRBi-GRU-SA includes semantic attention (SA) model for highlighting the most significant features based on attention score. Also, an Arithmetic Optimization Algorithm (AOA) is introduced to optimize the weight parameters of the SRBi-GRU-SA model. The evaluation results of the proposed optimized SRBi-GRU-SA model prove its effectiveness against the existing models by increasing the F1 score to 84.51%, 85.93%, 85.25%, and 87.12% on International Survey on Emotion Antecedents and Reactions (ISEAR), Stance Sentiment Emotion Corpus (SSEC), Daily dialogs, and Grounded-Emotions datasets respectively [ABSTRACT FROM AUTHOR]

Deep reinforcement learning (DRL) is poised to revolutionise the field of artificial intelligence (AI) by endowing autonomous systems with high levels of understanding of the real world. Currently, deep learning (DL) is enabling DRL to effectively solve various intractable problems in various fields including computer vision, natural language processing, healthcare, robotics, to name a few. Most importantly, DRL algorithms are also being employed in audio signal processing to learn directly from speech, music and other sound signals in order to create audio-based autonomous systems that have many promising applications in the real world. In this article, we conduct a comprehensive survey on the progress of DRL in the audio domain by bringing together research studies across different but related areas in speech and music. We begin with an introduction to the general field of DL and reinforcement learning (RL), then progress to the main DRL methods and their applications in the audio domain. We conclude by presenting important challenges faced by audio-based DRL agents and by highlighting open areas for future research and investigation. The findings of this paper will guide researchers interested in DRL for the audio domain. [ABSTRACT FROM AUTHOR]

Stance detection is an evolving opinion mining research area motivated by the vast increase in the variety and volume of user-generated content. In this regard, considerable research has been recently carried out in the area of stance detection. In this study, we review the different techniques proposed in the literature for stance detection as well as other applications such as rumor veracity detection. Particularly, we conducted a systematic literature review of empirical research on the machine learning (ML) models for stance detection that were published from January 2015 to October 2022. We analyzed 96 primary studies, which spanned eight categories of ML techniques. In this paper, we categorize the analyzed studies according to a taxonomy of six dimensions: approaches, target dependency, applications, modeling, language, and resources. We further classify and analyze the corresponding techniques from each dimension's perspective and highlight their strengths and weaknesses. The analysis reveals that deep learning models that adopt a mechanism of self-attention have been used more frequently than the other approaches. It is worth noting that emerging ML techniques such as few-shot learning and multitask learning have been used extensively for stance detection. A major conclusion of our analysis is that despite that ML models have shown to be promising in this field, the application of these models in the real world is still limited. Our analysis lists challenges and gaps to be addressed in future research. Furthermore, the taxonomy presented can assist researchers in developing and positioning new techniques for stance detection-related applications. [ABSTRACT FROM AUTHOR]

The rapid development of data and artificial intelligence technology has introduced new opportunities and challenges to aeronautical information intelligence. However, there are many obstacles in the sharing, reasoning and reusing aeronautical data due to the disunity of norms, the opacity of sharing and semantic ambiguity. To a large extent, as a basic method for processing, storing and deducing aeronautical data in the future, NER provides a new idea for the natural language processing of aeronautical information intelligence. In this paper, the problem with NER for aeronautical information is deeply analyzed, the relationship among the data model, the knowledge system and the named entity (NE) is combed, and the main characteristics of NE are summarized. At the same time, the resources that are useful to NER involving thematic databases, aviation domain ontology and evaluation indicators are described. Finally, two main directions of NER are suggested for further research, which is helpful in aviation development. This paper first provides a comprehensive survey of the approaches and directions of NER in a specific domain: aeronautical intelligence information. [ABSTRACT FROM AUTHOR]

In Chinese classics, the sentiment attitudes or thoughts of the ancients regarding specific environments, people, and events were generally expressed in the form of poetry. Compared with previous attempts to classify the polarity of poetry, sentiment terms can be used to detect more fine-grained humanity knowledge in literary information resources. However, the existing techniques of domain sentiment lexicon construction fail to take full advantage of deep learning and linguistic knowledge, which cannot ensure the term integrity and accuracy. To this end, this work proposes a novel approach for the construction of a sentiment lexicon via the combination of supervised sentiment term extraction and classification, aiming at incorporating multi-dimensional linguistic knowledge into a two-phase deep learning model. A character-sequence labeling model for term extraction is first constructed by fusing the emotion radical features of Chinese characters, and term embedding augmentation via word knowledge is then carried out to classify the extracted terms. Experiments on Chinese poetry and its appreciation texts validate the superiority of the proposed method, and the model incorporating linguistic knowledge is found to outperform the benchmark models in different metrics. A fine-grained sentiment lexicon with two first classes, five-second classes, 15 third classes, and 14,368 domain terms and unregistered terms is constructed via hierarchical term classification, thereby contributing to the advancement of the interpretability of the humanities computing of classical Chinese poetry. [ABSTRACT FROM AUTHOR]

Transformer-based language models are successfully used to address massive text-related tasks. DNA methylation is an important epigenetic mechanism, and its analysis provides valuable insights into gene regulation and biomarker identification. Several deep learning–based methods have been proposed to identify DNA methylation, and each seeks to strike a balance between computational effort and accuracy. Here, we introduce MuLan-Methyl, a deep learning framework for predicting DNA methylation sites, which is based on 5 popular transformer-based language models. The framework identifies methylation sites for 3 different types of DNA methylation: N6-adenine, N4-cytosine, and 5-hydroxymethylcytosine. Each of the employed language models is adapted to the task using the "pretrain and fine-tune" paradigm. Pretraining is performed on a custom corpus of DNA fragments and taxonomy lineages using self-supervised learning. Fine-tuning aims at predicting the DNA methylation status of each type. The 5 models are used to collectively predict the DNA methylation status. We report excellent performance of MuLan-Methyl on a benchmark dataset. Moreover, we argue that the model captures characteristic differences between different species that are relevant for methylation. This work demonstrates that language models can be successfully adapted to applications in biological sequence analysis and that joint utilization of different language models improves model performance. Mulan-Methyl is open source, and we provide a web server that implements the approach. [ABSTRACT FROM AUTHOR]

Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by “engineered” methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology. [ABSTRACT FROM AUTHOR]

While the recent emergence of a large number of online course resources has made life more convenient for many people, it has also caused information overload. According to a user's situation and behavior, course recommendation systems can recommend courses of interest to the user, so that the user can quickly sift through a massive amount of information to find courses that meet his or her needs. This paper provide a systematic review of deep learning-based recommendation systems in e-learning environments. Firstly, the concept of recommendation systems is introduced in e-learning environments, and present a comprehensive survey and classification of deep learning techniques for course recommendation. And then, a detailed analysis of existing recommendation system is conducted based on the collected literature, and an overall course recommendation system framework is presented. Subsequently, this artical main focus is on multilayer perceptual machines, recurrent neural networks, convolutional neural networks, neural attention mechanisms, and deep reinforcement learning-based recommendation, and summarize the existing research on the use of the five techniques mentioned above in e-learning environments. The last section discusses seven flaws in the current recommendation systems used in e-learning environments and identify opportunities for future research. [ABSTRACT FROM AUTHOR]

BERT, a pre-trained language model on the large-scale corpus, has made breakthrough progress in NLP tasks. However, the experimental data shows that the BERT model's application effect in Chinese tasks is not ideal. The reason is that we believe that only character-level embedding can be obtained through BERT. However, a single Chinese character often cannot express their comprehensive meaning. To improve the model's ability to understand phrase-level semantic information, this paper proposes an enhanced BERT based on the average pooling(AP-BERT). Our model uses an average pooling layer to act on token embedding and reconstructs the model's input embedding, which can effectively improve BERT's application effect in Chinese natural language processing. Experimental data show that our proposed method has been enhanced in the four tasks of Chinese text classification, named entity recognition, reading comprehension, and summary generation. This method can not only improve the application effect of the BERT model in Chinese tasks but also can be well applied to other pre-trained language models. [ABSTRACT FROM AUTHOR]

Due to its powerful learning capabilities for high-dimensional and complex data, deep semi-supervised clustering algorithms often outperform traditional semi-supervised clustering methods. However, most deep semi-supervised clustering methods cannot fully utilize prior knowledge and unlabeled data. Deep semi-supervised classification algorithms have recently made significant progress in using unlabeled data during training by combining a consistency regularization method. Consistency training encourages network predictions to remain consistent when the input is perturbed. Motivated by the success of consistency regularization methods, we proposed a new semi-supervised clustering framework based on Siamese networks. To leverage the additional structure of unlabeled data and to uncover more information hidden by pairwise constraints, we add a consistency regularization loss, calculated on unlabeled data and pairwise constraints, to our objective function. After consistency training, the connected data can be closer in the learned feature space, while the disconnected data can be far away. To verify the effectiveness of the proposed method, we conducted extensive experiments on several real-world data sets. Experimental results show that the proposed method is more effective than other state-of-the-art methods in clustering performance. [ABSTRACT FROM AUTHOR]

With significant advances in vision and natural language processing, the generation of image captions becomes a need. Mathews, Xie, and He extended a new model to generate styled captions by separating semantics and style. In continuation of their work, here, a new captioning model is developed, including an image encoder to extract the features, a mixture of recurrent networks to embed the set of extracted features to a group of words, and a sentence generator that combines the obtained words as a stylized sentence. This Mixture of Recurrent Experts (MoRE) system uses a new training algorithm that derives singular value decomposition from weighting matrices of Recurrent Neural Networks (RNNs) to increase the diversity of captions. Each decomposition step depends on a distinctive factor based on the number of RNNs in MoRE. The used sentence generator gives a stylized language corpus without paired images. Besides, the styled and diverse captions are extracted without training on a densely labeled or styled dataset. MoRE on the COCO dataset generated diverse and stylized image captions without the necessity of extra‐labeling and improved descriptions in terms of content accuracy. [ABSTRACT FROM AUTHOR]

Deep Learning in the field of Big Data has become essential for the analysis and perception of trends. Activation functions play a crucial role in the outcome of these deep learning frameworks. The existing activation functions are hugely focused on data translation from one neural layer to another. Although they have been proven useful and have given consistent results, they are static and mostly non-parametric. In this paper, we propose a new function for modified training of neural networks that is more flexible and adaptable to the data. The proposed catalysis function works over Rectified Linear Unit (ReLU), sigmoid, tanh and all other activation functions to provide adaptive feed-forward training. The function uses vector components of the activation function to provide variational flow of input. The performance of this algorithm is tested on Modified National Institute of Standards and Technology (MNIST) and Canadian Institute for Advanced Research (CIFAR10) datasets against the conventional activation functions. Visual Geometry Group (VGG) blocks and Residual Neural Network (ResNet) architectures are used for experimentation. The proposed function has shown significant improvements in comparison to the traditional functions with a 75 ± 2.5% acuuracy across activation functions. The adaptive nature of training has drastically decreased the probability of under-fitting. The parameterization has helped increase the data learning capacity of models. On performing sensitivity analysis, the catalysis activation show slight or no changes on varying initialization parameters. [ABSTRACT FROM AUTHOR]

In metal cutting processing, prediction of the tool wear or VB value can help early warning and timely tool replacement before the tool reaches the service life. Although the deep neural network is an effective method to predict the tool wear, the existing research predicts the tool wear only at its next time moment without considering the tool states at different time points. It ignores important information of the tool wear. In this paper, we propose a comprehensive model, which consists of a monitoring module and a prediction module, to monitor and predict the tool wear for the first time. In the monitoring module, a DenseNet model is constructed to monitor the tool wear via sensor signals. In addition, the prediction module based on attention mechanism is developed by simulating the human brain attention to selectively focus on the important part of processing sequence information. An encoder-decoder structure is introduced for a multi-step prediction of the tool wear. Several VB values in the nearest future are predicted by using sequential VB values monitored in the latest past. Experimental studies show that the short-term information has more influence on the tool wear prediction than the long-term information. The proposed method has been used to predict multi-step VB values in milling operations. [ABSTRACT FROM AUTHOR]

Purpose: Deep neural networks (DNNs) have been widely applied in medical image classification, benefiting from its powerful mapping capability among medical images. However, these existing deep learning‐based methods depend on an enormous amount of carefully labeled images. Meanwhile, noise is inevitably introduced in the labeling process, degrading the performance of models. Hence, it is significant to devise robust training strategies to mitigate label noise in the medical image classification tasks. Methods: In this work, we propose a novel Bayesian statistics‐guided label refurbishment mechanism (BLRM) for DNNs to prevent overfitting noisy images. BLRM utilizes maximum a posteriori probability in the Bayesian statistics and the exponentially time‐weighted technique to selectively correct the labels of noisy images. The training images are purified gradually with the training epochs when BLRM is activated, further improving classification performance. Results: Comprehensive experiments on both synthetic noisy images (public OCT & Messidor datasets) and real‐world noisy images (ANIMAL‐10N) demonstrate that BLRM refurbishes the noisy labels selectively, curbing the adverse effects of noisy data. Also, the anti‐noise BLRMs integrated with DNNs are effective at different noise ratio and are independent of backbone DNN architectures. In addition, BLRM is superior to state‐of‐the‐art comparative methods of anti‐noise. Conclusions: These investigations indicate that the proposed BLRM is well capable of mitigating label noise in medical image classification tasks. [ABSTRACT FROM AUTHOR]

A long time ago in the machine learning literature, the idea of incorporating a mechanism inspired by the human visual system into neural networks was introduced. This idea is named the attention mechanism, and it has gone through a long development period. Today, many works have been devoted to this idea in a variety of tasks. Remarkable performance has recently been demonstrated. The goal of this paper is to provide an overview from the early work on searching for ways to implement attention idea with neural networks until the recent trends. This review emphasizes the important milestones during this progress regarding different tasks. By this way, this study aims to provide a road map for researchers to explore the current development and get inspired for novel approaches beyond the attention. [ABSTRACT FROM AUTHOR]

Text Classification methods have been improving at an unparalleled speed in the last decade thanks to the success brought about by deep learning. Historically, state-of-the-art approaches have been developed for and benchmarked against English datasets, while other languages have had to catch up and deal with inevitable linguistic challenges. This paper offers a survey with practical and linguistic connotations, showcasing the complications and challenges tied to the application of modern Text Classification algorithms to languages other than English. We engage this subject from the perspective of the Italian language, and we discuss in detail issues related to the scarcity of task-specific datasets, as well as the issues posed by the computational expensiveness of modern approaches. We substantiate this by providing an extensively researched list of available datasets in Italian, comparing it with a similarly sought list for French, which we use for comparison. In order to simulate a real-world practical scenario, we apply a number of representative methods to custom-tailored multilabel classification datasets in Italian, French, and English. We conclude by discussing results, future challenges, and research directions from a linguistically inclusive perspective. [ABSTRACT FROM AUTHOR]

The accurate prediction of short-term passenger flow is of high importance to efficiently manage the passenger flow of metro systems and adjust timetable accordingly. However, the existing methods of passenger flow prediction cannot achieve adequate accurate results due to its complex nonlinear spatiotemporal characteristics. To improve the accuracy of short-term passenger flow prediction, this paper proposes a deep learning model based on a spatiotemporal framework. Firstly, the graph convolutional network, which incorporates prior domain knowledge (such as travel time and origin–destination demand), is used to extract spatial features of passenger flow. Secondly, the attention mechanism is integrated into the gated recurrent unit to extract the time correlation of passenger flow. Finally, external factors are introduced to capture their impact on passenger flow as well. A case study of the Beijing Subway system is illustrated to verify the performance of the proposed model. The results show that compared with the existing models, the proposed model achieves the highest prediction accuracy and strong robustness. Furthermore, we demonstrate that the adjacency matrix based on travel time outperforms the one based on OD demand, especially during evening peak hours. In addition, it is also verified that the attention mechanism and external factors can improve the prediction performance of the proposed model. [ABSTRACT FROM AUTHOR]

Recent advancements in artificial intelligence (AI) techniques have enabled the development of accurate prediction models using clinical big data. AI models for perioperative risk stratification, intraoperative event prediction, biosignal analyses, and intensive care medicine have been developed in the field of perioperative medicine. Some of these models have been validated using external datasets and randomized controlled trials. Once these models are implemented in electronic health record systems or software medical devices, they could help anesthesiologists improve clinical outcomes by accurately predicting complications and suggesting optimal treatment strategies in real-time. This review provides an overview of the AI techniques used in perioperative medicine and a summary of the studies that have been published using these techniques. Understanding these techniques will aid in their appropriate application in clinical practice. [ABSTRACT FROM AUTHOR]

Traumatic Brain Injury (TBI) could lead to intracranial hemorrhage (ICH), which has now been identified as a major cause of death after trauma if it is not adequately diagnosed and properly treated within the first 24 hours. CT examination is widely preferred for urgent ICH diagnosis, which enables the fast identification and detection of ICH regions. However, the use of it requires the clinical interpretation by experts to identify the subtypes of ICH. Besides, it is unable to provide the details needed to conduct quantitative assessment, such as the volume and thickness of hemorrhagic lesions, which may have prognostic importance to the decision-making on emergency treatment. In this paper, an optimal deep learning framework is proposed to assist the quantitative assessment for ICH diagnosis and the accurate detection of different subtypes of ICH through head CT scan. Firstly, the format of raw input data is converted from 3D DICOM to NIfTI. Secondly, a pre-trained multi-class semantic segmentation model is applied to each slice of CT images, so as to obtain a precise 3D mask of the whole ICH region. Thirdly, a fine-tuned classification neural network is employed to extract the key features from the raw input data and identify the subtypes of ICH. Finally, a quantitative assessment algorithm is adopted to automatically measure both thickness and volume via the 3D shape mask combined with the output probabilities of the classification network. The results of our extensive experiments demonstrate the effectiveness of the proposed framework where the average accuracy of 96.21 percent is achieved for three types of hemorrhage. The capability of our optimal classification model to distinguish between different types of lesion plays a significant role in reducing the false-positive rate in the existing work. Furthermore, the results suggest that our automatic quantitative assessment algorithm is effective in providing clinically relevant quantification in terms of volume and thickness. It is more important than the qualitative assessment conducted through visual inspection to the decision-making on emergency surgical treatment. [ABSTRACT FROM AUTHOR]

Background: Drug discovery is time-consuming and costly. Machine learning, especially deep learning, shows great potential in quantitative structure–activity relationship (QSAR) modeling to accelerate drug discovery process and reduce its cost. A big challenge in developing robust and generalizable deep learning models for QSAR is the lack of a large amount of data with high-quality and balanced labels. To address this challenge, we developed a self-training method, Partially LAbeled Noisy Student (PLANS), and a novel self-supervised graph embedding, Graph-Isomorphism-Network Fingerprint (GINFP), for chemical compounds representations with substructure information using unlabeled data. The representations can be used for predicting chemical properties such as binding affinity, toxicity, and others. PLANS-GINFP allows us to exploit millions of unlabeled chemical compounds as well as labeled and partially labeled pharmacological data to improve the generalizability of neural network models. Results: We evaluated the performance of PLANS-GINFP for predicting Cytochrome P450 (CYP450) binding activity in a CYP450 dataset and chemical toxicity in the Tox21 dataset. The extensive benchmark studies demonstrated that PLANS-GINFP could significantly improve the performance in both cases by a large margin. Both PLANS-based self-training and GINFP-based self-supervised learning contribute to the performance improvement. Conclusion: To better exploit chemical structures as an input for machine learning algorithms, we proposed a self-supervised graph neural network-based embedding method that can encode substructure information. Furthermore, we developed a model agnostic self-training method, PLANS, that can be applied to any deep learning architectures to improve prediction accuracies. PLANS provided a way to better utilize partially labeled and unlabeled data. Comprehensive benchmark studies demonstrated their potentials in predicting drug metabolism and toxicity profiles using sparse, noisy, and imbalanced data. PLANS-GINFP could serve as a general solution to improve the predictive modeling for QSAR modeling. [ABSTRACT FROM AUTHOR]