Search

Cardiac health diseases are one of the key causes of death around the globe. The number of heart patients has considerably increased during the pandemic. Therefore, it is crucial to assess and analyze the medical and cardiac images. Deep learning architectures, specifically convolutional neural networks have profoundly become the primary choice for the assessment of cardiac medical images. The left ventricle is a vital part of the cardiovascular system where the boundary and size perform a significant role in the evaluation of cardiac function. Due to automatic segmentation and good promising results, the left ventricle segmentation using deep learning has attracted a lot of attention. This article presents a critical review of deep learning methods used for the left ventricle segmentation from frequently used imaging modalities including magnetic resonance images, ultrasound, and computer tomography. This study also demonstrates the details of the network architecture, software, and hardware used for training along with publicly available cardiac image datasets and self-prepared dataset details incorporated. The summary of the evaluation matrices with results used by different researchers is also presented in this study. Finally, all this information is summarized and comprehended in order to assist the readers to understand the motivation and methodology of various deep learning models, as well as exploring potential solutions to future challenges in LV segmentation., Competing Interests: The authors declare that they have no conflicts of interest., (Copyright © 2023 Muhammad Ali Shoaib et al.)

One of the primary factors contributing to death across all age groups is cardiovascular disease. In the analysis of heart function, analyzing the left ventricle (LV) from 2D echocardiographic images is a common medical procedure for heart patients. Consistent and accurate segmentation of the LV exerts significant impact on the understanding of the normal anatomy of the heart, as well as the ability to distinguish the aberrant or diseased structure of the heart. Therefore, LV segmentation is an important and critical task in medical practice, and automated LV segmentation is a pressing need. The deep learning models have been utilized in research for automatic LV segmentation. In this work, three cutting-edge convolutional neural network architectures (SegNet, Fully Convolutional Network, and Mask R-CNN) are designed and implemented to segment the LV. In addition, an echocardiography image dataset is generated, and the amount of training data is gradually increased to measure segmentation performance using evaluation metrics. The pixel's accuracy, precision, recall, specificity, Jaccard index, and dice similarity coefficients are applied to evaluate the three models. The Mask R-CNN model outperformed the other two models in these evaluation metrics. As a result, the Mask R-CNN model is used in this study to examine the effect of training data. For 4,000 images, the network achieved 92.21% DSC value, 85.55% Jaccard index, 98.76% mean accuracy, 96.81% recall, 93.15% precision, and 96.58% specificity value. Relatively, the Mask R-CNN outperformed other architectures, and the performance achieves stability when the model is trained using more than 4,000 training images., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Shoaib, Lai, Chuah, Hum, Ali, Dhanalakshmi, Wang and Wu.)

Pneumonia is an infamous life-threatening lung bacterial or viral infection. The latest viral infection endangering the lives of many people worldwide is the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which causes COVID-19. This paper is aimed at detecting and differentiating viral pneumonia and COVID-19 disease using digital X-ray images. The current practices include tedious conventional processes that solely rely on the radiologist or medical consultant's technical expertise that are limited, time-consuming, inefficient, and outdated. The implementation is easily prone to human errors of being misdiagnosed. The development of deep learning and technology improvement allows medical scientists and researchers to venture into various neural networks and algorithms to develop applications, tools, and instruments that can further support medical radiologists. This paper presents an overview of deep learning techniques made in the chest radiography on COVID-19 and pneumonia cases., Competing Interests: All the authors declared that they have no conflict of interest., (Copyright © 2021 Woan Ching Serena Low et al.)

Social media platforms such as Twitter and Facebook have become popular channels for people to record and express their feelings, opinions, and feedback in the last decades. With proper extraction techniques such as sentiment analysis, this information is useful in many aspects, including product marketing, behavior analysis, and pandemic management. Sentiment analysis is a technique to analyze people's thoughts, feelings and emotions, and to categorize them into positive, negative, or neutral. There are many ways for someone to express their feelings and emotions. These sentiments are sometimes accompanied by sarcasm, especially when conveying intense emotion. Sarcasm is defined as a positive sentence with underlying negative intention. Most of the current research work treats them as two distinct tasks. To date, most sentiment and sarcasm classification approaches have been treated primarily and standalone as a text categorization problem. In recent years, research work using deep learning algorithms have significantly improved performance for these standalone classifiers. One of the major issues faced by these approaches is that they could not correctly classify sarcastic sentences as negative. With this in mind, we claim that knowing how to spot sarcasm will help sentiment classification and vice versa. Our work has shown that these two tasks are correlated. This paper proposes a multi-task learning-based framework utilizing a deep neural network to model this correlation to improve sentiment analysis's overall performance. The proposed method outperforms the existing methods by a margin of 3%, with an F1-score of 94%. [ABSTRACT FROM AUTHOR]

The segmentation of the left ventricle (LV) is one of the fundamental procedures that must be performed to obtain quantitative measures of the heart, such as its volume, area, and ejection fraction. In clinical practice, the delineation of LV is still often conducted semi-automatically, leaving it open to operator subjectivity. The automatic LV segmentation from echocardiography images is a challenging task due to poorly defined boundaries and operator dependency. Recent research has demonstrated that deep learning has the capability to employ the segmentation process automatically. However, the well-known state-of-the-art segmentation models still lack in terms of accuracy and speed. This study aims to develop a single-stage lightweight segmentation model that precisely and rapidly segments the LV from 2D echocardiography images. In this research, a backbone network is used to acquire both low-level and high-level features. Two parallel blocks, known as the spatial feature unit and the channel feature unit, are employed for the enhancement and improvement of these features. The refined features are merged by an integrated unit to segment the LV. The performance of the model and the time taken to segment the LV are compared to other established segmentation models, DeepLab, FCN, and Mask RCNN. The model achieved the highest values of the dice similarity index (0.9446), intersection over union (0.8445), and accuracy (0.9742). The evaluation metrics and processing time demonstrate that the proposed model not only provides superior quantitative results but also trains and segments the LV in less time, indicating its improved performance over competing segmentation models. [ABSTRACT FROM AUTHOR]

Digitalization is revolutionizing our way of life and catalyzing the transformation into smart city. Intelligent Transportation System (ITS) being an indispensable component of smart city leverages massive amount of collected information to improve traffic efficiency, thereby creating a safer and comfortable commuting environment for the users. One of the most important tasks in ITS is vehicle classification which aims to find out the vehicle identity, including vehicle segment, automobile maker, model, etc. In this article, we first present the vehicle classification taxonomy branched based on the nature of input data. We subsequently investigate diverse area of sensor-based vehicle classification followed by image-based vehicle classification which cover both the conventional and emerging techniques in a comprehensive manner. The methodologies together with the corresponding strengths and potential weaknesses are elucidated so that it serves as an invaluable reference for vehicle classification related applications in the future. More importantly, we express our views on future research direction with the intention to accelerate the development of vehicle classification field. In contrast to previous works, we aim to cover wide spectrum of vehicle classification methodologies in this review to provide more clarity when it comes to selecting a solution that suits individual need. They include both sensor-based and image-based vehicle classification for VTR, VLR and VMMR. We employ exhaustive coverage approach for the former to identify the extant solutions that are built upon various kinds of sensing technologies and they are further collated according to the installation methods. For image-based methods, we screen for the works that have been central to both pre- and post-deep learning era. The featured works either address the shortcoming of previous works in image domain or present novel concept to advance the classification performance. [ABSTRACT FROM AUTHOR]

• The proposed CVAE-based method surpassed classical data balancing methods (SMOTE & Adasyn). • A Stack Predictor for Heart Disease (SPFHD) is proposed for heart disease detection. • The SHAP framework is used to interpret the features that impact the SPFHD output. • A two-step statistical test verifies the performance disparity between the models. Worldwide, cardiovascular disease is the leading cause of death. Based on clinical data, a Machine Learning (ML) system can detect cardiac disease in its early stages, which enables a reduction in mortality rates. However, imbalanced and high dimensionality data have been a persistent challenge in ML, impeding accurate predictive data analysis in many real-world applications, such as the detection of cardiovascular disease. To address this, computational methods targeting heart disease detection have been developed. However, their performance is still inadequate. Hence, this study presents a new stack predictor for the heart disease model (termed SPFHD). SPFHD employs five common tree-based ensemble learning algorithms as base models for heart disease detection. In addition, the predictions from the base models are integrated using a support vector machine algorithm to enhance the accuracy of heart disease detection. A new conditional variational autoencoder (CVAE) based method is developed to overcome the imbalance issue, which performs better than the conventional balancing methods. Finally, the SPFHD model is tuned by Bayesian optimization. The results show that the proposed SPFHD model outperforms the state-of-art methods over four datasets achieving higher f1-score of 4.68 %, 4.55 %, 2 %, and 1 % for HD clinical, Z-Alizadeh Sani, Statlog, and Cleveland, respectively. Moreover, this new framework offers vital interpretations which assist in understanding model success by leveraging the powerful SHapley Additive explanation (SHAP) algorithm. This highlights the most significant attributes for detecting heart disease and overcoming the limitations of current 'Black-box' methods that cannot reveal causal relationships between features. [ABSTRACT FROM AUTHOR]

Recent works in image captioning have shown very promising raw performance. However, we realize that most of these encoder–decoder style networks with attention do not scale naturally to large vocabulary size, making them difficult to deploy on embedded systems with limited hardware resources. This is because the size of word and output embedding matrices grow proportionally with the size of vocabulary, adversely affecting the compactness of these networks. To address this limitation, this paper introduces a brand new idea in the domain of image captioning. That is, we tackle the problem of compactness of image captioning models which is hitherto unexplored. We showed that our proposed model, named COMIC for compact image captioning, achieves comparable results in five common evaluation metrics with state-of-the-art approaches on both MS-COCO and InstaPIC-1.1M datasets despite having an embedded vocabulary size that is 39×−99× smaller. [ABSTRACT FROM AUTHOR]

Driver distraction is one of the main causes of fatal traffic accidents. Therefore, the ability to detect driver inattention is essential in building a safe yet intelligent transportation system. Currently, the available driver distraction detection systems are not widely available or limited to specific class actions. Various research efforts have approached the problem through different techniques, including the usage of intrusive sensors, which are not feasible for mass production. Most of the work in early 2010s used traditional machine learning approaches to perform the detection task. With the emergence of deep learning algorithms, many research has been conducted to perform distraction detection using neural networks. Furthermore, most of the work in the field is conducted under simulation or lab environment, and did not validate the proposed system under naturalistic scenario. Most importantly, the research efforts in the field could be further subdivided into many subtasks. Thus, this paper aims to provide a comprehensive review of approaches used to detect driving distractions through various methods. We review all recent papers from 2014–2021 and categorized them according to the sensors used. Based on the reviewed articles, a simplified framework to visualize the detection flow, starting from the used sensors, collected data, measured data, computed events, inferred behaviour, and finally its inferred distraction type is proposed. Besides providing an in-depth review and concise summary of various published works, the practicality and relevancy of driver distraction detection towards increasing vehicle automation are discussed. Further, several open research challenges and provide suggestions for future research directions are provided. We believe that this review will remain helpful despite the development towards a higher level of vehicle automation. • An overview of usage of machine learning in driver distraction detection is provided. • Deep learning outperforms traditional machine learning in driver distraction detection. • A framework for driver distraction detection is introduced. • Research gap and future research for driver distraction detection is discussed. [ABSTRACT FROM AUTHOR]

Object detection in aerial images has been an active research area thanks to the vast availability of unmanned aerial vehicles (UAVs). Along with the increase of computational power, deep learning algorithms are commonly used for object detection tasks. However, aerial images have large variations, and the object sizes are usually small, rendering lower detection accuracy. Besides, real-time inferencing on low-cost edge devices remains an open-ended question. In this work, we explored the usage of state-of-the-art deep learning object detection on low-cost edge hardware. We propose YOLO-RTUAV, an improved version of YOLOv4-Tiny, as the solution. We benchmarked our proposed models with various state-of-the-art models on the VAID and COWC datasets. Our proposed model can achieve higher mean average precision (mAP) and frames per second (FPS) than other state-of-the-art tiny YOLO models, especially on a low-cost edge device such as the Jetson Nano 2 GB. It was observed that the Jetson Nano 2 GB can achieve up to 12.8 FPS with a model size of only 5.5 MB. [ABSTRACT FROM AUTHOR]

Cracks are significant indicators for the evaluation of the structural health and monitoring process. However, manual crack detection is a time-consuming and challenging task due to large areas, complex structure, and safety risks. Deep learning has emerged as a useful technique to automate the crack detection and identification process. For balanced data, existing deep learning models attempt to segment both crack pixels and non-crack pixels equally. However, due to the highly imbalanced ratio between crack pixels and non-crack pixels, the pixel-wise loss is dominantly guided by the non-crack region and has relatively little influence from the crack region. This leads to the low segmentation accuracy for crack pixels. To address the imbalance problem, this work proposes a local weighting factor with a sensitivity map to remove the network biasness and accurately predict the sensitive pixels. Furthermore, we implement a deep fully convolutional neural network for crack pixel segmentation based on residual blocks with a different number of filters in each convolutional operation that segments the crack pixels and non-crack pixels with unbiased probabilities. For performance evaluation, a new Multi Structure Crack Image (MSCI) dataset is built. By using the MSCI dataset, the proposed method achieved 98.19% crack pixel accuracy and 98.13% non-crack pixel accuracy along with 98.16% average accuracy. In addition, the training time for 10 epochs has dramatically decreased and the experimental results show that the proposed crack segmentation network (CSN) architecture along with local weighting factor and sensitivity map has better crack pixel segmentation accuracy than U-Net and SegNet architectures. [ABSTRACT FROM AUTHOR]

Distracted driving is the prime factor of motor vehicle accidents. Current studies on distraction detection focus on improving distraction detection performance through various techniques, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs). However, the research on detection of distracted drivers through pose estimation is scarce. This work introduces an ensemble of ResNets, which is named Optimally-weighted Image-Pose Approach (OWIPA), to classify the distraction through original and pose estimation images. The pose estimation images are generated from HRNet and ResNet. We use ResNet101 and ResNet50 to classify the original images and the pose estimation images, respectively. An optimum weight is determined through grid search method, and the predictions from both models are weighted through this parameter. The experimental results show that our proposed approach achieves 94.28% accuracy on AUC Distracted Driver Dataset. [ABSTRACT FROM AUTHOR]

The removal of shadows from images is a classic problem in computer vision, aiming to restore the lighting in shadowed areas, thereby reducing the information interference and loss caused by the presence of shadows. In recent years, numerous excellent shadow removal algorithms have emerged, particularly with the rapid development of deep learning technology, which has disrupted traditional physics-based approaches and significantly improved the effectiveness of shadow removal. In this paper, we conduct a comprehensive survey of shadow removal methods published from 2017 to the present. We first introduce background knowledge about image shadow removal, providing detailed explanations of both physics-based and learning-based shadow removal methods. We analyze and compare these algorithms from both quantitative and qualitative perspectives, reassessing all models that provided open-source result sets according to uniform criteria. Additionally, we introduce commonly used datasets and evaluation metrics in the field. Finally, we discuss applications of shadow removal in specific scenarios, along with research challenges and opportunities in this domain. • Surveys taxonomy of recently image shadow removal techniques, datasets and performance metrics. • Systematic quantitative evaluations are summarized. • The impacts, applications and importance of shadow removal are explored. • Challenges and future direction are explored and analyzed. [ABSTRACT FROM AUTHOR]

Recent research that applies Transformer-based architectures to image captioning has resulted in state-of-the-art image captioning performance, capitalising on the success of Transformers on natural language tasks. Unfortunately, though these models work well, one major flaw is their large model sizes. To this end, we present three parameter reduction methods for image captioning Transformers: Radix Encoding, cross-layer parameter sharing, and attention parameter sharing. By combining these methods, our proposed ACORT models have 3.7 × to 21.6 × fewer parameters than the baseline model without compromising test performance. Results on the MS-COCO dataset demonstrate that our ACORT models are competitive against baselines and SOTA approaches, with CIDEr score ⩾ 126. Finally, we present qualitative results and ablation studies to demonstrate the efficacy of the proposed changes further. Code and pre-trained models are publicly available at https://github.com/jiahuei/sparse-image-captioning. [ABSTRACT FROM AUTHOR]

• This is the first extensive attempt at exploring model pruning for image captioning task. Empirically, we show that deep captioning networks with 80% to 95% sparse are capable to either match or even slightly outperform their dense counterparts. In addition, we propose a pruning method - Supermask Pruning (SMP) that performs continuous and gradual sparsification during training stage based on parameter sensitivity in an end-to-end fashion. • We investigate an ideal way to combine pruning with fine-tuning of pre-trained CNN, and show that both decoder pruning and training should be done before pruning the encoder. • We release the pre-trained sparse models for UD and ORT that are capable of achieving CIDEr scores >120 on MS-COCO dataset; yet are only 8.7 MB (reduction of 96% compared to dense UD) and 14.5 MB (reduction of 94% compared to dense ORT) in model size. Our code and pre-trained models are publicly available at https://github.com/jiahuei/sparse-image-captioning With the advancement of deep models, research work on image captioning has led to a remarkable gain in raw performance over the last decade, along with increasing model complexity and computational cost. However, surprisingly works on compression of deep networks for image captioning task has received little to no attention. For the first time in image captioning research, we provide an extensive comparison of various unstructured weight pruning methods on three different popular image captioning architectures, namely Soft-Attention, Up-Down and Object Relation Transformer. Following this, we propose a novel end-to-end weight pruning method that performs gradual sparsification based on weight sensitivity to the training loss. The pruning schemes are then extended with encoder pruning, where we show that conducting both decoder pruning and training simultaneously prior to the encoder pruning provides good overall performance. Empirically, we show that an 80% to 95% sparse network (up to 75% reduction in model size) can either match or outperform its dense counterpart. The code and pre-trained models for Up-Down and Object Relation Transformer that are capable of achieving CIDEr scores > 120 on the MS-COCO dataset but with only 8.7 MB and 14.5 MB in model size (size reduction of 96% and 94% respectively against dense versions) are publicly available at https://github.com/jiahuei/sparse-image-captioning. [ABSTRACT FROM AUTHOR]