Search

Ultrasound images are often affected by limited resolution, artifacts, and inherent speckle noise. To address these challenges, researchers have explored denoising approaches. Recently, deep learning methods have demonstrated distinct advantages in ultrasound image denoising. However, further improvements are needed to preserve structural details, such as boundaries, edges, and margins. This paper proposes a hybrid CNN-transformer network called HCTSpeckle, an encoder-decoder network with a fusion block designed to enhance ultrasound images. The fusion block combines swin transformers to capture global modeling relationships, and convolutional neural networks to extract local modeling details. It is integrated into the encoder-decoder structure, allowing the model to focus on both local and global texture structural information. An improved swin block is also introduced into the network to improve robustness by extracting more significant features. HCTSpeckle was evaluated both quantitatively and qualitatively with clinical objectives using two public and two private datasets. Both results showed that HCTSpeckle significantly enhanced the ultrasound image quality and outperformed state-of-the-art methods in noise reduction and structure preservation across all four datasets. Compared to existing denoising methods, HCTSpeckle achieved notably faster performance in terms of complexity comparison, such as parameter counts, gigaFLOPs, and inference time. Moreover, this study assessed the effectiveness of HCTSpeckle for alveolar bone segmentation using dental images, demonstrating that HCTSpeckle significantly improved segmentation performance. Furthermore, an experienced radiologist blindly rated the 250 dental US images on a scale of 1 to 5, with 5 being the highest image quality, showing that HCTSpeckle consistently produced higher-quality images.

Objective: Measuring the severity of the lateral spinal curvature, or Cobb angle, is critical for monitoring and making treatment decisions for children with adolescent idiopathic scoliosis (AIS). However, manual measurement is time-consuming and subject to human error. Therefore, clinicians seek an automated measurement method to streamline workflow and improve accuracy. This paper reports on a novel machine learning algorithm of cascaded convolutional neural networks (CNN) to measure the Cobb angle on spinal radiographs automatically. Methods: The developed method consisted of spinal column segmentation using a CNN, vertebra localization and segmentation using iterative vertebra body location coupled with another CNN, point-set registration to correct vertebra segmentations, and Cobb angle measurement using the final segmentations. Measurement performance was evaluated with the circular mean absolute error (CMAE) and percentage within clinical acceptance ( $\le 5^{\circ }$ ) between automatic and manual measurements. Analysis was separated by curve severity to identify any potential systematic biases using independent samples Student’s t-tests. Results: The method detected 346 of the 352 manually measured Cobb angles (98%), with a CMAE of 2.8° and 91% of measurements within the 5° clinical acceptance. No statistically significant differences were found between the CMAEs of mild ( $ < 25^{\circ }$ ), moderate (25°-45°), and severe ( $\ge 45^{\circ }$ ) groups. The average measurement time per radiograph was 17.7±10.2s, improving upon the estimated average of 30s it takes an experienced rater to measure. Additionally, the algorithm outputs segmentations with the measurement, allowing clinicians to interpret measurement results. Discussion/Conclusion: The developed method measured Cobb angles on radiographs automatically with high accuracy, quick measurement time, and interpretability, suggesting clinical feasibility.

Visual inspections of aircraft are a vital aspect of aircraft maintenance. They involve ground personnel walking around the aircraft, mounting scaffolding or ladders, and fully inspecting the aircraft. A visual inspection, however, requires a significant amount of time to perform and may be prone to human error. To address these challenges, a deep-learning-based defect-detection model is proposed, which has the potential to reduce inspection time significantly and enhance inspection reliability. This approach utilises an Unmanned Aircraft System (UAS) equipped with an onboard camera system to capture images for analysis. The proposed machine-learning (ML) model is trained using a deep-learning framework, specifically YOLOv8, on a diverse dataset comprising of both defective and non-defective images of the aircraft. The labelling process for novel classes is accelerated through utilising a low accuracy pre-trained model. Following model analysis, the operator inspects the defects reported in the images. To facilitate efficient identification, object detection techniques, such as bounding boxes, are employed. In this study, various hyperparameter combinations to optimise the model’s performance are explored. Implementing this ML detection system would mitigate the risks associated with human error and improve inspection accuracy, providing a safer visual inspection process for ground personnel and a more efficient solution for aircraft visual inspections for preventative maintenance. Moreover, the utilisation of a UAS enables inspections of inaccessible or hazardous areas without compromising personnel safety. This research demonstrates the potential of deep learning for aircraft visual inspections. The defects analysed are damaged and missing rivets, filiform corrosion, and missing panels, while future research aims to expand the scope to include additional defects such as paint chipping, scratches, burns, and rust. The model’s capability to discern the condition of rivets, panels, and filiform corrosion is evidenced by its achieved validation mean Average Precision (mAP) of 85 %.

The self-regulated recognition of human activities from time-series smartphone sensor data is a growing research area in smart and intelligent health care. Deep learning (DL) approaches have exhibited improvements over traditional machine learning (ML) models in various domains, including human activity recognition (HAR). Several issues are involved with traditional ML approaches; these include handcrafted feature extraction, which is a tedious and complex task involving expert domain knowledge, and the use of a separate dimensionality reduction module to overcome overfitting problems and hence provide model generalization. In this article, we propose a DL-based approach for activity recognition with smartphone sensor data, i.e., accelerometer and gyroscope data. Convolutional neural networks (CNNs), autoencoders (AEs), and long short-term memory (LSTM) possess complementary modeling capabilities, as CNNs are good at automatic feature extraction, AEs are used for dimensionality reduction and LSTMs are adept at temporal modeling. In this study, we take advantage of the complementarity of CNNs, AEs, and LSTMs by combining them into a unified architecture. We explore the proposed architecture, namely, “ConvAE-LSTM”, on four different standard public datasets (WISDM, UCI, PAMAP2, and OPPORTUNITY). The experimental results indicate that our novel approach is practical and provides relative smartphone-based HAR solution performance improvements in terms of computational time, accuracy, F1-score, precision, and recall over existing state-of-the-art methods.

The early diagnosis of cerebral palsy is an area which has recently seen significant multi-disciplinary research. Diagnostic tools such as the General Movements Assessment (GMA), have produced some very promising results. However, the prospect of automating these processes may improve accessibility of the assessment and also enhance the understanding of movement development of infants. Previous works have established the viability of using pose-based features extracted from RGB video sequences to undertake classification of infant body movements based upon the GMA. In this paper, we propose a series of new and improved features, and a feature fusion pipeline for this classification task. We also introduce the RVI-38 dataset, a series of videos captured as part of routine clinical care. By utilising this challenging dataset we establish the robustness of several motion features for classification, subsequently informing the design of our proposed feature fusion framework based upon the GMA. We evaluate our proposed framework’s classification performance using both the RVI-38 dataset and the publicly available MINI-RGBD dataset. We also implement several other methods from the literature for direct comparison using these two independent datasets. Our experimental results and feature analysis show that our proposed pose-based method performs well across both datasets. The proposed features afford us the opportunity to include finer detail than previous methods, and further model GMA specific body movements. These new features also allow us to take advantage of additional body-part specific information as a means of improving the overall classification performance, whilst retaining GMA relevant, interpretable, and shareable features.

The early diagnosis of cerebral palsy is an area which has recently seen significant multi-disciplinary research. Diagnostic tools such as the General Movements Assessment (GMA), have produced some very promising results, however these manual methods can be laborious. The prospect of automating these processes is seen as key in advancing this field of study. In our previous works, we examined the viability of using pose-based features extracted from RGB video sequences to undertake classification of infant body movements based upon the GMA. In this paper, we propose a new deep learning framework for this classification task. We also propose a visualization framework which identifies body-parts with the greatest contribution towards a classification decision. The inclusion of a visualization framework is an important step towards automation as it helps make the decisions made by the machine learning framework interpretable. We directly compare the proposed framework’s classification with several other methods from the literature using two independent datasets. Our experimental results show that the proposed method performs more consistently and more robustly than our previous pose-based techniques as well as other features from related works in this setting. We also find that our visualization framework helps provide greater interpretability, enhancing the likelihood of the adoption of these technologies within the medical domain.

Crowd counting is considered as the essential computer vision application that uses the convolutional neural network to model the crowd density as the regression task. However, the vision-based models are hard to extract the feature under low-quality conditions. As we know, visual and audio are used widely as media platforms for human beings to touch the physical change of the world. The cross-modal information gives us an alternative method of solving the crowd counting task. In this case, in order to solve this problem, a model named the Audio-Visual Multi-Scale Network (AVMSN) is established to model the unconstrained visual and audio sources for completing the crowd counting task in this paper. Based on the Feature extraction and Multi-modal fusion module, in order to handle the objects of various sizes in the crowd scene, the Sample Convolutional Blocks are adopted by the AVMSN as the multi-scale Vision-end branch in the Feature extraction module to calculate the weighted-visual feature. Besides, the audio, which is the temporal domain transformed into the spectrogram information and the audio feature is learned by the audio-VGG network. Finally, the weighted-visual and audio features are fused by the Multi-modal fusion module, which adopts the cascade fusion architecture to calculate the estimated density map. The experimental results show the proposed AVMSN achieves a lower mean absolute error than other state-of-art crowd counting models under the low-quality conditions.

Person and vehicle re-identification (re-ID) are important challenges for the analysis of the burgeoning collection of urban surveillance videos. To efficiently evaluate such videos, which are populated with both vehicles and pedestrians, it would be preferable to have one unified framework with effective performance across both domains. Unfortunately, due to the contrasting composition of humans and vehicles, no architecture has yet been established that can adequately perform both tasks. We release a Person and Vehicle Unified Data Set (PVUD) comprising of both pedestrians and vehicles from popular existing re-ID data sets, in order to better model the data that we would expect to find in the real world. We exploit the generalisation ability of metric learning to propose a re-ID framework that can learn to re-identify humans and vehicles simultaneously. We design our network, MidTriNet, to harness the power of mid-level features to develop better representations for the re-ID tasks. We help the system to handle mixed data by appending unification terms with additional hard negative and hard positive mining to MidTriNet. We attain comparable accuracy training on PVUD to training on the comprising data sets separately, supporting the system's generalisation power. To further demonstrate the effectiveness of our framework, we also obtain results better than, or competitive with, the state-of-the-art on each of the Market-1501, CUHK03, VehicleID and VeRi data sets.

The pursuit of early diagnosis of cerebral palsy has been an active research area with some very promising results using tools such as the General Movements Assessment (GMA). In our previous work, we explored the feasibility of extracting pose-based features from video sequences to automatically classify infant body movement into two categories, normal and abnormal. The classification was based upon the GMA, which was carried out on the video data by an independent expert reviewer. In this paper we extend our previous work by extracting the normalised pose-based feature sets, Histograms of Joint Orientation 2D (HOJO2D) and Histograms of Joint Displacement 2D (HOJD2D), for use in new deep learning architectures. We explore the viability of using these pose-based feature sets for automated classification within a deep learning framework by carrying out extensive experiments on five new deep learning architectures. Experimental results show that the proposed fully connected neural network FCNet performed robustly across different feature sets. Furthermore, the proposed convolutional neural network architectures demonstrated excellent performance in handling features in higher dimensionality. We make the code, extracted features and associated GMA labels publicly available.

Impulsive noise plays an important role in power line communication among other applications. To improve the communication performance, this paper proposes a novel design of nonlinear processing which improves the fundamental performance of signal detection in impulsive noise. Power-law tails are firstly introduced in nonlinearity design to provide adjustable decay factors for different distributions. Four modes of nonlinearity functions are developed and analyzed. By taking the exponent and the threshold as two arguments, we formulate the nonlinearity design as an optimization problem of maximizing the efficacy function, which is the fundamental measurement for detecting a deterministic signal in impulsive noise. Given that the efficacy function is differentiable, unimodal but without closed-form derivatives, we propose to solve the optimization problem by derivative-free methods, e.g. the Nelder-Mead simplex method. As concept demonstration, our method is used for three commonly-used distribution examples. Results show that our nonlinearity design can achieve almost the same efficacy and detection performance as the locally optimal detector, with the advantage of easy-to-apply closed form expressions.

Foreground-background segmentation has been an active research area over the years. However, conventional models fail to produce accurate results when challenged with the videos of challenging illumination conditions. In this paper, we present a robust model that allows accurately extracting the foreground even in exceptionally dark or bright scenes and in continuously varying illumination in a video sequence. This is accomplished by a triple multi-task generative adversarial network (TMT-GAN) that effectively models the semantic relationship between the dark and bright images and performs binary segmentation end-to-end. Our contribution is twofold: first, we show that by jointly optimizing the GAN loss and the segmentation loss, our network simultaneously learns both tasks that mutually benefit each other. Second, fusing features of images with varying illumination into the segmentation branch vastly improve the performance of the network. Comparative evaluations on highly challenging real and synthetic benchmark datasets (ESI and SABS) demonstrate the robustness of TMT-GAN and its superiority over state-of-the-art approaches.