Search

Recently, scientists have widely utilized Artificial Intelligence (AI) approaches in intelligent agriculture to increase the productivity of the agriculture sector and overcome a wide range of problems. Detection and classification of plant diseases is a challenging problem due to the vast numbers of plants worldwide and the numerous diseases that negatively affect the production of different crops. Early detection and accurate classification of plant diseases is the goal of any AI-based system. This paper proposes a hybrid framework to improve classification accuracy for plant leaf diseases significantly. This proposed model leverages the strength of Convolutional Neural Networks (CNNs) and Vision Transformers (ViT), where an ensemble model, which consists of the well-known CNN architectures VGG16, Inception-V3, and DenseNet20, is used to extract robust global features. Then, a ViT model is used to extract local features to detect plant diseases precisely. The performance proposed model is evaluated using two publicly available datasets (Apple and Corn). Each dataset consists of four classes. The proposed hybrid model successfully detects and classifies multi-class plant leaf diseases and outperforms similar recently published methods, where the proposed hybrid model achieved an accuracy rate of 99.24% and 98% for the apple and corn datasets. [ABSTRACT FROM AUTHOR]

Objectives: Computer-aided schizophrenia (SZ) detection methods mainly depend on electroencephalogram and brain magnetic resonance images, which both capture physical signals from patients' brains. These inspection techniques take too much time and affect patients' compliance and cooperation, while difficult for clinicians to comprehend the principle of detection decisions. This study proposes a novel method using face diagnosis images based on traditional Chinese medicine principles, providing a non-invasive, efficient, and interpretable alternative for SZ detection. Methods: An innovative face diagnosis image analysis method for SZ detection, which learns feature representations based on Vision Transformer (ViT) directly from face diagnosis images. It provides a face features distribution visualization and quantitative importance of each facial region and is proposed to supplement interpretation and to increase efficiency in SZ detection while keeping a high detection accuracy. Results: A benchmarking platform comprising 921 face diagnostic images, 6 benchmark methods, and 4 evaluation metrics was established. The experimental results demonstrate that our method significantly improves SZ detection performance with a 3–10% increase in accuracy scores. Additionally, it is found that facial regions rank in descending order according to importance in SZ detection as eyes, mouth, forehead, cheeks, and nose, which is exactly consistent with the clinical traditional Chinese medicine experience. Conclusions: Our method fully leverages semantic feature representations of first-introduced face diagnosis images in SZ, offering strong interpretability and visualization capabilities. It not only opens a new path for SZ detection but also brings new tools and concepts to the research and application in the field of mental illness. [ABSTRACT FROM AUTHOR]

Accurate and interpretable segmentation of medical images is crucial for computer-aided diagnosis and image-guided interventions. This study explores the integration of semantic segmentation and explainable AI techniques on the MnMs-2 Cardiac MRI dataset. We propose a segmentation model that achieves competitive dice scores (nearly 90 %) and Hausdorff distance (less than 70), demonstrating its effectiveness for cardiac MRI analysis. Furthermore, we leverage Grad-CAM, and Feature Ablation, explainable AI techniques, to visualise the regions of interest guiding the model predictions for a target class. This integration enhances interpretability, allowing us to gain insights into the model decision-making process and build trust in its predictions. [ABSTRACT FROM AUTHOR]

Background: Tuberculosis (TB) is a bacterial disease that mainly affects the lungs, but it can also impact other parts of the body, such as the brain, bones, and kidneys. The disease is caused by a bacterium called Mycobacterium tuberculosis and spreads through the air when an infected person coughs or sneezes. TB can be inactive or active; in its active state, noticeable symptoms appear, and it can be transmitted to others. There are ongoing challenges in fighting TB, including resistance to medications, co-infections, and limited resources in areas heavily affected by the disease. These issues make it challenging to eradicate TB. Objective: Timely and precise diagnosis is essential for effective control, especially since TB often goes undetected and untreated, particularly in remote and under-resourced locations. Chest X-ray (CXR) images are commonly used to diagnose TB. However, difficulties can arise due to unusual findings on X-rays and a shortage of radiologists in high-infection areas. Method: To address these challenges, a computer-aided diagnosis (CAD) system that uses the vision transformer (ViT) technique has been developed to accurately identify TB in CXR images. This innovative hybrid CAD approach combines ViT with Principal Component Analysis (PCA) and machine learning (ML) techniques for TB classification, introducing a new method in this field. In the hybrid CAD system, ViT is used for deep feature extraction as a base model, PCA is used to reduce feature dimensions, and various ML methods are used to classify TB. This system allows for quickly identifying TB, enabling timely medical action and improving patient outcomes. Additionally, it streamlines the diagnostic process, reducing time and costs for patients and lessening the workload on healthcare professionals. The TB chest X-ray dataset was utilized to train and evaluate the proposed CAD system, which underwent pre-processing techniques like resizing, scaling, and noise removal to improve diagnostic accuracy. Results: The performance of our CAD model was assessed against existing models, yielding excellent results. The model achieved remarkable metrics: an average precision of 99.90%, recall of 99.52%, F1-score of 99.71%, accuracy of 99.84%, false negative rate (FNR) of 0.48%, specificity of 99.52%, and negative predictive value (NPV) of 99.90%. Conclusions: This evaluation highlights the superior performance of our model compared to the latest available classifiers. [ABSTRACT FROM AUTHOR]

Diabetic retinopathy (DR) is one of the most common causes of visual impairment worldwide and requires reliable automated detection methods. Numerous research efforts have developed various conventional methods for early detection of DR. Research in the field of DR remains insufficient, indicating the potential for advances in diagnosis. In this paper, a hybrid model (HybridFusionNet) that integrates vision transformer (VIT) and attention processes is presented. It improves classification in the binary ( B c l ) and multi-class ( M c l ) stages by utilizing deep features from the DR stages. As a result, both the SAN and VIT models improve the recognition accuracy (A c c) in both stages.The HybridFusionNet mechanism achieves a competitive improvement in multi-stage and binary stages, which is A c c in B c l and M c l , with 91% and 99%, respectively. This illustrates that this model is suitable for a better diagnosis of DR. [ABSTRACT FROM AUTHOR]

Glacial landforms, created by the continuous movements of glaciers over millennia, are crucial topics in geomorphological research. Their systematic analysis affords invaluable insights into past climatic oscillations and augments understanding of long-term climate change dynamics. The classification of these types of terrain traditionally depends on labor-intensive manual or semi-automated methods. However, the emergence of automated techniques driven by deep learning and neural networks holds promise for enhancing efficiency of terrain classification workflows. This study evaluated the effectiveness of Convolutional Neural Network (CNN) architectures, particularly Residual Neural Network (ResNet) and VGG in comparison with Vision Transformer (ViT) architecture in the glacial landform classification task. By using preprocessed input data from Digital Elevation Model (DEM) which covers regions such as the Lubawa Upland and Gardno-Leba Plain in Poland, as well as the Elise Glacier in Svalbard, Norway, comprehensive assessments of those methods were conducted. The final results highlight the unique ability of deep learning methods to accurately classify glacial landforms. Classification process presented in this study can be the efficient, repeatable and fast solution for automatic terrain classification.

Abstract To ensure the reliability of machining quality, it is crucial to predict tool wear accurately. In this paper, a novel deep learning-based model is proposed, which synthesizes the advantages of power spectral density (PSD), convolutional neural networks (CNN), and vision transformer model (ViT), namely PSD-CVT. PSD maps can provide a comprehensive understanding of the spectral characteristics of the signals. It makes the spectral characteristics more obvious and makes it easy to analyze and compare different signals. CNN focuses on local feature extraction, which can capture local information such as the texture, edge, and shape of the image, while the attention mechanism in ViT can effectively capture the global structure and long-range dependencies present in the image. Two fully connected layers with a ReLU function are used to obtain the predicted tool wear values. The experimental results on the PHM 2010 dataset demonstrate that the proposed model has higher prediction accuracy than the CNN model or ViT model alone, as well as outperforms several existing methods in accurately predicting tool wear. The proposed prediction method can also be applied to predict tool wear in other machining fields.

Breast cancer is ranked as the second most common cancer among women globally, highlighting the critical need for precise and early detection methods. Our research introduces a novel approach for classifying benign and malignant breast ultrasound images. We leverage advanced deep learning methodologies, mainly focusing on the vision transformer (ViT) model. Our method distinctively features progressive fine‐tuning, a tailored process that incrementally adapts the model to the nuances of breast tissue classification. Ultrasound imaging was chosen for its distinct benefits in medical diagnostics. This modality is noninvasive and cost‐effective and demonstrates enhanced specificity, especially in dense breast tissues where traditional methods may struggle. Such characteristics make it an ideal choice for the sensitive task of breast cancer detection. Our extensive experiments utilized the breast ultrasound images dataset, comprising 780 images of both benign and malignant breast tissues. The dataset underwent a comprehensive analysis using several pretrained deep learning models, including VGG16, VGG19, DenseNet121, Inception, ResNet152V2, DenseNet169, DenseNet201, and the ViT. The results presented were achieved without employing data augmentation techniques. The ViT model demonstrated robust accuracy and generalization capabilities with the original dataset size, which consisted of 637 images. Each model's performance was meticulously evaluated through a robust 10‐fold cross‐validation technique, ensuring a thorough and unbiased comparison. Our findings are significant, demonstrating that the progressive fine‐tuning substantially enhances the ViT model's capability. This resulted in a remarkable accuracy of 94.49% and an AUC score of 0.921, significantly higher than models without fine‐tuning. These results affirm the efficacy of the ViT model and highlight the transformative potential of integrating progressive fine‐tuning with transformer models in medical image classification tasks. The study solidifies the role of such advanced methodologies in improving early breast cancer detection and diagnosis, especially when coupled with the unique advantages of ultrasound imaging. [ABSTRACT FROM AUTHOR]

This study introduces Hybrid-DC, a hybrid deep-learning model integrating ResNet-50 and Vision Transformer (ViT) for high-accuracy steel surface defect classification. Hybrid-DC leverages ResNet-50 for efficient feature extraction at both low and high levels and utilizes ViT's global context learning to enhance classification precision. A unique hybrid attention layer and an attention fusion mechanism enable Hybrid-DC to adapt to the complex, variable patterns typical of steel surface defects. Experimental evaluations demonstrate that Hybrid-DC achieves substantial accuracy improvements and significantly reduced loss compared to traditional models like MobileNetV2 and ResNet, with a validation accuracy reaching 0.9944. The results suggest that this model, characterized by rapid convergence and stable learning, can be applied for real-time quality control in steel manufacturing and other high-precision industries, enhancing automated defect detection efficiency. [ABSTRACT FROM AUTHOR]

Fire detection is a critical task in environmental monitoring and disaster prevention, with traditional methods often limited in their ability to detect fire and smoke in real time over large areas. The rapid identification of fire and smoke in both indoor and outdoor environments is essential for minimizing damage and ensuring timely intervention. In this paper, we propose a novel approach to fire and smoke detection by integrating a vision transformer (ViT) with the YOLOv5s object detection model. Our modified model leverages the attention-based feature extraction capabilities of ViTs to improve detection accuracy, particularly in complex environments where fires may be occluded or distributed across large regions. By replacing the CSPDarknet53 backbone of YOLOv5s with ViT, the model is able to capture both local and global dependencies in images, resulting in more accurate detection of fire and smoke under challenging conditions. We evaluate the performance of the proposed model using a comprehensive Fire and Smoke Detection Dataset, which includes diverse real-world scenarios. The results demonstrate that our model outperforms baseline YOLOv5 variants in terms of precision, recall, and mean average precision (mAP), achieving a mAP@0.5 of 0.664 and a recall of 0.657. The modified YOLOv5s with ViT shows significant improvements in detecting fire and smoke, particularly in scenes with complex backgrounds and varying object scales. Our findings suggest that the integration of ViT as the backbone of YOLOv5s offers a promising approach for real-time fire detection in both urban and natural environments. [ABSTRACT FROM AUTHOR]

In the digital era, social media has become a cornerstone for educational institutions, driving public engagement and enhancing institutional communication. This study utilizes AI-driven image processing and Web-enhanced Deep Learning (DL) techniques to investigate the effectiveness of King Faisal University's (KFU's) social media strategy as a case study, particularly on Twitter. By categorizing images into five primary event management categories and subcategories, this research provides a robust framework for assessing the social media content generated by KFU's administrative units. Seven advanced models were developed, including an innovative integration of Vision Transformers (ViTs) with Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, VGG16, and ResNet. The AI-driven ViT-CNN hybrid model achieved perfect classification accuracy (100%), while the "Development and Partnerships" category demonstrated notable accuracy (98.8%), underscoring the model's unparalleled efficacy in strategic content classification. This study offers actionable insights for the optimization of AI-driven digital communication strategies and Web-enhanced data collection processes, aligning them with national development goals and Saudi Arabia's Vision 2030, thereby showcasing the transformative power of DL in event-centric management and the broader higher education landscape. [ABSTRACT FROM AUTHOR]

The three-dimensional reconstruction of forests is crucial in remote sensing technology, ecological monitoring, and forestry management, as it yields precise forest structure and tree parameters, providing essential data support for forest resource management, evaluation, and sustainable development. Nevertheless, forest 3D reconstruction now encounters obstacles including higher equipment costs, reduced data collection efficiency, and complex data processing. This work introduces a unique deep learning model, CPH-Fmnet, designed to enhance the accuracy and efficiency of 3D reconstruction in intricate forest environments. CPH-Fmnet enhances the FPN Encoder-Decoder Architecture by meticulously incorporating the Channel Attention Mechanism (CA), Path Aggregation Module (PA), and High-Level Feature Selection Module (HFS), alongside the integration of the pre-trained Vision Transformer (ViT), thereby significantly improving the model's global feature extraction and local detail reconstruction abilities. We selected three representative sample plots in Haidian District, Beijing, China, as the study area and took forest stand sequence photos with an iPhone for the research. Comparative experiments with the conventional SfM + MVS and MVSFormer models, along with comprehensive parameter extraction and ablation studies, substantiated the enhanced efficacy of the proposed CPH-Fmnet model in addressing difficult circumstances such as intricate occlusions, poorly textured areas, and variations in lighting. The test results show that the model does better on a number of evaluation criteria. It has an RMSE of 1.353, an MAE of only 5.1%, an r value of 1.190, and a forest reconstruction rate of 100%, all of which are better than current methods. Furthermore, the model produced a more compact and precise 3D point cloud while accurately determining the properties of the forest trees. The findings indicate that CPH-Fmnet offers an innovative approach for forest resource management and ecological monitoring, characterized by cheap cost, high accuracy, and high efficiency. [ABSTRACT FROM AUTHOR]

Polarimetric high-resolution range profile (HRRP), with its rich polarimetric and spatial information, has become increasingly important in radar automatic target recognition (RATR). This study proposes an interpretable target-aware vision Transformer (ITAViT) for polarimetric HRRP target recognition with a novel attention loss. In ITAViT, we initially fuse the polarimetric features and the amplitude of polarimetric HRRP with a polarimetric preprocessing layer (PPL) to obtain the feature map as the input of the subsequent network. The vision Transformer (ViT) is then used as the backbone to automatically extract both local and global features. Most importantly, we introduce a novel attention loss to optimize the alignment between the attention map and the HRRP span. Thus, it can improve the difference between the target and the background, and enable the model to more effectively focus on real target areas. Experiments on a simulated X-band dataset demonstrate that our proposed ITAViT outperforms comparative models under various experimental conditions. Ablation studies highlight the effectiveness of polarimetric preprocessing and attention loss. Furthermore, the visualization of the self-attention mechanism suggests that attention loss enhances the interpretability of the network. [ABSTRACT FROM AUTHOR]

Background: Skin cancer is one of the highly occurring diseases in human life. Early detection and treatment are the prime and necessary points to reduce the malignancy of infections. Deep learning techniques are supplementary tools to assist clinical experts in detecting and localizing skin lesions. Vision transformers (ViT) based on image segmentation classification using multiple classes provide fairly accurate detection and are gaining more popularity due to legitimate multiclass prediction capabilities. Materials and methods: In this research, we propose a new ViT Gradient‐Weighted Class Activation Mapping (GradCAM) based architecture named ViT‐GradCAM for detecting and classifying skin lesions by spreading ratio on the lesion's surface area. The proposed system is trained and validated using a HAM 10000 dataset by studying seven skin lesions. The database comprises 10 015 dermatoscopic images of varied sizes. The data preprocessing and data augmentation techniques are applied to overcome the class imbalance issues and improve the model's performance. Result: The proposed algorithm is based on ViT models that classify the dermatoscopic images into seven classes with an accuracy of 97.28%, precision of 98.51, recall of 95.2%, and an F1 score of 94.6, respectively. The proposed ViT‐GradCAM obtains better and more accurate detection and classification than other state‐of‐the‐art deep learning‐based skin lesion detection models. The architecture of ViT‐GradCAM is extensively visualized to highlight the actual pixels in essential regions associated with skin‐specific pathologies. Conclusion: This research proposes an alternate solution to overcome the challenges of detecting and classifying skin lesions using ViTs and GradCAM, which play a significant role in detecting and classifying skin lesions accurately rather than relying solely on deep learning models. [ABSTRACT FROM AUTHOR]

Bridges are crucial components of infrastructure networks that facilitate national connectivity and development. According to the National Bridge Inventory (NBI) and the Federal Highway Administration (FHWA), the cost to repair U.S. bridges was recently estimated at approximately USD 164 billion. Traditionally, bridge inspections are performed manually, which poses several challenges in terms of safety, efficiency, and accessibility. To address these issues, this research study introduces a method using Unmanned Aerial Systems (UASs) to help automate the inspection process. This methodology employs UASs to capture visual images of a concrete bridge deck, which are then analyzed using advanced machine learning techniques of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) to detect damage and delamination. A case study on the Beyer Road Concrete Bridge in Michigan is used to demonstrate the developed methodology. The findings demonstrate that the ViT model outperforms the CNN in detecting bridge deck damage, with an accuracy of 97%, compared to 92% for the CNN. Additionally, the ViT model showed a precision of 96% and a recall of 97%, while the CNN model achieved a precision of 93% and a recall of 61%. This technology not only enhances the maintenance of bridges but also significantly reduces the risks associated with traditional inspection methods. [ABSTRACT FROM AUTHOR]

Arabic handwriting recognition and conversion are crucial for financial operations, particularly for processing handwritten amounts on cheques and financial documents. Compared to other languages, research in this area is relatively limited, especially concerning Arabic. This study introduces an innovative AI-driven method for simultaneously recognizing and converting Arabic handwritten legal amounts into numerical courtesy forms. The framework consists of four key stages. First, a new dataset of Arabic legal amounts in handwritten form (".png" image format) is collected and labeled by natives. Second, a YOLO-based AI detector extracts individual legal amount words from the entire input sentence images. Third, a robust hybrid classification model is developed, sequentially combining ensemble Convolutional Neural Networks (CNNs) with a Vision Transformer (ViT) to improve the prediction accuracy of single Arabic words. Finally, a novel conversion algorithm transforms the predicted Arabic legal amounts into digital courtesy forms. The framework's performance is fine-tuned and assessed using 5-fold cross-validation tests on the proposed novel dataset, achieving a word level detection accuracy of 98.6% and a recognition accuracy of 99.02% at the classification stage. The conversion process yields an overall accuracy of 90%, with an inference time of 4.5 s per sentence image. These results demonstrate promising potential for practical implementation in diverse Arabic financial systems. [ABSTRACT FROM AUTHOR]

Wetland mapping is a critical component of environmental monitoring, requiring advanced techniques to accurately represent the complex land cover patterns and subtle class differences innate in these ecosystems. This study aims to address these challenges by proposing CVTNet, a novel deep learning (DL) model that integrates convolutional neural networks (CNNs) and vision transformer (ViT) architectures. CVTNet uses channel attention (CA) and spatial attention (SA) mechanisms to enhance feature extraction from Sentinel-1 (S1) and Sentinel-2 (S2) satellite data. The primary goal of this model is to achieve a balanced trade-off between Precision and Recall, which is essential for accurate wetland mapping. The class-specific analysis demonstrated CVTNet's proficiency across diverse classes, including pasture, shrubland, urban, bog, fen, and water. Comparative analysis showed that CVTNet outperforms contemporary algorithms such as Random Forest (RF), ViT, multi-layer perceptron mixer (MLP-mixer), and hybrid spectral net (HybridSN) classifiers. Additionally, the attention mechanism (AM) analysis and sensitivity analysis highlighted the crucial role of CA, SA, and ViT in focusing the model's attention on critical regions, thereby improving the mapping of wetland regions. Despite challenges at class boundaries, particularly between bog and fen, and misclassifications of swamp pixels, CVTNet presents a solution for wetland mapping. [ABSTRACT FROM AUTHOR]

Image denoising is a fundamental problem in computer vision (CV). Vision Transformer (ViT) is an improvement in CV after convolutional neural networks (CNNs). Recently, it has been demonstrated that the RepVGG network of the structural re-parameterization performs well in image tasks. Experiments indicate that the ViT-based uniformer transformer network has successfully balanced local and global information. As image denoising tasks require more local and global information about the image, a novel image denoising model named structural Re-parameterization Uniformer Transformer-UNet (Rep-UUNet) is proposed in this paper. The model is structural and re-parameterized the structure of the Uniformer Transformer network and uses the UNet skip connections to reconstruct the output image. To complete image downstream tasks, the RepVGG model which utilizes the image local information is used. Indicators such as PSNR, SSIM and others are used to assess the performance of image denoising. Experimental results demonstrate that our Rep-UUNet network model outperforms other five models. [ABSTRACT FROM AUTHOR]

The vision transformer(ViT), pre-trained on large datasets, outperforms convolutional neural networks (CNN) in computer vision(CV). However, if not pre-trained, the transformer architecture doesn’t work well on small datasets and is surpassed by CNN. Through analysis, we found that:(1) the division and processing of tokens in the ViT discard the marginalized information between token. (2) the isolated multi-head self-attention (MSA) lacks prior knowledge. (3) the local inductive bias capability of stacked transformer block is much inferior to that of CNN. We propose a novel architecture for small data paradigms without pre-training, named Add-Vit, which uses progressive tokenization with feature supplementation in patch embedding. The model’s representational ability is enhanced by using a convolutional prediction module shortcut to connect MSA and capture local features as additional representations of the token. Without the need for pre-training on large datasets, our best model achieved 81.25 % accuracy when trained from scratch on the CIFAR-100. [ABSTRACT FROM AUTHOR]

In the realm of autonomous vehicle technology, the multimodal vehicle detection network (MVDNet) represents a significant leap forward, particularly in the challenging context of weather conditions. This paper focuses on the enhancement of MVDNet through the integration of a multi-head attention layer, aimed at refining its performance. The integrated multi-head attention layer in the MVDNet model is a pivotal modification, advancing the network's ability to process and fuse multimodal sensor information more efficiently. The paper validates the improved performance of MVDNet with multi-head attention through comprehensive testing, which includes a training dataset derived from the Oxford Radar RobotCar. The results clearly demonstrate that the multi-head MVDNet outperforms the other related conventional models, particularly in the average precision (AP) of estimation, under challenging environmental conditions. The proposed multi-head MVDNet not only contributes significantly to the field of autonomous vehicle detection but also underscores the potential of sophisticated sensor fusion techniques in overcoming environmental limitations. [ABSTRACT FROM AUTHOR]

Background: In response to the escalating global concerns surrounding skin cancer, this study aims to address the imperative for precise and efficient diagnostic methodologies. Focusing on the intricate task of eight-class skin cancer classification, the research delves into the limitations of conventional diagnostic approaches, often hindered by subjectivity and resource constraints. The transformative potential of Artificial Intelligence (AI) in revolutionizing diagnostic paradigms is underscored, emphasizing significant improvements in accuracy and accessibility. Methods: Utilizing cutting-edge deep learning models on the ISIC2019 dataset, a comprehensive analysis is conducted, employing a diverse array of pre-trained ImageNet architectures and Vision Transformer models. To counteract the inherent class imbalance in skin cancer datasets, a pioneering “Naturalize” augmentation technique is introduced. This technique leads to the creation of two indispensable datasets—the Naturalized 2.4K ISIC2019 and groundbreaking Naturalized 7.2K ISIC2019 datasets—catalyzing advancements in classification accuracy. The “Naturalize” augmentation technique involves the segmentation of skin cancer images using the Segment Anything Model (SAM) and the systematic addition of segmented cancer images to a background image to generate new composite images. Results: The research showcases the pivotal role of AI in mitigating the risks of misdiagnosis and under-diagnosis in skin cancer. The proficiency of AI in analyzing vast datasets and discerning subtle patterns significantly augments the diagnostic prowess of dermatologists. Quantitative measures such as confusion matrices, classification reports, and visual analyses using Score-CAM across diverse dataset variations are meticulously evaluated. The culmination of these endeavors resulted in an unprecedented achievement—100% average accuracy, precision, recall, and F1-score—within the groundbreaking Naturalized 7.2K ISIC2019 dataset. Conclusion: This groundbreaking exploration highlights the transformative capabilities of AI-driven methodologies in reshaping the landscape of skin cancer diagnosis and patient care. The research represents a pivotal stride towards redefining dermatological diagnosis, showcasing the remarkable impact of AI-powered solutions in surmounting the challenges inherent in skin cancer diagnosis. The attainment of 100% across crucial metrics within the Naturalized 7.2K ISIC2019 dataset serves as a testament to the transformative capabilities of AI-driven approaches in reshaping the trajectory of skin cancer diagnosis and patient care. This pioneering work paves the way for a new era in dermatological diagnostics, heralding the dawn of unprecedented precision and efficacy in the identification and classification of skin cancers.

Diabetic retinopathy (DR) is one of the most common causes of visual impairment worldwide and requires reliable automated detection methods. Numerous research efforts have developed various conventional methods for early detection of DR. Research in the field of DR remains insufficient, indicating the potential for advances in diagnosis. In this paper, a hybrid model (HybridFusionNet) that integrates vision transformer (VIT) and attention processes is presented. It improves classification in the binary (Bcl) and multi-class (Mcl) stages by utilizing deep features from the DR stages. As a result, both the SAN and VIT models improve the recognition accuracy (Acc) in both stages.The HybridFusionNet mechanism achieves a competitive improvement in multi-stage and binary stages, which is Acc in Bcl and Mcl, with 91% and 99%, respectively. This illustrates that this model is suitable for a better diagnosis of DR.

Cervical cancer is a major health concern worldwide, highlighting the urgent need for better early detection methods to improve outcomes for patients. In this study, we present a novel digital pathology classification approach that combines Low-Rank Adaptation (LoRA) with the Vision Transformer (ViT) model. This method is aimed at making cervix type classification more efficient through a deep learning classifier that does not require as much data. The key innovation is the use of LoRA, which allows for the effective training of the model with smaller datasets, making the most of the ability of ViT to represent visual information. This approach performs better than traditional Convolutional Neural Network (CNN) models, including Residual Networks (ResNets), especially when it comes to performance and the ability to generalize in situations where data are limited. Through thorough experiments and analysis on various dataset sizes, we found that our more streamlined classifier is highly accurate in spotting various cervical anomalies across several cases. This work advances the development of sophisticated computer-aided diagnostic systems, facilitating more rapid and accurate detection of cervical cancer, thereby significantly enhancing patient care outcomes. [ABSTRACT FROM AUTHOR]

Natural hazards are considered to have a strong link with climate change and human activities. With the rapid advancements in remote sensing technology, real-time monitoring and high-resolution remote-sensing images have become increasingly available, which provide precise details about the Earth's surface and enable prompt updates to support risk identification and management. This paper proposes a new network framework with Transformer architecture and a Residual network for detecting the changes in high-resolution remote-sensing images. The proposed model is trained using remote-sensing images from Shandong and Anhui Provinces of China in 2021 and 2022 while one district in 2023 is used to test the prediction accuracy. The performance of the proposed model is evaluated by using five matrices and further compared to both convention-based and attention-based models. The results demonstrated that the proposed structure integrates the great capability of conventional neural networks for image feature extraction with the ability to obtain global context from the attention mechanism, resulting in significant improvements in balancing positive sample identification while avoiding false positives in complex image change detection. Additionally, a toolkit supporting image preprocessing is developed for practical applications. [ABSTRACT FROM AUTHOR]

Audio-visual speech synthesis (AVSS) has garnered attention in recent years for its utility in the realm of audio-visual learning. AVSS transforms one speaker's speech into another's audio-visual stream while retaining linguistic content. This approach extends existing AVSS methods by first modifying vocal features from the source to the target speaker, akin to voice conversion (VC), and then synthesizing the audio-visual stream for the target speaker, termed audio-visual synthesis (AVS). In this work, a novel AVSS approach is proposed using vision transformer (ViT)-based Autoencoders (AEs), enriched with a combination of cycle consistency and reconstruction loss functions, with the aim of enhancing synthesis quality. Leveraging ViT's attention mechanism, this method effectively captures spectral and temporal features from input speech. The combination of cycle consistency and reconstruction loss improves synthesis quality and aids in preserving essential information. The proposed framework is trained and tested on benchmark datasets, and compared extensively with state-of-the-art (SOTA) methods. The experimental results demonstrate the superiority of the proposed approach over existing SOTA models, in terms of quality and intelligibility for AVSS, indicating the potential for real-world applications. [ABSTRACT FROM AUTHOR]

In today's AI-driven era, deep learning (DL) algorithms play a crucial role in automatically detecting life-threatening skin cancers, thereby significantly enhancing survival rates. It makes skin cancer detection using DL algorithms an exciting area of exploration. While much of the prior research has focused on single-model approaches, combining ensembles of multiple models can enhance classification accuracy. Previous studies mainly relied on deep convolutional neural networks (DCNNs), which have limitations in capturing global features. Recent advancements have introduced capsule networks (Caps-Net) and vision transformers (ViT) for more effective feature extraction. In our study, we harness the power of DCNN, Caps-Net, and ViT frameworks to extract diverse image embeddings. These obtained feature vectors work as input data to train an ensemble model based on a majority voting mechanism. This ensemble model consists of five machine-learning models, including Random Forest, XGBoost, SVM, KNN, and logistic regression. The incorporation of this ensemble mechanism leads to a significant improvement in the overall model's performance. It is noteworthy that the proposed ensemble model serves as a lightweight model, which achieves an impressive accuracy of 91.6% when considering the melanoma skin cancer dataset. These results underscore the superiority of our proposed mechanism over individual state-of-the-art (SOTA) models. [ABSTRACT FROM AUTHOR]

In today's digital age, the rapid spread of fake news is a pressing concern. Fake news, whether intentional or inadvertent, manipulates public sentiment and threatens the integrity of online information. To address this, effective detection and prevention methods are vital. Detecting and addressing this multimodal fake news is an intricate challenge as, unlike traditional news articles that predominantly rely on textual content, multimodal fake news leverages the persuasive power of visual elements, making its identification a formidable task. Manipulated images can significantly sway individuals' perceptions and beliefs, making the detection of such deceptive content complex. Our research introduces an innovative approach to multimodal fake news identification by presenting a fusion-based methodology that harnesses the capabilities of Text Graph Convolutional Neural Networks (TextGCN) and Vision Transformers (ViT) to effectively utilise both text and image modalities. The proposed Methodology starts with preprocessing textual content using TextGCN, allowing for the capture of intricate structural dependencies among words and phrases. Simultaneously, visual features are extracted from associated images using ViT. Through a fusion mechanism, these modalities seamlessly integrate, yielding superior embeddings. The primary contributions encompass an in-depth exploration of multimodal fake news detection through a fusion-based approach. What sets our approach apart from existing techniques is its integration of graph-based feature extraction through TextGCN. While previous methods predominantly rely on text or image features, our approach harnesses the additional semantic information and intricate relationships within a graph structure, in addition to image embeddings. This enables our method to capture more comprehensive understanding of the data, resulting in increased accuracy and reliability. Our experiments demonstrate the exceptional performance of our fusion-based approach, which leverages multiple modalities and incorporates graph-based representations and semantic relationships. This method outperformed single modalities of text or image, achieving an impressive accuracy of 94.17% using a neural network after fusion. By seamlessly integrating graph-based representations and semantic relationships, our fusion-based technique represents a significant stride in addressing the challenges posed by multimodal fake news. [ABSTRACT FROM AUTHOR]

Ultrasound (US) has become a widely used imaging modality in clinical practice, characterized by its rapidly evolving technology, advantages, and unique challenges, such as a low imaging quality and high variability. There is a need to develop advanced automatic US image analysis methods to enhance its diagnostic accuracy and objectivity. Vision transformers, a recent innovation in machine learning, have demonstrated significant potential in various research fields, including general image analysis and computer vision, due to their capacity to process large datasets and learn complex patterns. Their suitability for automatic US image analysis tasks, such as classification, detection, and segmentation, has been recognized. This review provides an introduction to vision transformers and discusses their applications in specific US image analysis tasks, while also addressing the open challenges and potential future trends in their application in medical US image analysis. Vision transformers have shown promise in enhancing the accuracy and efficiency of ultrasound image analysis and are expected to play an increasingly important role in the diagnosis and treatment of medical conditions using ultrasound imaging as technology progresses. [ABSTRACT FROM AUTHOR]

Background: In response to the escalating global concerns surrounding skin cancer, this study aims to address the imperative for precise and efficient diagnostic methodologies. Focusing on the intricate task of eight-class skin cancer classification, the research delves into the limitations of conventional diagnostic approaches, often hindered by subjectivity and resource constraints. The transformative potential of Artificial Intelligence (AI) in revolutionizing diagnostic paradigms is underscored, emphasizing significant improvements in accuracy and accessibility. Methods: Utilizing cutting-edge deep learning models on the ISIC2019 dataset, a comprehensive analysis is conducted, employing a diverse array of pre-trained ImageNet architectures and Vision Transformer models. To counteract the inherent class imbalance in skin cancer datasets, a pioneering "Naturalize" augmentation technique is introduced. This technique leads to the creation of two indispensable datasets—the Naturalized 2.4K ISIC2019 and groundbreaking Naturalized 7.2K ISIC2019 datasets—catalyzing advancements in classification accuracy. The "Naturalize" augmentation technique involves the segmentation of skin cancer images using the Segment Anything Model (SAM) and the systematic addition of segmented cancer images to a background image to generate new composite images. Results: The research showcases the pivotal role of AI in mitigating the risks of misdiagnosis and under-diagnosis in skin cancer. The proficiency of AI in analyzing vast datasets and discerning subtle patterns significantly augments the diagnostic prowess of dermatologists. Quantitative measures such as confusion matrices, classification reports, and visual analyses using Score-CAM across diverse dataset variations are meticulously evaluated. The culmination of these endeavors resulted in an unprecedented achievement—100% average accuracy, precision, recall, and F1-score—within the groundbreaking Naturalized 7.2K ISIC2019 dataset. Conclusion: This groundbreaking exploration highlights the transformative capabilities of AI-driven methodologies in reshaping the landscape of skin cancer diagnosis and patient care. The research represents a pivotal stride towards redefining dermatological diagnosis, showcasing the remarkable impact of AI-powered solutions in surmounting the challenges inherent in skin cancer diagnosis. The attainment of 100% across crucial metrics within the Naturalized 7.2K ISIC2019 dataset serves as a testament to the transformative capabilities of AI-driven approaches in reshaping the trajectory of skin cancer diagnosis and patient care. This pioneering work paves the way for a new era in dermatological diagnostics, heralding the dawn of unprecedented precision and efficacy in the identification and classification of skin cancers. [ABSTRACT FROM AUTHOR]

Currently, convolutional neural networks (CNNs) and vision transformers (ViTs) are widely adopted as the predominant neural network architectures for remote sensing image scene classification. Although CNNs have lower computational complexity, ViTs have a higher performance ceiling, making both suitable as backbone networks for remote sensing scene classification tasks. However, remote sensing imagery has high intraclass variation and interclass similarity, which poses a challenge for existing methods. To address this issue, we propose the class-aware self-distillation (CASD) framework. This framework uses an end-to-end distillation mechanism to mine class-aware knowledge, effectively reducing the impact of significant intraclass variation and interclass similarity in remote sensing imagery. Specifically, our approach involves constructing pairs of images: similar pairs consisting of images belonging to the same class, and dissimilar pairs consisting of images from different classes. We then apply a distillation loss that we designed, which distills the corresponding probability distributions to ensure that the distributions of similar pairs become more consistent, and those of dissimilar pairs become more distinct. In addition, the enforced learnable $\alpha$ added to the distillation loss further amplifies the network's ability to comprehend class-aware knowledge. The experiment section demonstrates that our method CASD outperforms other methods on four publicly available datasets. And the ablation experiments demonstrate the effectiveness of the method.

Pansharpening is a fundamental and crucial image processing task for many remote sensing applications, which generates a high-resolution multispectral image by fusing a low-resolution multispectral image and a high-resolution panchromatic image. Recently, vision transformers have been introduced into the pansharpening task for utilizing global contextual information. However, long-range and local dependencies modeling and multiscale feature learning are all essential to the pansharpening task. Learning and exploiting these various information raises a big challenge and limits the performance and efficiency of existing pansharpening methods. To solve this issue, we propose a pansharpening network based on multiscale embedding and dual attention transformers (MDPNet). Specifically, a multiscale embedding block is proposed to embed multiscale information of the images into vectors. Thus, transformers only need to process a multispectral embedding sequence and a panchromatic embedding sequence to efficiently use multiscale information. Furthermore, an additive hybrid attention transformer is proposed to fuse the embedding sequences in an additive injection manner. Finally, a channel self-attention transformer is proposed to utilize channel correlations for high-quality detail generation. Experiments over QuickBird and WorldView-3 datasets demonstrate the proposed MDPNet outperforms state-of-the-art methods visually and quantitatively with low running time. Ablation studies further verify the effectiveness of the proposed multiscale embedding and transformers in pansharpening.

Seizure prediction are necessary for epileptic patients. The global spatial interactions among channels, and long-range temporal dependencies play a crucial role in seizure onset prediction. In addition, it is necessary to search for seizure prediction features in a vast space to learn new generalized feature representations. Many previous deep learning algorithms have achieved some results in automatic seizure prediction. However, most of them do not consider global spatial interactions among channels and long-range temporal dependencies together, and only learn the feature representation in the deep space. To tackle these issues, in this study, an novel bi-level programming seizure prediction model, B2-ViT Net, is proposed for learning the new generalized spatio-temporal long-range correlation features, which can characterize the global interactions among channels in spatial, and long-range dependencies in temporal required for seizure prediction. In addition, the proposed model can comprehensively learn generalized seizure prediction features in a vast space due to its strong deep and broad feature search capabilities. Sufficient experiments are conducted on two public datasets, CHB-MIT and Kaggle datasets. Compared with other existing methods, our proposed model has shown promising results in automatic seizure prediction tasks, and provides a certain degree of interpretability.

The Vision Transformer (ViT) model serves as a powerful model to capture and comprehend global information, particularly when trained on extensive datasets. Conversely, the Convolutional Neural Network (CNN) model is beneficial to training with small datasets for retaining essential local information. Inspired by these properties of ViT and CNN models, we introduce a hybrid framework that smoothly increases the cross-domain generalization in Semi-supervised Domain Adaptation (SSDA). To achieve this goal, we first train the ViT model on abundant labeled source data, while the CNN model is trained on a few labeled target samples. Then, these models dynamically exchange their knowledge for potential generalization to unlabeled target data via the proposed method, named Dual Dynamic Consistency Regularization (D2CR). Specifically, the ViT model provides its pseudo labels to update the global perspective for the CNN model. Similarly, the CNN model offers pseudo labels to complement the local perspective for the ViT model. The previous methods use a fixed threshold algorithm for the pseudo-labeling process. However, we utilize the dynamic threshold strategy to create pseudo labels for the bi-directional consistency regularization learning between the ViT and CNN models. We verify our approach across several SSDA benchmark datasets. The outstanding experimental results provide strong evidence of the effectiveness and superiority of our approach over previous state-of-the-art SSDA methods.

With the development of deep learning technology, the application of convolutional neural network (CNN) and vision transformer (ViT) for polarimetric synthetic aperture radar (PolSAR) image classification has been deepened. However, the PolSAR image has very rich information due to its special data form, which makes it difficult for the existing single network structure to comprehensively extract such effective information. In addition, deep learning methods require a large amount of data for training, whereas PolSAR labeled data is scarce and difficult to obtain. Therefore, a multigranularity hybrid CNN-ViT model based on external tokens and cross-attention is proposed for PolSAR image classification. First of all, CNN is able to learn local features very well. Thus, a CNN-based external feature extractor is designed to extract local features from the PolSAR image. Then, ViT can focus on global features. So, a multigranularity attention structure is constructed for extracting global information at multiple scales. With these two modules, the model can fully access the feature information contained in PolSAR images, which is more advantageous than a single network structure. Next, to further utilize these features, a cross-attention feature fusion module is built for fusing global–local information of different granularities. Finally, by connecting with the softmax classifier, the network outputs the final prediction results. Experimental results on three benchmark datasets show that the present method using a small amount of labeled data for training also achieves the highest level of classification among the compared methods.

The multitemporal polarimetric SAR (PolSAR) data contains the scattering change information during the growth of crops. However, the current classification methods usually directly use the addition of features extracted at single-temporal or use the temporal and spatial variations of certain features, not really exploring the complete scattering variation information. The specific data representation models for multitemporal PolSAR data should combine time with polarimetry to characterize the scattering variations. However, the characterization and utilization of such kind of models are inadequate. In this article, we construct data representation model based on the power form of coherence matrix to comprehensively represent all kinds of scattering mechanism variation, which is full-rank positive semidefinite Hermitian matrix. We extract new time-variant scattering features and design vision transformer classifier accordingly for crop classification. Experiment results on RADARSAT-2 datasets show that the proposed power representation model outperforms other models.

Ensuring the security and safety of passengers and cargo through effective baggage screening presents a critical challenge in high-traffic environments like airports, where traditional manual processes are affected by high error rates, fatigue among security personnel, and privacy concerns. These issues underscore the urgent need for sophisticated automated systems capable of accurately detecting concealed threats. Advancements in deep learning have introduced more refined approaches to baggage screening, yet they struggle with capturing the global context of threats and challenges presented by clutter, concealment, and occlusion. Addressing these issues, this paper presents the Multi-Scale Hierarchical Transformer for X-ray Detection (MHT-X), a novel framework that surpasses conventional limitations by integrating a multi-scale contour mapping technique with a hierarchical feature extraction process using visual Transformers. MHT-X is distinctive in its ability to generate multi-level contour maps that unveil complex features of threat objects, facilitating their accurate identification. It employs a top-down approach for hierarchical feature processing, capturing vital spatial characteristics essential for precise threat localization through an efficient encoder-decoder mechanism. MHT-X represents a significant advancement in the detection of concealed and obstructed threat objects in X-ray baggage scans. By combining multi-scale contour maps with a hierarchical feature extraction process, it significantly enhances detection accuracy and outperforms traditional Convolutional Neural Network (CNN) models by effectively capturing the global context of illegal items with advanced feature representation. The proposed framework is rigorously validated on two public datasets, dubbed CLCXray, and PIDray, where it outperforms the state-of-the-art (SOTA) methods by achieving the mean average precision score of 65.26%, and 78.19%, respectively.

Clouds in remote sensing images inevitably affect information extraction, which hinders the following analysis of satellite images. Hence, cloud detection is a necessary preprocessing procedure. However, most existing methods have numerous calculations and parameters. In this article, a lightweight convolutional neural network (CNN)-Transformer network, CD-CTFM, is proposed to solve the problem, which is based on encoder–decoder architecture and incorporates the attention mechanism. In the encoder part, we utilize a lightweight network combing CNN and Transformer as backbone, which is conducive to extracting local and global features simultaneously. The backbone of CD-CTFM also incorporates attention gate based on dark channel extraction module. Moreover, a lightweight feature pyramid module is designed to fuse multiscale features with contextual information. In the decoder part, a lightweight channel-spatial attention module is integrated into each skip connection between encoder and decoder to extract low-level features while suppressing irrelevant information without introducing many parameters. Finally, the proposed model is evaluated on two cloud datasets, 38-Cloud and MODIS. The results demonstrate that CD-CTFM achieves comparable accuracy as the state-of-art methods and outperforms in terms of efficiency.

The scattering properties of targets in polarimetric synthetic aperture radar (PolSAR) images are directly influenced by the targets' orientations, as the scattering properties from the same target with different orientations can be very different. This interpretation diversity caused by the target orientations is one of the primary technical bottlenecks in PolSAR image interpretation. In this article, a 3-D convolutional vision transformer (3-D-Conv-ViT) is proposed to describe the relationship between polarimetric coherent matrices with different polarization orientation angles (POAs) for PolSAR image classification and change detection. First, 3-D convolutional neural networks are used to capture the high-level feature representations of the polarimetric coherent matrix sequence. Second, a new Rotation-3-D-ViT block is proposed to learn the local and global representations of the high-level feature maps. The self-attention mechanism in the ViT can express the regularity of polarimetric coherent matrices with different POAs and improve the PolSAR image interpretation performance. Third, combined with different classifiers, the proposed 3-D-Conv-ViT can be applied to both PolSAR image classification and change detection. Experiments on real PolSAR image datasets demonstrate that the proposed method can overcome the problem of the interpretation ambiguity caused by the target orientation. The classification accuracies of the proposed method can reach 94.01%–99.48%, and the change detection accuracies can reach 93.84%–96.86%.

In this paper, we propose a method to improve the accuracy of speech emotion recognition (SER) by using vision transformer (ViT) to attend to the correlation of frequency (y-axis) with time (x-axis) in spectrogram and transferring positional information between ViT through knowledge transfer. The proposed method has the following originality i) We use vertically segmented patches of log-Mel spectrogram to analyze the correlation of frequencies over time. This type of patch allows us to correlate the most relevant frequencies for a particular emotion with the time they were uttered. ii) We propose the use of image coordinate encoding, an absolute positional encoding suitable for ViT. By normalizing the x, y coordinates of the image to −1 to 1 and concatenating them to the image, we can effectively provide valid absolute positional information for ViT. iii) Through feature map matching, the locality and location information of the teacher network is effectively transmitted to the student network. Teacher network is a ViT that contains locality of convolutional stem and absolute position information through image coordinate encoding, and student network is a structure that lacks positional encoding in the basic ViT structure. In feature map matching stage, we train through the mean absolute error (L1 loss) to minimize the difference between the feature maps of the two networks. To validate the proposed method, three emotion datasets (SAVEE, EmoDB, and CREMA-D) consisting of speech were converted into log-Mel spectrograms for comparison experiments. As a result of the experiment, the proposed method achieved 99.47%, 99.76%, and 95.24% accuracy, significantly exceeding the state-of-the-art methods, with much less computational complexity on three emotion datasets.

By improving image quality and resolution, Single Image Super-Resolution (SISR) models help advance understanding of underwater environments. Super-resolution techniques hold promise in addressing these issues by enhancing image details. The transformer architecture has recently gained considerable popularity in low-level vision tasks, including image super-resolution (SR). In this paper, we present a novel multi-distillation algorithm based on the swin transformer and wavelet transform (KDSWT) for underwater image super-resolution. A wavelet block is added to the swin transformer to alleviate the loss of detailed features because wavelet transforms can preserve the high-frequency components, subtle variations, and localized features that contribute to the overall richness in images. Based on a large dataset of high-resolution underwater images, a ViT-based student model is trained to learn the intricate features of underwater images from the teacher model. Using knowledge distillation, the student model learns to upscale input images and capture the domain-specific features essential to understanding underwater scenes. To address the computational cost issue in ViT, we introduce a lightweight network training strategy that combines multi-anchor distillation and progressive learning, enabling the lightweight network to achieve exceptional performance and the fastest inference time compared to recent lightweight state-of-art methods. Our experimental results show that KDSWT lightweight (student) significantly improves peak signal-to-noise ratio (PSNR) over the (original) teacher model while decreasing the number of parameters by 54% and the calculation cost by 22%. Concurrently, KDSWT-SR excels beyond state-of-the-art SR techniques on three benchmark underwater datasets, leading to noteworthy improvements in both PSNR and structural similarity index (SSIM).

In recent years, collaborative classification of multimodal data, e.g., hyperspectral image (HSI) and light detection and ranging (LiDAR), has been widely used to improve remote sensing image classification accuracy. However, existing fusion approaches for HSI and LiDAR suffer from limitations. Fusing the heterogeneous features of HSI and LiDAR proved to be challenging, leading to incomplete utilization of information for category representation. In addition, during the extraction of spatial features from HSI, the spectral and spatial information are often disjointed. It leads to the difficulty of fully exploiting the rich spectral information in hyperspectral data. To address these issues, we proposed a multimodal data fusion framework specifically designed for HSI and LiDAR fusion classification, called modality fusion vision transformer. We have designed a stackable modality fusion block as the core of our model. Specifically, these blocks mainly consist of multimodal cross-attention modules and spectral self-attention modules. The proposed novel multimodal cross-attention module for feature fusion addresses the issue of insufficient fusion of heterogeneous features from HSI and LiDAR for category representation. Compared to other cross-attention methods, it reduces the alignment requirements between modal feature spaces during cross-modal fusion. The spectral self-attention module can preserve spatial features while exploiting the rich spectral information and participating in the process of extracting spatial features from HSI. Ultimately, we achieve overall classification accuracies of 99.91%, 99.59%, and 96.98% on three benchmark datasets respectively, surpassing all state-of-the-art methods, demonstrating the stability and effectiveness of our model.

In recent decades, weakly supervised object localization (WSOL) has gained increasing attention in remote sensing. However, unlike optical images, remote sensing images (RSIs) often contain more complex scenes, which poses challenges for WSOL. Traditional convolutional neural network (CNN)-based WSOL methods are often limited by a small receptive field and yield unsatisfactory results. Transformer-based methods can obtain global perception, addressing the limitations of receptive fields in CNN-based methods, yet it may also introduce attention diffusion. To address the aforementioned problems, this article proposes a novel WSOL method based on an interpretable vision transformer (ViT), RPGV. We introduce a feature fusion enhancement module to obtain the saliency map that captures global information. Simultaneously, we solve the problem of discrete attention in the traditional ViT and eliminate local distortion in the feature map by introducing a global semantic screening module. We conduct comprehensive experiments on DIOR and HRRSD datasets, demonstrating the superior performance of our method compared to current state-of-the-art methods.

Emotional health significantly impacts physical and psychological well-being, with emotional imbalances and cognitive disorders leading to various health issues. Timely diagnosis of mental illnesses is crucial for preventing severe disorders and enhancing medical care quality. Physiological signals, such as Electrocardiograms (ECG) and Electroencephalograms (EEG), which reflect cardiac and neuronal activities, are reliable for emotion recognition as they are less susceptible to manipulation than physical signals. Galvanic Skin Response (GSR) is also closely linked to emotional states. Researchers have developed various methods for classifying signals to detect emotions. However, these signals are susceptible to noise and are inherently non-stationary, meaning they change constantly over time. Consequently, emotions can vary rapidly. Traditional techniques for analyzing physiological signals may not be adequate to study the dynamic changes in emotional states. This research introduces a deep learning approach using a combination of advanced signal processing and machine learning to analyze physiological signals for emotion recognition. We propose a CNN-Vision Transformer (CVT) based method with ensemble classification. The process involves decomposing signals into segments, removing noise, and extracting features using 1D CNN and Vision Transformers. These features are integrated into a single vector for classification by an ensemble of LSTM, ELM, and SVM classifiers. The outputs are then synthesized using Model Agnostic Meta Learning (MAML) to improve prediction accuracy. Validated on AMIGOS and DEAP datasets with 10-fold cross-validation, our method achieved accuracies up to 98.2%, sensitivity of 99.15%, and specificity of 99.53%, outperforming existing emotion charting techniques. This novel method provides significant improvements 3 to 4% in the accuracy, sensitivity, and specificity of emotion detection, leveraging physiological signals for comprehensive emotional assessments.

High resolution remote sensing imagery plays a crucial role in monitoring coastal wetlands. Coastal wetland landscapes exhibit diverse features, ranging from fragmented patches to expansive areas. Mainstream convolutional neural networks cannot effectively analyze spatial relationships among consecutive image elements. This limitation impedes their performance in accurately classifying coastal wetlands. In order to tackle the above issues, we propose a Vision Transformer based UNet (ViT-UNet) model. This model extracts wetland features from high resolution remote sensing images by sensing and optimizing multiscale features. To establish global dependencies, the Vision Transformer (ViT) is introduced to replace the convolutional layer in the UNet encoder. Simultaneously, the model incorporates a convolutional block attention module and a multiple hierarchies attention module to restore attentional features and reduce feature loss. In addition, a skip connection is added to the single-skip structure of the original UNet model. This connection simultaneously links the output of the entire transformer and internal attention features to the corresponding decoder level. This enhancement aims to furnish the decoder with comprehensive global information guidance. Finally, all the extracted feature information is fused using Bilinear Polymerization Pooling (BPP). The BPP assists the network in obtaining a more comprehensive and detailed feature representation. Experimental results on the Gaofen-1 dataset demonstrate that the proposed ViT-UNet method achieves a Precision score of 93.50$\%$, outperforming the original UNet model by 4.10$\%$. Compared with other state-of-the-art networks, ViT-UNet performs more accurately and finer in the extraction of wetland information in the Yellow River Delta.

In the digital era, social media has become a cornerstone for educational institutions, driving public engagement and enhancing institutional communication. This study utilizes AI-driven image processing and Web-enhanced Deep Learning (DL) techniques to investigate the effectiveness of King Faisal University’s (KFU’s) social media strategy as a case study, particularly on Twitter. By categorizing images into five primary event management categories and subcategories, this research provides a robust framework for assessing the social media content generated by KFU’s administrative units. Seven advanced models were developed, including an innovative integration of Vision Transformers (ViTs) with Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, VGG16, and ResNet. The AI-driven ViT-CNN hybrid model achieved perfect classification accuracy (100%), while the “Development and Partnerships” category demonstrated notable accuracy (98.8%), underscoring the model’s unparalleled efficacy in strategic content classification. This study offers actionable insights for the optimization of AI-driven digital communication strategies and Web-enhanced data collection processes, aligning them with national development goals and Saudi Arabia’s Vision 2030, thereby showcasing the transformative power of DL in event-centric management and the broader higher education landscape.

Agricultural productivity plays a vital role in global economic development and growth. When crops are affected by diseases, it adversely impacts a nation’s economic resources and agricultural output. Early detection of crop diseases can minimize losses for farmers and enhance production. In this study, we propose a new hybrid deep learning model, PLDPNet, designed to automatically predict potato leaf diseases. The PLDPNet framework encompasses image collection, pre-processing, segmentation, feature extraction and fusion, and classification. We employ an ensemble approach by combining deep features from two well-established models (VGG19 and Inception-V3) to generate more powerful features. The hybrid approach leverages the concept of vision transformers for final prediction. To train and evaluate PLDPNet, we utilize the public potato leaf dataset: early blight, late blight, and healthy leaves. Utilizing the strength of segmentation and fusion feature, the proposed approach achieves an overall accuracy of 98.66%, and F1-score of 96.33%. A comprehensive validation study is conducted using Apple (4 classes) and tomato (10 classes) datasets achieving impressive accuracies of 96.42% and 94.25%, respectively. These experimental findings confirm that the proposed hybrid framework provides more effective and accurate detection and prediction of potato crop diseases, making it a promising candidate for practical applications.

Seizure prediction are necessary for epileptic patients. The global spatial interactions among channels, and long-range temporal dependencies play a crucial role in seizure onset prediction. In addition, it is necessary to search for seizure prediction features in a vast space to learn new generalized feature representations. Many previous deep learning algorithms have achieved some results in automatic seizure prediction. However, most of them do not consider global spatial interactions among channels and long-range temporal dependencies together, and only learn the feature representation in the deep space. To tackle these issues, in this study, an novel bi-level programming seizure prediction model, B2-ViT Net, is proposed for learning the new generalized spatio-temporal long-range correlation features, which can characterize the global interactions among channels in spatial, and long-range dependencies in temporal required for seizure prediction. In addition, the proposed model can comprehensively learn generalized seizure prediction features in a vast space due to its strong deep and broad feature search capabilities. Sufficient experiments are conducted on two public datasets, CHB-MIT and Kaggle datasets. Compared with other existing methods, our proposed model has shown promising results in automatic seizure prediction tasks, and provides a certain degree of interpretability. [ABSTRACT FROM AUTHOR]