Search

Plant diseases significantly impact the quality and yield of agricultural products, leading to considerable economic losses. Most existing plant disease recognition systems are limited to identifying categories within the training set, which poses potential systemic risks. Rejecting unknown samples is crucial for the safety and reliability of practical applications. This study aims to harness the strong generalization capabilities of vision-language models to address plant disease anomaly detection. To this end, we comprehensively explore prompt tuning paradigms based on vision-language models. We observe that anomaly detection methods guided by textual concepts perform poorly in the fine-grained task of plant disease due to their focus on concept matching. We argue that visual information is crucial for anomaly detection in plant diseases. Therefore, we propose guiding the vision-language model with visual information to address this issue. Additionally, we find that utilizing the general knowledge extracted by the original vision-language model can further enhance anomaly detection performance. Extensive experimental results demonstrate that our method significantly improves the current baseline methods by incorporating visual information. Notably, deploying our method under vision-language prompt tuning achieved an AUROC score of 99.85% in the all-shot setting. Even in a challenge 2-shot setting, our approach achieves an AUROC score of 93.81%, significantly outperforming CoCoOp fine-tuned on the entire dataset (88.61%). We believe that our study will contribute to the community and, to fuel the field, our code will be released.

Zero-shot remote sensing scene classification aims to solve the scene classification problem on unseen categories and has attracted numerous research attention in the remote sensing field. Existing methods mostly use shallow networks for visual and semantic feature learning, and the semantic encoder networks are usually fixed during the zero-shot learning process, thus failing to capture powerful feature representations for classification. In this work, we introduced a vision-language model for remote sensing scene classification based on contrastive vision-language supervision. Our method is capable of learning semantic-aware visual representations using a contrastive vision-language loss in the embedding space. By pretraining on large-scale image–text datasets, our baseline method shows good transferring ability on remote sensing scenes. To enable model training in zero-shot settings, we introduced a pseudo-labeling technique that can automatically generate pseudo labels from unlabeled data. A curriculum learning strategy is developed to boost the performance of zero-shot remote sensing scene classification with multiple stages of model finetuning. We conducted experiments on four benchmark datasets and showed considerable performance improvement on both zero-shot and few-shot remote sensing scene classification. The proposed RS-CLIP method achieved a zero-shot classification accuracy of 95.94%, 95.97%, 85.76%, and 87.52% on the novel classes of UCM-21, WHU-RS19, NWPU-RESISC45, and AID-30 datasets respectively. Our code will be released at https://github.com/lx709/RS-CLIP.

The escalating demand for artificial intelligence (AI) systems that can monitor and supervise human errors and abnormalities in healthcare presents unique challenges. Recent advances in vision-language models reveal the challenges of monitoring AI by understanding both visual and textual concepts and their semantic correspondences. However, there has been limited success in the application of vision-language models in the medical domain. Current vision-language models and learning strategies for photographic images and captions call for a web-scale data corpus of image and text pairs which is not often feasible in the medical domain. To address this, we present a model named medical cross-attention vision-language model (Medical X-VL), which leverages key components to be tailored for the medical domain. The model is based on the following components: self-supervised unimodal models in medical domain and a fusion encoder to bridge them, momentum distillation, sentencewise contrastive learning for medical reports, and sentence similarity-adjusted hard negative mining. We experimentally demonstrated that our model enables various zero-shot tasks for monitoring AI, ranging from the zero-shot classification to zero-shot error correction. Our model outperformed current state-of-the-art models in two medical image datasets, suggesting a novel clinical application of our monitoring AI model to alleviate human errors. Our method demonstrates a more specialized capacity for fine-grained understanding, which presents a distinct advantage particularly applicable to the medical domain., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier B.V. All rights reserved.)