Search

With the increasing availability and openness of remote sensing (RS) data collected from diverse sensors, there has been a growing interest in multimodal RS data classification. Nowadays, in the area of deep learning, there is a paradigm shift with the rise of foundation models, which are trained on large-scale datasets and are adaptable to a wide range of downstream tasks. In this study, the potential and effectiveness of foundation models for multimodal RS data classification is investigated. The training datasets of foundation models and multimodal RS datasets are quite different, and therefore, it is difficult to use a pretrained foundation model for multimodal RS data classification directly. To mitigate this difficulty, this article proposes a foundation model adaptation (FMA) framework for multimodal RS data classification without fine-tuning the parameters. Specifically, two learnable modules, i.e., cross-spatial interaction module and cross-channel interaction module, are proposed to add to the foundation model for extracting multimodal-specific representations. The cross-spatial and cross-channel interaction modules extract the characteristics of unimodal features along the spatial dimension and channel dimension, respectively. To effectively tackle the disparities among various RS modalities, an alignment approach (FMA2) is further explored based on the FMA. The FMA2 describes dependencies between different modalities by establishing a coupling score function, which can further enhance classification performance. To demonstrate the effectiveness and superiority of the FMA framework, comprehensive experiments are conducted on three multimodal RS datasets, showing improvement over the advanced multimodal RS data classification image methods.

Recently, metric-based few-shot learning (FSL) methods have achieved good performance in the hyperspectral image (HSI) classification. However, existing methods suffer from two problems: over-reliance on image modality information leads to inaccurate prototype representation, where a prototype refers to the centroid of each class in the dataset, and the impact of redundant and noisy pixels on model discriminability is rarely considered. These problems result in insufficient discriminability of the model for the target domain. To address the above issues, we propose a cross-domain few-shot HSI classification framework with cross-modal alignment and supervised contrastive learning (CDFS-CASCL). It is well known that human visual learning greatly benefits from the input of various modal information such as vision, language, and video. Inspired by the way humans abstract image class concepts in language form and understand the essence of classes, we perform cross-modal alignment (CA) between similar image and text prototypes and use abstract text semantics to guide the model to learn semantic-related features with good generalization ability in images, so as to improve the accuracy of image prototypes representation of the prototypes. In addition, through supervised contrastive learning (SCL) based on neighborhood pixel mask in the target domain, the enhanced sample features belonging to the same class are closer, while the enhanced sample features belonging to different classes are pulled further, enabling the model to learn mask-robust discriminative feature representations, suppressing the negative impact of redundant and noisy pixels, and improving the model’s discriminability. The experimental results demonstrate the superiority of the proposed CDFS-CASCL. The code is available at https://github.com/Li-ZK/CDFS-CASCL-2024.

The purpose of this study was to explore the potential of the development and application of Paecilomyces hepiali, a fungus with edible and medicinal value, as a foodborne antioxidant and anti-aging agent. Its n-butanol extract (PHE) from rice cultures was selected for anti-aging experiment because of significant free radical scavenging activity in vitro. In vivo, PHE could significantly prolong the mean lifespan, 50 % survival days, and the maximum lifespan of Drosophilaon a high-fat diet.It is amazing that the mean lifespan increased from 19.1 days to 32.9 days, 50 % survival days increased from 15.7 days to 34.3 days, and the maximum lifespan extended from 44.7 days to 52.7 days, when the high-fat female Drosophilamodel was fed with 10 μg/mL PHE. Further research showed that PHE reduced the accumulation of peroxide products and increased the activity of antioxidant enzymes. Then, through antioxidant activity tracking, dimerumic acid (compound 1,the IC50value of 3.4 μg/mL on DPPH free radicals scavenging activity), 4,5-dihydroxy-3-methoxypentanoic acid (compound 2, new compound), and thymidine (compound 3) were isolated from PHE. It is worth mentioning that dimerumic acid, the major antioxidant compound of PHE (content up to 3 %), was discovered in P. hepialifor the first time. It was concluded that PHE showed excellent anti-aging activity at a very low concentration on fat diet-induced high fat Drosophila melanogaster, and dimerumic acid may be its main material basis. These results indicated that PHE had the potential to be developed as antioxidant and anti-aging agent in the healthcare industry.