Search

Multispectral pedestrian detection is attractive for around-the-clock applications due to the complementary information between RGB and thermal modalities. However, current models often fail to detect pedestrians in certain cases (e.g., thermal-obscured pedestrians), particularly due to the modality bias learned from statistically biased datasets. In this paper, we investigate how to mitigate modality bias in multispectral pedestrian detection using Large Language Models (LLMs). Accordingly, we design a Multispectral Chain-of-Thought (MSCoT) prompting strategy, which prompts the LLM to perform multispectral pedestrian detection. Moreover, we propose a novel Multispectral Chain-of-Thought Detection (MSCoTDet) framework that integrates MSCoT prompting into multispectral pedestrian detection. To this end, we design a Language-driven Multi-modal Fusion (LMF) strategy that enables fusing the outputs of MSCoT prompting with the detection results of vision-based multispectral pedestrian detection models. Extensive experiments validate that MSCoTDet effectively mitigates modality biases and improves multispectral pedestrian detection., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

RGBT multispectral pedestrian detection has emerged as a promising solution for safety-critical applications that require day/night operations. However, the modality bias problem remains unsolved as multispectral pedestrian detectors learn the statistical bias in datasets. Specifically, datasets in multispectral pedestrian detection mainly distribute between ROTO (day) and RXTO (night) data; the majority of the pedestrian labels statistically co-occur with their thermal features. As a result, multispectral pedestrian detectors show poor generalization ability on examples beyond this statistical correlation, such as ROTX data. To address this problem, we propose a novel Causal Mode Multiplexer (CMM) framework that effectively learns the causalities between multispectral inputs and predictions. Moreover, we construct a new dataset (ROTX-MP) to evaluate modality bias in multispectral pedestrian detection. ROTX-MP mainly includes ROTX examples not presented in previous datasets. Extensive experiments demonstrate that our proposed CMM framework generalizes well on existing datasets (KAIST, CVC-14, FLIR) and the new ROTX-MP. We will release our new dataset to the public for future research., Comment: CVPR2024

We present the construction of an image similarity retrieval engine for the morphological classification of galaxies using the Convolutional AutoEncoder (CAE). The CAE is trained on 90,370 preprocessed Sloan Digital Sky Survey galaxy images listed in the Galaxy Zoo 2 (GZ2) catalog. The visually similar output images returned by the trained CAE suggest that the encoder efficiently compresses input images into latent features, which are then used to calculate similarity parameters. Our Tool for Searching a similar Galaxy Image based on a Convolutional Autoencoder using Similarity (TSGICAS) leverages this similarity parameter to classify galaxies' morphological types, enabling the identification of a wider range of classes with high accuracy compared to traditional supervised ML techniques. This approach streamlines the researcher's work by allowing quick prioritization of the most relevant images from the latent feature database. We investigate the accuracy of our automatic morphological classifications using three galaxy catalogs: GZ2, Extraction de Formes Id\'ealis\'ees de Galaxies en Imagerie (EFIGI), and Nair $\&$ Abraham (NA10). The correlation coefficients between the morphological types of input and retrieved galaxy images were found to be 0.735, 0.811, and 0.815 for GZ2, EFIGI, and NA10 catalogs, respectively. Despite differences in morphology tags between input and retrieved galaxy images, visual inspection showed that the two galaxies were very similar, highlighting TSGICAS's superior performance in image similarity search. We propose that morphological classifications of galaxies using TSGICAS are fast and efficient, making it a valuable tool for detailed galaxy morphological classifications in other imaging surveys., Comment: Accepted for publication to PASP; 19 pages, 18 figures

Metallicity distributions (MDs) of globular clusters (GCs) provide crucial clues for the assembly and star formation history of their host galaxies. GC colors, when GCs are old, have been used as a proxy of GC metallicities. Bimodal GC color distributions (CDs) observed in most early-type galaxies have been interpreted as bimodal MDs for decades, suggesting the presence of merely two GC subpopulations within single galaxies. However, the conventional view has been challenged by a new theory that nonlinear metallicity-to-color conversion can cause bimodal CDs from unimodal MDs. The unimodal MDs seem natural given that MDs involved many thousand protogalaxies. The new theory has been tested and corroborated by various observational and theoretical studies. Here we examine the nonlinear nature of GC color-metallicity relations (CMRs) using photometric and spectroscopic GC data of NGC 5128 (Centaurus A) and NGC 4594 (Sombrero), in comparison with stellar population simulations. We find that, with a slight offset in color, the overall shapes of observed and modeled CMRs agree well for all available colors. Diverse color-depending morphologies of GC CDs of the two galaxies are well reproduced based on their observed spectroscopic MDs via our CMR models. The results corroborate the nonlinear CMR interpretation of the GC color bimodality, shedding further light on theories of galaxy formation., Comment: 25 pages, 12 figures