Search

Large-scale pre-trained image-text models demonstrate remarkable versatility across diverse tasks, benefiting from their robust representational capabilities and effective multimodal alignment. We extend the application of these models, specifically CLIP, to the domain of sound source localization. Unlike conventional approaches, we employ the pre-trained CLIP model without explicit text input, relying solely on the audio-visual correspondence. To this end, we introduce a framework that translates audio signals into tokens compatible with CLIP's text encoder, yielding audio-driven embeddings. By directly using these embeddings, our method generates audio-grounded masks for the provided audio, extracts audio-grounded image features from the highlighted regions, and aligns them with the audio-driven embeddings using the audio-visual correspondence objective. Our findings suggest that utilizing pre-trained image-text models enable our model to generate more complete and compact localization maps for the sounding objects. Extensive experiments show that our method outperforms state-of-the-art approaches by a significant margin., Comment: WACV 2024

The goal of this work is to localize sound sources in visual scenes with a self-supervised approach. Contrastive learning in the context of sound source localization leverages the natural correspondence between audio and visual signals where the audio-visual pairs from the same source are assumed as positive, while randomly selected pairs are negatives. However, this approach brings in noisy correspondences; for example, positive audio and visual pair signals that may be unrelated to each other, or negative pairs that may contain semantically similar samples to the positive one. Our key contribution in this work is to show that using a less strict decision boundary in contrastive learning can alleviate the effect of noisy correspondences in sound source localization. We propose a simple yet effective approach by slightly modifying the contrastive loss with a negative margin. Extensive experimental results show that our approach gives on-par or better performance than the state-of-the-art methods. Furthermore, we demonstrate that the introduction of a negative margin to existing methods results in a consistent improvement in performance., Comment: Submitted to ICASSP 2023. SOTA performance in Audio-Visual Sound Localization. 5 Pages

We report our experiment on sideband cooling with two Raman transitions in a $\Lambda$ configuration that allows selective coherent population trapping (CPT) of the motional ground state. The cooling method is applied to $^{87}$Rb atoms in a circularly-polarized one-dimensional optical lattice. Owing to the vector polarizability, the vibration frequency of a trapped atom depends on its Zeeman quantum number, and CPT resonance for a pair of bound states in the $\Lambda$ configuration depends on their vibrational quantum numbers. We call this scheme motion-selective coherent population trapping (MSCPT) and it is a trapped-atom analogue to the velocity-selective CPT developed for free He atoms. We observe a pronounced dip in temperature near a detuning for the Raman beams to satisfy the CPT resonance condition for the motional ground state. Although the lowest temperature we obtain is ten times the recoil limit owing to the large Lamb-Dicke parameter of 2.3 in our apparatus, the experiment demonstrates that MSCPT enhances the effectiveness of Raman sideband cooling and enlarges the range of its application. Discussions on design parameters optimized for MSCPT on $^{87}$Rb atoms and opportunities provided by diatomic polar molecules, whose Stark shift shows strong dependence on the rotational quantum number, are included., Comment: 13 pages, 6 figures, 1 table

We propose a scheme that combines velocity-selective coherent population trapping (CPT) and Raman sideband cooling (RSC) for subrecoil cooling of optically trapped atoms outside the Lamb-Dicke regime. This scheme is based on an inverted $\mathsf{Y}$ configuration in an alkali-metal atom. It consists of a $\Lambda$ formed by two Raman transitions between the ground hyperfine levels and the $D$ transition, allowing RSC along two paths and formation of a CPT dark state. Using state-dependent difference in vibration frequency of the atom in a circularly polarized trap, we can tune the $\Lambda$ to make only the motional ground state a CPT dark state. We call this scheme motion-selective coherent population trapping (MSCPT). We write the master equations for RSC and MSCPT and solve them numerically for a $^{87}$Rb atom in a one-dimensional optical lattice when the Lamb-Dicke parameter is 1. Although MSCPT reaches the steady state slowly compared with RSC, the former consistently produces colder atoms than the latter. The numerical results also show that subrecoil cooling by MSCPT outside the Lamb-Dicke regime is possible under a favorable, yet experimentally feasible, condition. We explain this performance quantitatively by calculating the relative darkness of each motional state. Finally, we discuss on application of the MSCPT scheme to an optically trapped diatomic polar molecule whose Stark shift and vibration frequency exhibit large variations depending on the rotational quantum number., Comment: 25 pages, 10 figures, planning to submit to Phys. Rev. A