Search

The seen birds twitter, the running cars accompany with noise, etc. These naturally audiovisual correspondences provide the possibilities to explore and understand the outside world. However, the mixed multiple objects and sounds make it intractable to perform efficient matching in the unconstrained environment. To settle this problem, we propose to adequately excavate audio and visual components and perform elaborate correspondence learning among them. Concretely, a novel unsupervised audiovisual learning model is proposed, named as \Deep Multimodal Clustering (DMC), that synchronously performs sets of clustering with multimodal vectors of convolutional maps in different shared spaces for capturing multiple audiovisual correspondences. And such integrated multimodal clustering network can be effectively trained with max-margin loss in the end-to-end fashion. Amounts of experiments in feature evaluation and audiovisual tasks are performed. The results demonstrate that DMC can learn effective unimodal representation, with which the classifier can even outperform human performance. Further, DMC shows noticeable performance in sound localization, multisource detection, and audiovisual understanding., Comment: Accepted by CVPR2019

The energy spectrum of tearing mode turbulence in a sheared background magnetic field is studied in this work. We consider the scenario where the nonlinear interaction of overlapping large-scale modes stirs up a broad spectrum of small-scale modes, generating tearing mode turbulence. The spectrum of such turbulence is of interest since it is relevant to the small-scale back-reaction to the large-scale field. The turbulence we discuss here differs from traditional MHD turbulence mainly in two aspects, one being the existence of many linearly stable small-scale modes which cause an effective damping during energy cascade; the other being the scale independent anisotropy induced by the large scale modes tilting the sheared background field, as opposed to the scale dependent anisotropy frequently encountered in traditional weak turbulence or critically balanced turbulence theory. These two differences result in the deviation of energy spectrum from a simple power law behavior, taking the form of a power law multiplied by an exponential falloff. Numerical simulation is carried out using viscous resistive MHD equations to verify our theoretical prediction, and reasonable agreement is found between numerical result and our model., Comment: 29 pages, 9 figures

Image is usually taken for expressing some kinds of emotions or purposes, such as love, celebrating Christmas. There is another better way that combines the image and relevant song to amplify the expression, which has drawn much attention in the social network recently. Hence, the automatic selection of songs should be expected. In this paper, we propose to retrieve semantic relevant songs just by an image query, which is named as the image2song problem. Motivated by the requirements of establishing correlation in semantic/content, we build a semantic-based song retrieval framework, which learns the correlation between image content and lyric words. This model uses a convolutional neural network to generate rich tags from image regions, a recurrent neural network to model lyric, and then establishes correlation via a multi-layer perceptron. To reduce the content gap between image and lyric, we propose to make the lyric modeling focus on the main image content via a tag attention. We collect a dataset from the social-sharing multimodal data to study the proposed problem, which consists of (image, music clip, lyric) triplets. We demonstrate that our proposed model shows noticeable results in the image2song retrieval task and provides suitable songs. Besides, the song2image task is also performed., Comment: 13 pages, 13 figures, accepted by ICCV 2017

With the increasing demand of massive multimodal data storage and organization, cross-modal retrieval based on hashing technique has drawn much attention nowadays. It takes the binary codes of one modality as the query to retrieve the relevant hashing codes of another modality. However, the existing binary constraint makes it difficult to find the optimal cross-modal hashing function. Most approaches choose to relax the constraint and perform thresholding strategy on the real-value representation instead of directly solving the original objective. In this paper, we first provide a concrete analysis about the effectiveness of multimodal networks in preserving the inter- and intra-modal consistency. Based on the analysis, we provide a so-called Deep Binary Reconstruction (DBRC) network that can directly learn the binary hashing codes in an unsupervised fashion. The superiority comes from a proposed simple but efficient activation function, named as Adaptive Tanh (ATanh). The ATanh function can adaptively learn the binary codes and be trained via back-propagation. Extensive experiments on three benchmark datasets demonstrate that DBRC outperforms several state-of-the-art methods in both image2text and text2image retrieval task., Comment: 8 pages, 5 figures, accepted by ACM Multimedia 2017

A low cost scheme to determine the frequency sweep nonlinearity using atomic saturated absorption spectroscopy is demonstrated. The frequency modulation rate is determined by directly measuring the interference fringe number and frequency gap between two atomic transition peaks of rubidium atom. Experimental results show that the frequency sweep nonlinearity is ~7.68%, with the average frequency modulation rate of ~28.95 GHz/s, which is in good agreement with theoretical expectation. With this method, the absolute optical frequency and optical path difference between two laser beams are simultaneously measured. This novel technique can be used for applications such as optical frequency sweep nonlinearity correction and real-time frequency monitor., Comment: 6 pages, 5 figures