Search

Understanding of video creativity and content often varies among individuals, with differences in focal points and cognitive levels across different ages, experiences, and genders. There is currently a lack of research in this area, and most existing benchmarks suffer from several drawbacks: 1) a limited number of modalities and answers with restrictive length; 2) the content and scenarios within the videos are excessively monotonous, transmitting allegories and emotions that are overly simplistic. To bridge the gap to real-world applications, we introduce a large-scale Subjective Response Indicators for Advertisement Videos dataset, namely SRI-ADV. Specifically, we collected real changes in Electroencephalographic (EEG) and eye-tracking regions from different demographics while they viewed identical video content. Utilizing this multi-modal dataset, we developed tasks and protocols to analyze and evaluate the extent of cognitive understanding of video content among different users. Along with the dataset, we designed a Hypergraph Multi-modal Large Language Model (HMLLM) to explore the associations among different demographics, video elements, EEG, and eye-tracking indicators. HMLLM could bridge semantic gaps across rich modalities and integrate information beyond different modalities to perform logical reasoning. Extensive experimental evaluations on SRI-ADV and other additional video-based generative performance benchmarks demonstrate the effectiveness of our method. The codes and dataset will be released at https://github.com/mininglamp-MLLM/HMLLM., Comment: Accepted by ACM MULTIMEDIA 2024

The kinematics of human movements and locomotion are closely linked to the activation and contractions of muscles. To investigate this, we present a multimodal dataset with benchmarks collected using a novel pair of Intelligent Knee Sleeves (Texavie MarsWear Knee Sleeves) for human pose estimation. Our system utilizes synchronized datasets that comprise time-series data from the Knee Sleeves and the corresponding ground truth labels from the visualized motion capture camera system. We employ these to generate 3D human models solely based on the wearable data of individuals performing different activities. We demonstrate the effectiveness of this camera-free system and machine learning algorithms in the assessment of various movements and exercises, including extension to unseen exercises and individuals. The results show an average error of 7.21 degrees across all eight lower body joints when compared to the ground truth, indicating the effectiveness and reliability of the Knee Sleeve system for the prediction of different lower body joints beyond the knees. The results enable human pose estimation in a seamless manner without being limited by visual occlusion or the field of view of cameras. Our results show the potential of multimodal wearable sensing in a variety of applications from home fitness to sports, healthcare, and physical rehabilitation focusing on pose and movement estimation., Comment: Accepted by Thirty-seventh Conference on Neural Information Processing Systems (Neurips) D&B Track

We perform a study of phase transitions of RN-AdS black hole at its Davies points according to a generalized Ehrenfest classification of phase transition established on the basis of fractional derivatives. Davies points label the positions where heat capacity diverges. According to the usual Ehrenfest classification, second-order phase transitions occur there. For RN-AdS black hole, the Davies points can be classified into two types. The first type corresponds to the extreme values of temperature and the second type corresponds to the infection point(namely the critical point) of temperature. Employing the generalized Ehrenfest classification, we find that the orders of phase transition at the two types of Davies points are different. It is $3/2$-order for the first type and $4/3$-order for the second type. Thus this finer-grained classification can discriminate phase transitions that are supposed to be in the same category, which may provide some new insights toward a better understanding of black hole thermodynamics., Comment: 13 pages, 4 figures, to be published in Chinese Physics C

In a low-disorder two-dimensional electron system, when two Landau levels of opposite spin or pseudospin cross at the Fermi level, the dominance of the exchange energy can lead to a ferromagnetic, quantum Hall ground state whose gap is determined by the exchange energy and has skyrmions as its excitations. This is normally achieved via applying either hydrostatic pressure or uniaxial strain. We study here a very high-quality, low-density, two-dimensional hole system, confined to a 30-nm-wide (001) GaAs quantum well, in which the two lowest-energy Landau levels can be gate tuned to cross at and near filling factor $\nu=1$. As we tune the field position of the crossing from one side of $\nu=1$ to the other by changing the hole density, the energy gap for the quantum Hall state at $\nu=1$ remains exceptionally large, and only shows a small dip near the crossing. The gap overall follows a $\sqrt{B}$ dependence, expected for the exchange energy. Our data are consistent with a robust quantum Hall ferromagnet as the ground state., Comment: 7+2 pages, 4+4 figures

Following Barrow's idea of fractal black hole horizon, we re-derive black hole entropy of static spherically symmetric black holes. When a black hole absorbs matter its horizon area will increase. Given the spherically fractal structure, we conjecture that the minimal increase of the horizon area should be the area of the smallest bubble sphere. From this, we find the black hole entropy has a logarithmic form, which is similar to that of Boltzmann entropy if we consider $A/A_{pl}$ as the number of microscopic states. We further calculate temperatures and heat capacities of Schwarzschild, Reissner-Nordstr{\"o}m(RN), and RN-AdS black holes. It is found that their temperatures are all monotonically increasing and the heat capacities are all positive, which means these black holes are thermodynamically stable. Besides, for RN-AdS black hole we find its heat capacity has Schottky anomaly-like behavior, which may reflect the existence of the discrete energy level and restricted microscopical degree of freedom., Comment: 10 pages, 5 figures

As an important algorithm in deep reinforcement learning, advantage actor critic (A2C) has been widely succeeded in both discrete and continuous control tasks with raw pixel inputs, but its sample efficiency still needs to improve more. In traditional reinforcement learning, actor-critic algorithms generally use the recursive least squares (RLS) technology to update the parameter of linear function approximators for accelerating their convergence speed. However, A2C algorithms seldom use this technology to train deep neural networks (DNNs) for improving their sample efficiency. In this paper, we propose two novel RLS-based A2C algorithms and investigate their performance. Both proposed algorithms, called RLSSA2C and RLSNA2C, use the RLS method to train the critic network and the hidden layers of the actor network. The main difference between them is at the policy learning step. RLSSA2C uses an ordinary first-order gradient descent algorithm and the standard policy gradient to learn the policy parameter. RLSNA2C uses the Kronecker-factored approximation, the RLS method and the natural policy gradient to learn the compatible parameter and the policy parameter. In addition, we analyze the complexity and convergence of both algorithms, and present three tricks for further improving their convergence speed. Finally, we demonstrate the effectiveness of both algorithms on 40 games in the Atari 2600 environment and 11 tasks in the MuJoCo environment. From the experimental results, it is shown that our both algorithms have better sample efficiency than the vanilla A2C on most games or tasks, and have higher computational efficiency than other two state-of-the-art algorithms., Comment: 13 pages, 2 figures, 4 tables, IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, under review

Convolutional neural networks (CNNs) have succeeded in many practical applications. However, their high computation and storage requirements often make them difficult to deploy on resource-constrained devices. In order to tackle this issue, many pruning algorithms have been proposed for CNNs, but most of them can't prune CNNs to a reasonable level. In this paper, we propose a novel algorithm for training and pruning CNNs based on the recursive least squares (RLS) optimization. After training a CNN for some epochs, our algorithm combines inverse input autocorrelation matrices and weight matrices to evaluate and prune unimportant input channels or nodes layer by layer. Then, our algorithm will continue to train the pruned network, and won't do the next pruning until the pruned network recovers the full performance of the old network. Besides for CNNs, the proposed algorithm can be used for feedforward neural networks (FNNs). Three experiments on MNIST, CIFAR-10 and SVHN datasets show that our algorithm can achieve the more reasonable pruning and have higher learning efficiency than other four popular pruning algorithms.

Semi-supervised learning (SSL) over graph-structured data emerges in many network science applications. To efficiently manage learning over graphs, variants of graph neural networks (GNNs) have been developed recently. By succinctly encoding local graph structures and features of nodes, state-of-the-art GNNs can scale linearly with the size of graph. Despite their success in practice, most of existing methods are unable to handle graphs with uncertain nodal attributes. Specifically whenever mismatches between training and testing data distribution exists, these models fail in practice. Challenges also arise due to distributional uncertainties associated with data acquired by noisy measurements. In this context, a distributionally robust learning framework is developed, where the objective is to train models that exhibit quantifiable robustness against perturbations. The data distribution is considered unknown, but lies within a Wasserstein ball centered around empirical data distribution. A robust model is obtained by minimizing the worst expected loss over this ball. However, solving the emerging functional optimization problem is challenging, if not impossible. Advocating a strong duality condition, we develop a principled method that renders the problem tractable and efficiently solvable. Experiments assess the performance of the proposed method.