Search

Video Anomaly Detection (VAD) systems can autonomously monitor and identify disturbances, reducing the need for manual labor and associated costs. However, current VAD systems are often limited by their superficial semantic understanding of scenes and minimal user interaction. Additionally, the prevalent data scarcity in existing datasets restricts their applicability in open-world scenarios. In this paper, we introduce Hawk, a novel framework that leverages interactive large Visual Language Models (VLM) to interpret video anomalies precisely. Recognizing the difference in motion information between abnormal and normal videos, Hawk explicitly integrates motion modality to enhance anomaly identification. To reinforce motion attention, we construct an auxiliary consistency loss within the motion and video space, guiding the video branch to focus on the motion modality. Moreover, to improve the interpretation of motion-to-language, we establish a clear supervisory relationship between motion and its linguistic representation. Furthermore, we have annotated over 8,000 anomaly videos with language descriptions, enabling effective training across diverse open-world scenarios, and also created 8,000 question-answering pairs for users' open-world questions. The final results demonstrate that Hawk achieves SOTA performance, surpassing existing baselines in both video description generation and question-answering. Our codes/dataset/demo will be released at https://github.com/jqtangust/hawk.

There is a growing demand to deploy computation-intensive deep learning (DL) models on resource-constrained mobile devices for real-time intelligent applications. Equipped with a variety of processing units such as CPUs, GPUs, and NPUs, the mobile devices hold potential to accelerate DL inference via parallel execution across heterogeneous processors. Various efficient parallel methods have been explored to optimize computation distribution, achieve load balance, and minimize communication cost across processors. Yet their practical effectiveness in the dynamic and diverse real-world mobile environment is less explored. This paper presents a holistic empirical study to assess the capabilities and challenges associated with parallel DL inference on heterogeneous mobile processors. Through carefully designed experiments covering various DL models, mobile software/hardware environments, workload patterns, and resource availability, we identify limitations of existing techniques and highlight opportunities for cross-level optimization.

Establishing appropriate mathematical models for complex systems in natural phenomena not only helps deepen our understanding of nature but can also be used for state estimation and prediction. However, the extreme complexity of natural phenomena makes it extremely challenging to develop full-order models (FOMs) and apply them to studying many quantities of interest. In contrast, appropriate reduced-order models (ROMs) are favored due to their high computational efficiency and ability to describe the key dynamics and statistical characteristics of natural phenomena. Taking the viscous Burgers equation as an example, this paper constructs a Convolutional Autoencoder-Reservoir Computing-Normalizing Flow algorithm framework, where the Convolutional Autoencoder is used to construct latent space representations, and the Reservoir Computing-Normalizing Flow framework is used to characterize the evolution of latent state variables. In this way, a data-driven stochastic parameter reduced-order model is constructed to describe the complex system and its dynamic behavior.

Van der Waals multiferroic tunnel junctions (vdW-MFTJs) with multiple nonvolatile resistive states are highly suitable for new physics and next-generation storage electronics. However, currently reported vdW-MFTJs are based on two types of materials, i.e., vdW ferromagnetic and ferroelectric materials, forming a multiferroic system. This undoubtedly introduces additional interfaces, increasing the complexity of experimental preparation. Herein, we engineer vdW intrinsic MFTJs utilizing bilayer VS$_2$. By employing the nonequilibrium Green's function combined with density functional theory, we systematically investigate the influence of three types of electrodes (including non-vdW pure metal Ag/Au, vdW metallic 1T-MoS$_2$/2H-PtTe$_2$, and vdW ferromagnetic metallic Fe$_3$GaTe$_2$/Fe$_3$GeTe$_2$) on the electronic transport properties of VS$_2$-based intrinsic MFTJs. We demonstrate that these MFTJs manifest a giant electrode-dependent electronic transport characteristic effect. Comprehensively comparing these electrode pairs, the Fe$_3$GaTe$_2$/Fe$_3$GeTe$_2$ electrode combination exhibits optimal transport properties, the maximum TMR (TER) can reach 10949\% (69\%) and the minimum resistance-area product (RA) is 0.45 $\Omega$$\mu$m$^{2}$, as well as the perfect spin filtering and negative differential resistance effects. More intriguingly, TMR (TER) can be further enhanced to 34000\% (380\%) by applying an external bias voltage (0.1 V), while RA can be reduced to 0.16 $\Omega$$\mu$m$^{2}$ under the influence of biaxial stress (-3\%). Our proposed concept of designing vdW-MFTJs using intrinsic multiferroic materials points towards new avenues in experimental exploration., Comment: 13 pages, 9 figures

We calculate the tensor bispectrum mediated by an excited scalar field during inflation and find that the bispectrum peaks in the squeezed configuration, which is different from that of gravitational waves induced by enhanced curvature perturbations re-entering the horizon in the radiation-dominated era. Measuring the bispectrum provides a promising way to distinguish the stochastic gravitational-wave background generated during inflation from that generated after inflation., Comment: 10 pages, 6 figures

To enhance robotic grasping capabilities, we are developing new contactless fingertip sensors to measure distance in close proximity and simultaneously detect the type of material and the interior structure. These sensors are referred to as pre-touch dual-modal and dual-mechanism (PDM$^2$) sensors, and they operate using both pulse-echo ultrasound (US) and optoacoustic (OA) modalities. We present the design of a PDM$^2$ sensor that utilizes a pulsed laser beam and a customized ultrasound transceiver with a wide acoustic bandwidth for ranging and sensing. Both US and OA signals are collected simultaneously, triggered by the same laser pulse. To validate our design, we have fabricated a prototype of the PDM$^2$ sensor and integrated it into an object scanning system. We have also developed algorithms to enable the sensor, including time-of-flight (ToF) auto estimation, ranging rectification, sensor and system calibration, distance ranging, material/structure detection, and object contour detection and reconstruction. The experimental results demonstrate that the new PDM$^2$ sensor and its algorithms effectively enable the object scanning system to achieve satisfactory ranging and contour reconstruction performances, along with satisfying material/structure detection capabilities. In conclusion, the PDM$^2$ sensor offers a practical and powerful solution to improve grasping of unknown objects with the robotic gripper by providing advanced perception capabilities.

Radar has stronger adaptability in adverse scenarios for autonomous driving environmental perception compared to widely adopted cameras and LiDARs. Compared with commonly used 3D radars, the latest 4D radars have precise vertical resolution and higher point cloud density, making it a highly promising sensor for autonomous driving in complex environmental perception. However, due to the much higher noise than LiDAR, manufacturers choose different filtering strategies, resulting in an inverse ratio between noise level and point cloud density. There is still a lack of comparative analysis on which method is beneficial for deep learning-based perception algorithms in autonomous driving. One of the main reasons is that current datasets only adopt one type of 4D radar, making it difficult to compare different 4D radars in the same scene. Therefore, in this paper, we introduce a novel large-scale multi-modal dataset featuring, for the first time, two types of 4D radars captured simultaneously. This dataset enables further research into effective 4D radar perception algorithms.Our dataset consists of 151 consecutive series, most of which last 20 seconds and contain 10,007 meticulously synchronized and annotated frames. Moreover, our dataset captures a variety of challenging driving scenarios, including many road conditions, weather conditions, nighttime and daytime with different lighting intensities and periods. Our dataset annotates consecutive frames, which can be applied to 3D object detection and tracking, and also supports the study of multi-modal tasks. We experimentally validate our dataset, providing valuable results for studying different types of 4D radars. This dataset is released on https://github.com/adept-thu/Dual-Radar.

The emerging field of artificial intelligence of things (AIoT, AI+IoT) is driven by the widespread use of intelligent infrastructures and the impressive success of deep learning (DL). With the deployment of DL on various intelligent infrastructures featuring rich sensors and weak DL computing capabilities, a diverse range of AIoT applications has become possible. However, DL models are notoriously resource-intensive. Existing research strives to realize near-/realtime inference of AIoT live data and low-cost training using AIoT datasets on resource-scare infrastructures. Accordingly, the accuracy and responsiveness of DL models are bounded by resource availability. To this end, the algorithm-system co-design that jointly optimizes the resource-friendly DL models and model-adaptive system scheduling improves the runtime resource availability and thus pushes the performance boundary set by the standalone level. Unlike previous surveys on resource-friendly DL models or hand-crafted DL compilers/frameworks with partially fine-tuned components, this survey aims to provide a broader optimization space for more free resource-performance tradeoffs. The cross-level optimization landscape involves various granularity, including the DL model, computation graph, operator, memory schedule, and hardware instructor in both on-device and distributed paradigms. Furthermore, due to the dynamic nature of AIoT context, which includes heterogeneous hardware, agnostic sensing data, varying user-specified performance demands, and resource constraints, this survey explores the context-aware inter-/intra-device controllers for automatic cross-level adaptation. Additionally, we identify some potential directions for resource-efficient AIoT systems. By consolidating problems and techniques scattered over diverse levels, we aim to help readers understand their connections and stimulate further discussions.

Deep learning models usually require sufficient training data to achieve high accuracy, but obtaining labeled data can be time-consuming and labor-intensive. Here we introduce a template-based training method to train a 3D U-Net model from scratch using only one population-averaged brain MRI template and its associated segmentation label. The process incorporated visual perception augmentation to enhance the model's robustness in handling diverse image inputs and mitigating overfitting. Leveraging this approach, we trained 3D U-Net models for mouse, rat, marmoset, rhesus, and human brain MRI to achieve segmentation tasks such as skull-stripping, brain segmentation, and tissue probability mapping. This tool effectively addresses the limited availability of training data and holds significant potential for expanding deep learning applications in image analysis, providing researchers with a unified solution to train deep neural networks with only one image sample.

The heating mechanisms of solar white-light flares remain unclear. We present an X1.0 white-light flare on 2022 October 2 (SOL2022-10-02T20:25) observed by the Chinese \ha\ Solar Explorer (CHASE) that provides two-dimensional spectra in the visible light for the full solar disk with a seeing-free condition. The flare shows a prominent enhancement of $\sim$40\% in the photospheric \fe\ line at 6569.2 \AA, and the nearby continuum also exhibits a maximum enhancement of $\sim$40\%. For the continuum near the \fe\ line at 6173 \AA\ from the Helioseismic and Magnetic Imager (HMI) on board the Solar Dynamics Observatory (SDO), it is enhanced up to $\sim$20\%. At the white-light kernels, the \fe\ line at 6569.2 \AA\ has a symmetric Gaussian profile that is still in absorption and the H$\alpha$ line at 6562.8 \AA\ displays a very broad emission profile with a central reversal plus a red or blue asymmetry. The white-light kernels are co-spatial with the microwave footpoint sources observed by the Expanded Owens Valley Solar Array (EOVSA) and the time profile of the white-light emission matches that of the hard X-ray emission above 30 keV from the Gamma-ray Burst Monitor (GBM) on Fermi. These facts indicate that the white-light emission is qualitatively related to a nonthermal electron beam. We also perform a radiative hydrodynamic simulation with the electron beam parameters constrained by the hard X-ray observations from Fermi/GBM. The result reveals that the white-light enhancement cannot be well explained by a pure electron-beam heating together with its induced radiative backwarming but may need additional heating sources such as \alfven\ waves., Comment: 15 pages, 5 figures, and 1 table

Neuroscience research has expanded dramatically over the past 30 years by advancing standardization and tool development to support rigor and transparency. Consequently, the complexity of the data pipeline has also increased, hindering access to FAIR (Findable, Accessible, Interoperabile, and Reusable) data analysis to portions of the worldwide research community. brainlife.io was developed to reduce these burdens and democratize modern neuroscience research across institutions and career levels. Using community software and hardware infrastructure, the platform provides open-source data standardization, management, visualization, and processing and simplifies the data pipeline. brainlife.io automatically tracks the provenance history of thousands of data objects, supporting simplicity, efficiency, and transparency in neuroscience research. Here brainlife.io's technology and data services are described and evaluated for validity, reliability, reproducibility, replicability, and scientific utility. Using data from 4 modalities and 3,200 participants, we demonstrate that brainlife.io's services produce outputs that adhere to best practices in modern neuroscience research.

The exploration of solid-solid phase transition suffers from the uncertainty of how atoms in two crystal structures match. We devised a theoretical framework to describe and classify crystal-structure matches (CSM). Such description fully exploits the translational and rotational symmetries and is independent of the choice of supercells. This is enabled by the use of the Hermite normal form, an analog of reduced echelon form for integer matrices. With its help, exhausting all CSMs is made possible, which goes beyond the conventional optimization schemes. In an example study of the martensitic transformation of steel, our enumeration algorithm finds many candidate CSMs with lower strains than known mechanisms. Two long-sought CSMs accounting for the most commonly observed Kurdjumov-Sachs orientation relationship and the Nishiyama-Wassermann orientation relationship are unveiled. Given the comprehensiveness and efficiency, our enumeration scheme provide a promising strategy for solid-solid phase transition mechanism research., Comment: main text (6 pages, 4 figures) + supplemental material (10 pages, 8 figures)

The prediction of stochastic dynamical systems and the capture of dynamical behaviors are profound problems. In this article, we propose a data-driven framework combining Reservoir Computing and Normalizing Flow to study this issue, which mimics error modeling to improve traditional Reservoir Computing performance and integrates the virtues of both approaches. With few assumptions about the underlying stochastic dynamical systems, this model-free method successfully predicts the long-term evolution of stochastic dynamical systems and replicates dynamical behaviors. We verify the effectiveness of the proposed framework in several experiments, including the stochastic Van der Pal oscillator, El Ni\~no-Southern Oscillation simplified model, and stochastic Lorenz system. These experiments consist of Markov/non-Markov and stationary/non-stationary stochastic processes which are defined by linear/nonlinear stochastic differential equations or stochastic delay differential equations. Additionally, we explore the noise-induced tipping phenomenon, relaxation oscillation, stochastic mixed-mode oscillation, and replication of the strange attractor., Comment: 36 pages, 15 figures

Time series classification faces two unavoidable problems. One is partial feature information and the other is poor label quality, which may affect model performance. To address the above issues, we create a label correction method to time series data with meta-learning under a multi-task framework. There are three main contributions. First, we train the label correction model with a two-branch neural network in the outer loop. While in the model-agnostic inner loop, we use pre-existing classification models in a multi-task way and jointly update the meta-knowledge so as to help us achieve adaptive labeling on complex time series. Second, we devise new data visualization methods for both image patterns of the historical data and data in the prediction horizon. Finally, we test our method with various financial datasets, including XOM, S\&P500, and SZ50. Results show that our method is more effective and accurate than some existing label correction techniques.

The Si I 6560.58 \r{A} line in the H$\alpha$ blue wing is blended with a telluric absorption line from water vapor in ground-based observations. Recent observations with the space-based telescope CHASE provide a new window to study this line. We aim to study the Si I line statistically and to explore possible diagnostics. We select three scannings in the CHASE observations, and measure the equivalent width (EW) and the full width at half maximum (FWHM) for each pixel on the solar disk. We then calculate the theoretical EW and FWHM from the VALC model. An active region is also studied in particular for difference in the quiet Sun and the sunspots. The Si I line is formed at the bottom of the photosphere. The EW of this line increases from the disk center to $\mu$ = 0.2, and then decreases toward the solar limb, while the FWHM shows a monotonically increasing trend. Theoretically predicted EW agrees well with observations, while the predicted FWHM is far smaller due to the absence of unresolved turbulence in models. The macroturbulent velocity is estimated to be 2.80 km s$^{-1}$ at the disk center, and increases to 3.52 km s$^{-1}$ at $\mu$ = 0.2. We do not find any response to flare heating in current observations. Doppler shifts and line widths of the Si I 6560.58 \r{A} and Fe I 6569.21 \r{A} lines can be used to study the mass flows and turbulence of the different photospheric layers. The Si I line has good potentials to diagnose the dynamics and energy transport in the photosphere., Comment: 7 pages, 10 figures, 2 tables. Accepted for publication in A&A. CHASE data are available at https://ssdc.nju.edu.cn