Search

Sequential recommendation tasks, which aim to predict the next item a user will interact with, typically rely on models trained solely on historical data. However, in real-world scenarios, user behavior can fluctuate in the long interaction sequences, and training data may be limited to model this dynamics. To address this, Test-Time Training (TTT) offers a novel approach by using self-supervised learning during inference to dynamically update model parameters. This allows the model to adapt to new user interactions in real-time, leading to more accurate recommendations. In this paper, we propose TTT4Rec, a sequential recommendation framework that integrates TTT to better capture dynamic user behavior. By continuously updating model parameters during inference, TTT4Rec is particularly effective in scenarios where user interaction sequences are long, training data is limited, or user behavior is highly variable. We evaluate TTT4Rec on three widely-used recommendation datasets, demonstrating that it achieves performance on par with or exceeding state-of-the-art models. The codes are available at https://github.com/ZhaoqiZachYang/TTT4Rec.

Video question answering (VideoQA) is a task to predict the correct answer to questions posed about a given video. The system must comprehend spatial and temporal relationships among objects extracted from videos to perform causal and temporal reasoning. While prior works have focused on modeling individual object movements using transformer-based methods, they falter when capturing complex scenarios involving multiple objects (e.g., "a boy is throwing a ball in a hoop"). We propose a contrastive language event graph representation learning method called CLanG to address this limitation. Aiming to capture event representations associated with multiple objects, our method employs a multi-layer GNN-cluster module for adversarial graph representation learning, enabling contrastive learning between the question text and its relevant multi-object event graph. Our method outperforms a strong baseline, achieving up to 2.2% higher accuracy on two challenging VideoQA datasets, NExT-QA and TGIF-QA-R. In particular, it is 2.8% better than baselines in handling causal and temporal questions, highlighting its strength in reasoning multiple object-based events., Comment: presented at MIRU2024

Capturing complex hierarchical human activities, from atomic actions (e.g., picking up one present, moving to the sofa, unwrapping the present) to contextual events (e.g., celebrating Christmas) is crucial for achieving high-performance video question answering (VideoQA). Recent works have expanded multimodal models (e.g., CLIP, LLaVA) to process continuous video sequences, enhancing the model's temporal reasoning capabilities. However, these approaches often fail to capture contextual events that can be decomposed into multiple atomic actions non-continuously distributed over relatively long-term sequences. In this paper, to leverage the spatial visual context representation capability of the CLIP model for obtaining non-continuous visual representations in terms of contextual events in videos, we convert long-term video sequences into a spatial image domain and finetune the multimodal model LLaVA for the VideoQA task. Our approach achieves competitive performance on the STAR task, in particular, with a 78.4% accuracy score, exceeding the current state-of-the-art score by 2.8 points on the NExTQA task., Comment: presented at MIRU2024

We present a comprehensive study based on multi-wavelength observations from the NuSTAR, NICER, Swift, Fermi, NEOWISE, and ATCA telescopes during the 2022 outburst of the black hole X-ray binary IGR J17091-3624. Our investigation concentrates on the heartbeat-like variability in the X-ray emission, with the aim of using it as a tool to unravel the origin of the non-thermal emission during the heartbeat state. Through X-ray timing and spectral analysis, we observe that the heartbeat-like variability correlates with changes in the disk temperature, supporting the disk radiation pressure instability scenario. Moreover, in addition to a Comptonization component, our time-averaged and phase-resolved spectroscopy reveal the presence of a power-law component that varies independently from the disk component. Combined with the radio to X-ray spectral energy distribution fitting, our results suggest that the power-law component could originate from synchrotron self-Compton radiation in the jet, which requires a strong magnetic field of about $B = (0.3$-$3.5)\times10^6$ G. Additionally, assuming that IGR J17091-3624 and GRS 1915+105 share the same radio-X-ray correlation coefficient during both the hard and the heartbeat states, we obtain a distance of $13.7\pm2.3$ kpc for IGR J17091-3624., Comment: 19 pages, 11 figures, 3 tables; accepted for publication in ApJ

Metric learning minimizes the gap between similar (positive) pairs of data points and increases the separation of dissimilar (negative) pairs, aiming at capturing the underlying data structure and enhancing the performance of tasks like audio-visual cross-modal retrieval (AV-CMR). Recent works employ sampling methods to select impactful data points from the embedding space during training. However, the model training fails to fully explore the space due to the scarcity of training data points, resulting in an incomplete representation of the overall positive and negative distributions. In this paper, we propose an innovative Anchor-aware Deep Metric Learning (AADML) method to address this challenge by uncovering the underlying correlations among existing data points, which enhances the quality of the shared embedding space. Specifically, our method establishes a correlation graph-based manifold structure by considering the dependencies between each sample as the anchor and its semantically similar samples. Through dynamic weighting of the correlations within this underlying manifold structure using an attention-driven mechanism, Anchor Awareness (AA) scores are obtained for each anchor. These AA scores serve as data proxies to compute relative distances in metric learning approaches. Extensive experiments conducted on two audio-visual benchmark datasets demonstrate the effectiveness of our proposed AADML method, significantly surpassing state-of-the-art models. Furthermore, we investigate the integration of AA proxies with various metric learning methods, further highlighting the efficacy of our approach., Comment: 9 pages, 5 figures. Accepted by ACM ICMR 2024

Considering the variability of amplitude and phase patterns in electrocardiogram (ECG) signals due to cardiac activity and individual differences, existing entropy-based studies have not fully utilized these two patterns and lack integration. To address this gap, this paper proposes a novel fusion entropy metric, morphological ECG entropy (MEE) for the first time, specifically designed for ECG morphology, to comprehensively describe the fusion of amplitude and phase patterns. MEE is computed based on beat-level samples, enabling detailed analysis of each cardiac cycle. Experimental results demonstrate that MEE achieves rapid, accurate, and label-free localization of abnormal ECG arrhythmia regions. Furthermore, MEE provides a method for assessing sample diversity, facilitating compression of imbalanced training sets (via representative sample selection), and outperforms random pruning. Additionally, MEE exhibits the ability to describe areas of poor quality. By discussing, it proves the robustness of MEE value calculation to noise interference and its low computational complexity. Finally, we integrate this method into a clinical interactive interface to provide a more convenient and intuitive user experience. These findings indicate that MEE serves as a valuable clinical descriptor for ECG characterization. The implementation code can be referenced at the following link: https://github.com/fdu-harry/ECG-MEE-metric., Comment: 16 pages, 12 figures

Many real-world systems exhibit temporal, dynamic behaviors, which are captured as time series of complex agent interactions. To perform temporal reasoning, current methods primarily encode temporal dynamics through simple sequence-based models. However, in general these models fail to efficiently capture the full spectrum of rich dynamics in the input, since the dynamics is not uniformly distributed. In particular, relevant information might be harder to extract and computing power is wasted for processing all individual timesteps, even if they contain no significant changes or no new information. Here we propose TimeGraphs, a novel approach that characterizes dynamic interactions as a hierarchical temporal graph, diverging from traditional sequential representations. Our approach models the interactions using a compact graph-based representation, enabling adaptive reasoning across diverse time scales. Adopting a self-supervised method, TimeGraphs constructs a multi-level event hierarchy from a temporal input, which is then used to efficiently reason about the unevenly distributed dynamics. This construction process is scalable and incremental to accommodate streaming data. We evaluate TimeGraphs on multiple datasets with complex, dynamic agent interactions, including a football simulator, the Resistance game, and the MOMA human activity dataset. The results demonstrate both robustness and efficiency of TimeGraphs on a range of temporal reasoning tasks. Our approach obtains state-of-the-art performance and leads to a performance increase of up to 12.2% on event prediction and recognition tasks over current approaches. Our experiments further demonstrate a wide array of capabilities including zero-shot generalization, robustness in case of data sparsity, and adaptability to streaming data flow.

Linear RMS-flux relation has been well established in different spectral states of all accreting systems. In this work, we study the evolution of the frequency-dependent RMS-flux relation of MAXI J1820+070 during the initial decaying phase of the 2018 outburst with Insight-HXMT over a broad energy range 1-150 keV. As the flux decreases, we first observe a linear RMS-flux relation at frequencies from 2 mHz to 10 Hz, while such a relation breaks at varying times for different energies, leading to a substantial reduction in the slope. Moreover, we find that the low-frequency variability exhibits the highest sensitivity to the break, which occurs prior to the hard-to-hard state transition time determined through time-averaged spectroscopy, and the time deviation increases with energy. The overall evolution of the RMS-flux slope and intercept suggests the presence of a two-component Comptonization system. One component is radially extended, explaining the strong disk-corona coupling before the break, while the other component extends vertically, contributing to the reduction of the disk-corona coupling after the break. A further vertical expansion of the latter component is required to accommodate the dynamic evolution observed in the RMS-flux slope. In conclusion, we suggest that the RMS-flux slope in 1-150 keV band can be employed as an indicator of the disk-corona coupling and the hard-to-hard state transition in MAXI J1820+070 could be partially driven by the changes in the corona geometry., Comment: 12 pages, 6 figures; accepted for publication in ApJ

The tidal disruption of a star around a supermassive black hole (SMBH) offers a unique opportunity to study accretion onto a SMBH on a human-timescale. We present results from our 1000+ days NICER, Swift and Chandra monitoring campaign of AT 2019avd, a nuclear transient with TDE-like properties. Our primary finding is that approximately 225 days following the peak of X-ray emission, there is a rapid drop in luminosity exceeding two orders of magnitude. This X-ray drop-off is accompanied by X-ray spectral hardening, followed by a 740-day plateau phase. During this phase, the spectral index decreases from 6.2+-1.1 to 2.3+-0.4, while the disk temperature remains constant. Additionally, we detect pronounced X-ray variability, with an average fractional root mean squared amplitude of 47%, manifesting over timescales of a few dozen minutes. We propose that this phenomenon may be attributed to intervening clumpy outflows. The overall properties of AT 2019avd suggest that the accretion disk evolves from a super-Eddington to a sub-Eddington luminosity state, possibly associated with a compact jet. This evolution follows a pattern in the hardness-intensity diagram similar to that observed in stellar-mass black holes, supporting the mass invariance of accretion-ejection processes around black holes., Comment: 21 pages, 11 figures, 1 table; accepted for publication in ApJ

In X-ray observations of hard state black hole X-ray binaries, rapid variations in accretion disc and coronal power-law emission are correlated and show Fourier-frequency-dependent time lags. On short (~0.1 s) time-scales, these lags are thought to be due to reverberation and therefore may depend strongly on the geometry of the corona. Low-frequency quasi-periodic oscillations (QPOs) are variations in X-ray flux that have been suggested to arise because of geometric changes in the corona, possibly due to General Relativistic Lense-Thirring precession. Therefore one might expect the short-term time lags to vary on the QPO time-scale. We performed novel spectral-timing analyses on NICER observations of the black hole X-ray binary MAXI J1820+070 during the hard state of its outburst in 2018 to investigate how the short-term time lags between a disc-dominated and a coronal power-law-dominated energy band vary on different time-scales. Our method can distinguish between variability due to the QPO and broadband noise, and we find a linear correlation between the power-law flux and lag amplitude that is strongest at the QPO frequency. We also introduce a new method to resolve the QPO signal and determine the QPO-phase-dependence of the flux and lag variations, finding that both are very similar. Our results are consistent with a geometric origin of QPOs, but also provide evidence for a dynamic corona with a geometry varying in a similar way over a broad range of time-scales, not just the QPO time-scale., Comment: 19 pages, 14 figures, 3 tables, published in MNRAS, includes link to reproduction package

When a star passes through the tidal disruption radius of a massive black hole (BH), it can be torn apart by the tidal force of the BH, known as the Tidal Disruption Event (TDE). Since the UV/optical emitting region inferred from the blackbody radius is significantly larger than the circularization radius predicted by the classical TDE theory, two competing models, stream collision and envelope reprocessing, were proposed to explain the unexpectedly large UV/optical emitting size. Here, we investigate the variability behaviors (cross-correlation and time delay) of three representative TDEs with continuum reverberation mapping. Our results demonstrate that TDE behavior is clearly inconsistent with the envelope reprocessing scenario. In contrast, the picture of the stream collision, together with the late-time formed accretion disk, can explain heterogeneous observations. This provides compelling evidence that the UV/optical emission originates from stream collisions during the early-stage of TDE evolution and gradually transitions to being dominated by accretion disk with detectable X-ray emission in a late stage. After fading back to a quiescent state, recurrent flares may be observed in some occasions, such as partial TDEs., Comment: 56 pages, 21 figures, 5 tables. Submitted, comments welcome!

AT 2019azh is a H+He tidal disruption event (TDE) with one of the most extensive ultraviolet and optical data sets available to date. We present our photometric and spectroscopic observations of this event starting several weeks before and out to approximately two years after the g-band peak brightness and combine them with public photometric data. This extensive data set robustly reveals a change in the light-curve slope and a possible bump in the rising light curve of a TDE for the first time, which may indicate more than one dominant emission mechanism contributing to the pre-peak light curve. Indeed, we find that the MOSFiT-derived parameters of AT 2019azh, which assume reprocessed accretion as the sole source of emission, are not entirely self-consistent. We further confirm the relation seen in previous TDEs whereby the redder emission peaks later than the bluer emission. The post-peak bolometric light curve of AT 2019azh is better described by an exponential decline than by the canonical t^{-5/3} (and in fact any) power-law decline. We find a possible mid-infrared excess around the peak optical luminosity, but cannot determine its origin. In addition, we provide the earliest measurements of the Halpha emission-line evolution and find no significant time delay between the peak of the V-band light curve and that of the Halpha luminosity. These results can be used to constrain future models of TDE line formation and emission mechanisms in general. More pre-peak 1-2 days cadence observations of TDEs are required to determine whether the characteristics observed here are common among TDEs. More importantly, detailed emission models are needed to fully exploit such observations for understanding the emission physics of TDEs., Comment: Submitted to ApJ

The paper reports an analytical formula for the production cross section of $e^{+}e^{-}$ annihilation to hadrons in the vicinity of a narrow resonance, particularly in the $\tau$-charm region, while considering initial state radiation. Despite some approximations in its derivation, the comparison between the analytical formula and direct integration of ISR shows good accuracy, indicating that the analytical formula meets current experimental requirements. Furthermore, the paper presents a comparison of the cross section between the analytical formula and calculations using the {\sc ConExc} Monte Carlo generator. The efficiency of the analytical formula in significantly reducing computing time makes it a favorable choice for the regression procedure to extract the parameters of narrow charmonium resonances in experiments., Comment: 13 pages, 16 figures

In recent years, certain luminous extragalactic optical transients have been observed to last only a few days. Their short observed duration implies a different powering mechanism from the most common luminous extragalactic transients (supernovae) whose timescale is weeks. Some short-duration transients, most notably AT2018cow, display blue optical colours and bright radio and X-ray emission. Several AT2018cow-like transients have shown hints of a long-lived embedded energy source, such as X-ray variability, prolonged ultraviolet emission, a tentative X-ray quasiperiodic oscillation, and large energies coupled to fast (but subrelativistic) radio-emitting ejecta. Here we report observations of minutes-duration optical flares in the aftermath of an AT2018cow-like transient, AT2022tsd (the "Tasmanian Devil"). The flares occur over a period of months, are highly energetic, and are likely nonthermal, implying that they arise from a near-relativistic outflow or jet. Our observations confirm that in some AT2018cow-like transients the embedded energy source is a compact object, either a magnetar or an accreting black hole., Comment: 79 pages, 3 figures (main text) + 7 figures (extended data) + 2 figures (supplementary information). Published online in Nature on 15 November 2023

Multimodal transfer learning aims to transform pretrained representations of diverse modalities into a common domain space for effective multimodal fusion. However, conventional systems are typically built on the assumption that all modalities exist, and the lack of modalities always leads to poor inference performance. Furthermore, extracting pretrained embeddings for all modalities is computationally inefficient for inference. In this work, to achieve high efficiency-performance multimodal transfer learning, we propose VideoAdviser, a video knowledge distillation method to transfer multimodal knowledge of video-enhanced prompts from a multimodal fundamental model (teacher) to a specific modal fundamental model (student). With an intuition that the best learning performance comes with professional advisers and smart students, we use a CLIP-based teacher model to provide expressive multimodal knowledge supervision signals to a RoBERTa-based student model via optimizing a step-distillation objective loss -- first step: the teacher distills multimodal knowledge of video-enhanced prompts from classification logits to a regression logit -- second step: the multimodal knowledge is distilled from the regression logit of the teacher to the student. We evaluate our method in two challenging multimodal tasks: video-level sentiment analysis (MOSI and MOSEI datasets) and audio-visual retrieval (VEGAS dataset). The student (requiring only the text modality as input) achieves an MAE score improvement of up to 12.3% for MOSI and MOSEI. Our method further enhances the state-of-the-art method by 3.4% mAP score for VEGAS without additional computations for inference. These results suggest the strengths of our method for achieving high efficiency-performance multimodal transfer learning., Comment: Accepted by IEEE Access

The recently released artificial intelligence conversational agent, ChatGPT, has gained significant attention in academia and real life. A multitude of early ChatGPT users eagerly explore its capabilities and share their opinions on it via social media. Both user queries and social media posts express public concerns regarding this advanced dialogue system. To mine public concerns about ChatGPT, a novel Self-Supervised neural Topic Model (SSTM), which formalizes topic modeling as a representation learning procedure, is proposed in this paper. Extensive experiments have been conducted on Twitter posts about ChatGPT and queries asked by ChatGPT users. And experimental results demonstrate that the proposed approach could extract higher quality public concerns with improved interpretability and diversity, surpassing the performance of state-of-the-art approaches., Comment: The paper requires major revision

Energy harvesting is essential for the internet-of-things networks where a tremendous number of sensors require power. Thermoelectric generators (TEGs), especially those based on silicon (Si), are a promising source of clean and sustainable energy for these sensors. However, the reported performance of planar-type Si TEGs never exceeded power factors of 0.1 ${\mu} Wcm^{-2} K^{-2}$ due to the poor thermoelectric performance of Si and the suboptimal design of the devices. Here, we report a planar-type Si TEG with a power factor of 1.3 ${\mu} Wcm^{-2} K^{-2}$ around room temperature. The increase in thermoelectric performance of Si by nanostructuring based on the phonon-glass electron-crystal concept and optimized three-dimensional heat-guiding structures resulted in a significant power factor. In-field testing demonstrated that our Si TEG functions as a 100-${\mu}W$-class harvester. This result is an essential step toward energy harvesting with a low-environmental load and cost-effective material with high throughput, a necessary condition for energy-autonomous sensor nodes for the trillion sensors universe.