Search

Prior methods propose to offset the escalating costs of modern foundation models by dropping specific parts of their contexts with hand-designed rules, while attempting to preserve their original performance. We overcome this trade-off with Neural Attention Memory Models (NAMMs), introducing a learned network for memory management that improves both the performance and efficiency of transformers. We evolve NAMMs atop pre-trained transformers to provide different latent contexts focusing on the most relevant information for individual layers and attention heads. NAMMs are universally applicable to any model using self-attention as they condition exclusively on the values in the produced attention matrices. Learning NAMMs on a small set of problems, we achieve substantial performance improvements across multiple long-context benchmarks while cutting the model's input contexts up to a fraction of the original sizes. We show the generality of our conditioning enables zero-shot transfer of NAMMs trained only on language to entirely new transformer architectures even across input modalities, with their benefits carrying over to vision and reinforcement learning., Comment: 29 pages, 14 figures. Preprint, under submission. Source code is available at https://github.com/SakanaAI/evo-memory

We show that under Space Curve Shortening flow any closed immersed curve in $\mathbb R^n$ whose projection onto $\mathbb{R}^2\times\{\vec{0}\}$ is convex remains smooth until it shrinks to a point. Throughout its evolution, the projection of the curve onto $\mathbb{R}^2\times\{\vec{0}\}$ remains convex. As an application, we show that any closed immersed curve in $\mathbb R^n$ can be perturbed to an immersed curve in $\mathbb R^{n+2}$ whose evolution by Space Curve Shortening shrinks to a point., Comment: 33 pages, 5 figures. Added affiliation, no other changes

Neural operators, which learn mappings between the function spaces, have been applied to solve boundary value problems in various ways, including learning mappings from the space of the forcing terms to the space of the solutions with the substantial requirements of data pairs. In this work, we present a data-free neural operator integrated with physics, which learns the Green kernel directly. Our method proceeds in three steps: 1. The governing equations for the Green's function are reformulated into an interface problem, where the delta Dirac function is removed; 2. The interface problem is embedded in a lifted space of higher-dimension to handle the jump in the derivative, but still solved on a two-dimensional surface without additional sampling cost; 3. Deep neural networks are employed to address the curse of dimensionality caused by this lifting operation. The approximate Green's function obtained through our approach is then used to construct preconditioners for the linear systems allowed by its mathematical properties. Furthermore, the spectral bias of it revealed through both theoretical analysis and numerical validation contrasts with the smoothing effects of traditional iterative solvers, which motivates us to propose a hybrid iterative method that combines these two solvers. Numerical experiments demonstrate the effectiveness of our approximate Green's function in accelerating iterative methods, proving fast convergence for solving indefinite problems even involving discontinuous coefficients.

Point cloud is a critical 3D representation with many emerging applications. Because of the point sparsity and irregularity, high-quality rendering of point clouds is challenging and often requires complex computations to recover the continuous surface representation. On the other hand, to avoid visual discomfort, the motion-to-photon latency has to be very short, under 10 ms. Existing rendering solutions lack in either quality or speed. To tackle these challenges, we present a framework that unlocks interactive, free-viewing and high-fidelity point cloud rendering. We train a generic neural network to estimate 3D elliptical Gaussians from arbitrary point clouds and use differentiable surface splatting to render smooth texture and surface normal for arbitrary views. Our approach does not require per-scene optimization, and enable real-time rendering of dynamic point cloud. Experimental results demonstrate the proposed solution enjoys superior visual quality and speed, as well as generalizability to different scene content and robustness to compression artifacts. The code is available at https://github.com/huzi96/gaussian-pcloud-render ., Comment: Published at CVPR 2024 Workshop on AIS: Vision, Graphics and AI for Streaming (https://ai4streaming-workshop.github.io/)

Large multimodal models have demonstrated impressive problem-solving abilities in vision and language tasks, and have the potential to encode extensive world knowledge. However, it remains an open challenge for these models to perceive, reason, plan, and act in realistic environments. In this work, we introduce Can-Do, a benchmark dataset designed to evaluate embodied planning abilities through more diverse and complex scenarios than previous datasets. Our dataset includes 400 multimodal samples, each consisting of natural language user instructions, visual images depicting the environment, state changes, and corresponding action plans. The data encompasses diverse aspects of commonsense knowledge, physical understanding, and safety awareness. Our fine-grained analysis reveals that state-of-the-art models, including GPT-4V, face bottlenecks in visual perception, comprehension, and reasoning abilities. To address these challenges, we propose NeuroGround, a neurosymbolic framework that first grounds the plan generation in the perceived environment states and then leverages symbolic planning engines to augment the model-generated plans. Experimental results demonstrate the effectiveness of our framework compared to strong baselines. Our code and dataset are available at https://embodied-planning.github.io.

The IW And phenomenon in cataclysmic variables presents a significant challenge to the accretion disk instability model. Using photometric data from the All-Sky Automated Survey for Supernovae, the Zwicky Transient Facility, and the Transiting Exoplanet Survey Satellite, we identify Karachurin 12 as a new IW And-type object with a cycle period of 35.69(3) days. We also report for the first time that Karachurin 12 is a negative superhump (NSH) system featuring a precessing tilted disk, with precession, orbital, and NSH periods of 4.9588(2) days, 0.3168895(13) days, and 0.2979861(8) days, respectively. Our analysis, using dips as indexes and NSHs as probes, reveals diverse cycle patterns in Karachurin 12, with NSH amplitude varying throughout the cycle. Analyses of NSH behaviour using multiple methods shows that the amplitude decreases with increasing outbursts and increases with weakening outbursts, which we suggest is related to changes in the radius of the accretion disk. The mass transfer outburst model fails to explain the observed NSH amplitude variations, whereas the tilted thermally unstable disk model accounts for most of the observed cycles and dip patterns, particularly the disappearance of the IW And phenomenon when NSHs become undetectable. However, no significant changes in the NSH period were observed. These findings suggest a potential link between the IW And phenomenon and a tilted disk. Further improvements in simulations are needed to better understand the detailed dynamics of these cycles., Comment: 22 pages, 9 figures and 3 tables

Whisper and other large-scale automatic speech recognition models have made significant progress in performance. However, their performance on many low-resource languages, such as Kazakh, is not satisfactory. It is worth researching how to utilize low-cost data to improve the performance of Whisper on under-represented languages. In this study, we utilized easily accessible unpaired speech and text data and combined the language model GPT with Whisper on Kazakh. We implemented end of transcript (EOT) judgment modification and hallucination penalty to improve the performance of speech recognition. Further, we employed the decoding average token log probability as a criterion to select samples from unlabeled speech data and used pseudo-labeled data to fine-tune the model to further improve its performance. Ultimately, we achieved more than 10\% absolute WER reduction in multiple experiments, and the whole process has the potential to be generalized to other under-represented languages., Comment: Accepted by INTERSPEECH 2024;Minor typo correction

Heartbeat stars (HBSs) with tidally excited oscillations (TEOs) are ideal astrophysical laboratories for studying the internal properties of the systems. In this paper, five new HBSs exhibiting TEOs are discovered using TESS photometric data. The orbital parameters are derived using a corrected version of Kumar et al.'s model based on the Markov Chain Monte Carlo (MCMC) method. The TEOs in these objects are examined, and their pulsation phases and modes are identified. The pulsation phases of the TEOs in TIC 266809405, TIC 266894805, and TIC 412881444 are consistent with the dominant $l=2$, $m=0$, or $\pm2$ spherical harmonic. For TIC 11619404, although the TEO phase is close to the $m=+2$ mode, the $m = 0$ mode cannot be excluded because of the low inclination in this system. The TEO phase in TIC 447927324 shows a large deviation ($>2\sigma$) from the adiabatic expectations, suggesting that it is expected to be a traveling wave rather than a standing wave. In addition, these TEOs occur at relatively low orbital harmonics, and we cautiously suggest that this may be an observational bias. These objects are valuable sources for studying the structure and evolution of eccentricity orbit binaries and extending the TESS HBS catalog with TEOs., Comment: 9pages, 5figures

Heartbeat stars (HBSs) are ideal astrophysical laboratories to study the formation and evolution of binary stars in eccentric orbits and the internal structural changes of their components under strong tidal action. We discover 23 new HBSs based on TESS photometric data. The orbital parameters, including orbital period, eccentricity, orbital inclination, argument of periastron, and epoch of periastron passage of these HBSs are derived by using a corrected version of Kumar et al.'s model based on the Markov Chain Monte Carlo (MCMC) method. The preliminary results show that these HBSs have orbital periods in the range from 2.7 to 20 days and eccentricities in the range from 0.08 to 0.70. The eccentricity-period relation of these objects shows a positive correlation between eccentricity and period, and also shows the existence of orbital circularization. The Hertzsprung-Russell diagram shows that the HBSs are not all located in a particular area. The distribution of the derived parameters suggests a selection bias within the TESS survey towards massive HBSs with shorter orbital periods, higher temperatures and luminosities. These objects are a very useful source to study the structure and evolution of eccentricity orbit binaries and to extend the TESS HBS catalog., Comment: 10 pages, 7 figures

The data circulation is a complex scenario involving a large number of participants and different types of requirements, which not only has to comply with the laws and regulations, but also faces multiple challenges in technical and business areas. In order to systematically and comprehensively address these issues, it is essential to have a comprehensive and profound understanding of 'data circulation'. The traditional analysis method tends to proceed based on the traditional circulation model of commodities, that is, tangible objects, which has some defects and shortcomings, and tends to be a formalized approach, which is faced numerous challenges in practice. This paper analyzes the circulation of data with a philosophical approach, obtains the new explication of data and executing entity, and provides a new definition of the concepts of data utilization and data key stakeholders (objects). At the same time, it puts forward the idea of ``data alienation'', and constructs a new interpretive framework of ``data circulation''. Based on the framework of this interpretation, it is clearly proposed that ``data alienation'' is the core of ``data circulation'', benefit distribution is the driving force, and legal compliance is the foundation, and further discussed the three modes of ``data circulation''. It further discusses the three modes of ``data circulation''. It is pointed out that ``data circulation'' is different from traditional ``commodity circulation''. To achieve ``data circulation'',a comprehensive information infrastructure needs to be established. from a theoretical point of view, it lays a solid foundation for the development of ``data circulation''.

Rocket recycling is a crucial pursuit in aerospace technology, aimed at reducing costs and environmental impact in space exploration. The primary focus centers on rocket landing control, involving the guidance of a nonlinear underactuated rocket with limited fuel in real-time. This challenging task prompts the application of reinforcement learning (RL), yet goal-oriented nature of the problem poses difficulties for standard RL algorithms due to the absence of intermediate reward signals. This paper, for the first time, significantly elevates the success rate of rocket landing control from 8% with a baseline controller to 97% on a high-fidelity rocket model using RL. Our approach, called Random Annealing Jump Start (RAJS), is tailored for real-world goal-oriented problems by leveraging prior feedback controllers as guide policy to facilitate environmental exploration and policy learning in RL. In each episode, the guide policy navigates the environment for the guide horizon, followed by the exploration policy taking charge to complete remaining steps. This jump-start strategy prunes exploration space, rendering the problem more tractable to RL algorithms. The guide horizon is sampled from a uniform distribution, with its upper bound annealing to zero based on performance metrics, mitigating distribution shift and mismatch issues in existing methods. Additional enhancements, including cascading jump start, refined reward and terminal condition, and action smoothness regulation, further improve policy performance and practical applicability. The proposed method is validated through extensive evaluation and Hardware-in-the-Loop testing, affirming the effectiveness, real-time feasibility, and smoothness of the proposed controller., Comment: IROS 2024 Oral

The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts., Comment: 11 pages, 6 figures. This is the author's version of the article that has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics (TVCG)

Intelligence is key to advancing integrated circuit (IC) fabrication. Recent breakthroughs in Large Multimodal Models (LMMs) have unlocked unparalleled abilities in understanding images and text, fostering intelligent fabrication. Leveraging the power of LMMs, we introduce FabGPT, a customized IC fabrication large multimodal model for wafer defect knowledge query. FabGPT manifests expertise in conducting defect detection in Scanning Electron Microscope (SEM) images, performing root cause analysis, and providing expert question-answering (Q&A) on fabrication processes. FabGPT matches enhanced multimodal features to automatically detect minute defects under complex wafer backgrounds and reduce the subjectivity of manual threshold settings. Besides, the proposed modulation module and interactive corpus training strategy embed wafer defect knowledge into the pre-trained model, effectively balancing Q&A queries related to defect knowledge and original knowledge and mitigating the modality bias issues. Experiments on in-house fab data (SEM-WaD) show that our FabGPT achieves significant performance improvement in wafer defect detection and knowledge querying.

Despite their nearly universal adoption for large language models, the internal workings of transformers are not well understood. We aim to better understand the impact of removing or reorganizing information throughout the layers of a pretrained transformer. Such an understanding could both yield better usage of existing models as well as to make architectural improvements to produce new variants. We present a series of empirical studies on frozen models that show that the lower and final layers of pretrained transformers differ from middle layers, but that middle layers have a surprising amount of uniformity. We further show that some classes of problems have robustness to skipping layers, running the layers in an order different from how they were trained, or running the layers in parallel. Our observations suggest that even frozen pretrained models may gracefully trade accuracy for latency by skipping layers or running layers in parallel., Comment: 12 pages total, including references and appendices

Synthesizing realistic 3D indoor scenes is a challenging task that traditionally relies on manual arrangement and annotation by expert designers. Recent advances in autoregressive models have automated this process, but they often lack semantic understanding of the relationships and hierarchies present in real-world scenes, yielding limited performance. In this paper, we propose Forest2Seq, a framework that formulates indoor scene synthesis as an order-aware sequential learning problem. Forest2Seq organizes the inherently unordered collection of scene objects into structured, ordered hierarchical scene trees and forests. By employing a clustering-based algorithm and a breadth-first traversal, Forest2Seq derives meaningful orderings and utilizes a transformer to generate realistic 3D scenes autoregressively. Experimental results on standard benchmarks demonstrate Forest2Seq's superiority in synthesizing more realistic scenes compared to top-performing baselines, with significant improvements in FID and KL scores. Our additional experiments for downstream tasks and ablation studies also confirm the importance of incorporating order as a prior in 3D scene generation., Comment: ECCV 2024

Tilted disk precession exists in different objects. Negative superhumps (NSHs) in cataclysmic variable stars (CVs) are believed to arise from the interaction between the reverse precession of a tilted disk and the streams from the secondary star.Utilizing TESS photometry, we present a comprehensive investigation into the tilted disk precession and NSHs in the dwarf nova (DN) HS 2325+8205, employing eclipse minima, eclipse depths, NSH frequencies, and NSH amplitudes and the correlation between them as the windows. We identified NSHs with a period of 0.185671(17) days in HS 2325+8205. The NSH frequency exhibits variability with a period of 3.943(9) days, akin to the tilted disk precession period validated in novae-like stars (NLs, SDSS J0812) and intermediate polars (IPs, TV Col).The O-C of eclipse minima were similarly found to vary cyclically in period 4.135(5) days, characterized by a faster rise than fall. Furthermore, the NSH amplitude exhibits complex and diverse variations, which may be linked to changes in the disk radius, mass transfer rate, and the apparent area of the hot spot. For the first time in DNe, we observe bi-periodic variations in eclipse depth (P1= 4.131(4) d and P2= 2.065(2) d ~ Pprec/2), resembling those seen in IPs, suggesting that variations with P2 are not attributable to an accretion curtain, as previously suspected. Moreover, NSH amplitude and eclipse depth decrease with increasing NSH frequency, while NSH amplitude correlates positively with eclipse depth.These complex variations observed across multiple observational windows provide substantial evidence for understanding of tilted disk precession and NSHs., Comment: 19 pages, 10 figures and 3 tables; accepted for publication in ApJ

Low-light videos often exhibit spatiotemporal incoherent noise, compromising visibility and performance in computer vision applications. One significant challenge in enhancing such content using deep learning is the scarcity of training data. This paper introduces a novel low-light video dataset, consisting of 40 scenes with various motion scenarios under two distinct low-lighting conditions, incorporating genuine noise and temporal artifacts. We provide fully registered ground truth data captured in normal light using a programmable motorized dolly and refine it via an image-based approach for pixel-wise frame alignment across different light levels. We provide benchmarks based on four different technologies: convolutional neural networks, transformers, diffusion models, and state space models (mamba). Our experimental results demonstrate the significance of fully registered video pairs for low-light video enhancement (LLVE) and the comprehensive evaluation shows that the models trained with our dataset outperform those trained with the existing datasets. Our dataset and links to benchmarks are publicly available at https://doi.org/10.21227/mzny-8c77., Comment: arXiv admin note: text overlap with arXiv:2402.01970

Neural image representations have recently emerged as a promising technique for storing, streaming, and rendering visual data. Coupled with learning-based workflows, these novel representations have demonstrated remarkable visual fidelity and memory efficiency. However, existing neural image representations often rely on explicit uniform data structures without content adaptivity or computation-intensive implicit models, limiting their adoption in real-time graphics applications. Inspired by recent advances in radiance field rendering, we propose Image-GS, a content-adaptive image representation. Using anisotropic 2D Gaussians as the basis, Image-GS shows high memory efficiency, supports fast random access, and offers a natural level of detail stack. Leveraging a tailored differentiable renderer, Image-GS fits a target image by adaptively allocating and progressively optimizing a set of 2D Gaussians. The generalizable efficiency and fidelity of Image-GS are validated against several recent neural image representations and industry-standard texture compressors on a diverse set of images. Notably, its memory and computation requirements solely depend on and linearly scale with the number of 2D Gaussians, providing flexible controls over the trade-off between visual fidelity and run-time efficiency. We hope this research offers insights for developing new applications that require adaptive quality and resource control, such as machine perception, asset streaming, and content generation.

The merger of compact binary stars produces short gamma-ray bursts (sGRBs), involving channels such as neutron star - neutron star (BNS) and neutron star - black hole (NS-BH). The association between sGRB 170817A and gravitational wave GW 170817 provides reliable evidence for the BNS channel. The spatial distribution and merger rate differ between BNS mergers and NS-BH mergers. Some speculations suggest that sGRBs with extended emission (EE) may represent another distinct population. We compared the offset distributions of these two types of samples and found that they follow the same distribution. Utilizing non-parametric methods, we investigated the origin of these burst types in terms of their formation rate. We examined the luminosity function and formation rate of sGRBs without any assuming. The luminosity function can be described as $\psi(L_{0}) \propto L_{0}^{-0.09 \pm 0.01}$ for $L_{0} < L_0^b$ ($\psi(L_{0}) \propto L_{0}^{-0.57 \pm 0.02}$ for $L_{0} > L_0^b$) for standard sGRBs and $\psi(L_{0}) \propto L_{0}^{-0.11 \pm 0.004}$ for $L_{0} < L_0^b$ ($\psi(L_{0}) \propto L_{0}^{-0.61 \pm 0.01}$ for $L_{0} > L_0^b$) for sGRBs with EE. The formation rate is characterized as $\rho(z) \propto (1 + z)^{-4.21 \pm 0.22}$ for $z < 0.8$ and $\rho(z) \propto (1 + z)^{-0.22 \pm 0.74}$ for $0.8 < z < 3$ for standard sGRBs, while for sGRBs with EE, it is $\rho(z) \propto (1 + z)^{-4.30 \pm 0.13}$ for $z < 0.8$ and $\rho(z) \propto (1 + z)^{-0.33 \pm 0.66}$ for $0.8 < z < 3$. Based on these findings, we suggest that there is no significant difference in the progenitor stars of sGRBs with and without EE, considering the spatial offset and formation rate perspectives., Comment: 16 pages, 11 figures and 2 tables

Accurately digitizing physical objects is central to many applications, including virtual/augmented reality, industrial design, and e-commerce. Prior research has demonstrated efficient and faithful reconstruction of objects' geometric shapes and visual appearances, which suffice for digitally representing rigid objects. In comparison, physical properties, such as elasticity and pressure, are also indispensable to the behavioral fidelity of digitized deformable objects. However, existing approaches to acquiring these quantities either rely on invasive specimen collection or expensive/bulky laboratory setups, making them inapplicable to consumer-level usage. To fill this gap, we propose a wearable and non-invasive computing framework that allows users to conveniently estimate the material elasticity and internal pressure of deformable objects through finger touches. This is achieved by modeling their local surfaces as pressurized elastic shells and analytically deriving the two physical properties from finger-induced wrinkling patterns. Together with photogrammetry-reconstructed geometry and textures, the two estimated physical properties enable us to faithfully replicate the motion and deformation behaviors of several deformable objects. For the pressure estimation, our model achieves a relative error of 3.5%. In the interaction experiments, the virtual-physical deformation discrepancy measures less than 10.1%. Generalization to objects of irregular shape further demonstrates the potential of our approach in practical applications. We envision this work to provide insights for and motivate research toward democratizing the ubiquitous and pervasive digitization of our physical surroundings in daily, industrial, and scientific scenarios., Comment: 10 pages, 6 figures

In recent years, the increasing demand for dynamic 3D assets in design and gaming applications has given rise to powerful generative pipelines capable of synthesizing high-quality 4D objects. Previous methods generally rely on score distillation sampling (SDS) algorithm to infer the unseen views and motion of 4D objects, thus leading to unsatisfactory results with defects like over-saturation and Janus problem. Therefore, inspired by recent progress of video diffusion models, we propose to optimize a 4D representation by explicitly generating multi-view videos from one input image. However, it is far from trivial to handle practical challenges faced by such a pipeline, including dramatic temporal inconsistency, inter-frame geometry and texture diversity, and semantic defects brought by video generation results. To address these issues, we propose DG4D, a novel multi-stage framework that generates high-quality and consistent 4D assets without score distillation. Specifically, collaborative techniques and solutions are developed, including an attention injection strategy to synthesize temporal-consistent multi-view videos, a robust and efficient dynamic reconstruction method based on Gaussian Splatting, and a refinement stage with diffusion prior for semantic restoration. The qualitative results and user preference study demonstrate that our framework outperforms the baselines in generation quality by a considerable margin. Code will be released at \url{https://github.com/jasongzy/EG4D}.

We reported a cyclic variation of $O-C$ diagram with a semi-amplitude of 0.0033 days and a period of 1.05 years for the pulsating eclipsing binary HZ Dra. The cyclic variation can be explained by the light travel-time effect via the presence of a close-in third body orbiting around HZ Dra in an elliptical orbit with a maximum semi-major axis of 0.92 au. Based on the W-D code, the contribution of the third light to the total system is determined to be 29 $\%$, which is in agreement with the estimated value. Our light curve modelling indicates an evolving hot and cool spot on the surface of the primary and secondary components, respectively. Their positions are roughly symmetrical to the inner Lagrangian point L1, which could be used to explain the variation in the O$^{'}$Connell effect. Our frequency analysis detects 1 radial p-mode, 7 non-radial p-modes and 1 non-radial g-mode. In addition, a total of 6 multiplets are identified, spaced by the orbital frequency, which can be explained as a tidally split mode caused by the equilibrium tides of the close binary system with a circular orbit. These pulsating features suggest that the primary of HZ Dra is a $\delta$ Scuti star, pulsating in both p- and g-mode and influenced by tidal forces., Comment: 15 pages, 11 figures. The first version is Under Review in ApJ

The rising popularity of multimodal large language models (MLLMs) has sparked a significant increase in research dedicated to evaluating these models. However, current evaluation studies predominantly concentrate on the ability of models to comprehend and reason within a unimodal (vision-only) context, overlooking critical performance evaluations in complex multimodal reasoning tasks that integrate both visual and text contexts. Furthermore, tasks that demand reasoning across multiple modalities pose greater challenges and require a deep understanding of multimodal contexts. In this paper, we introduce a comprehensive assessment framework named MM-InstructEval, which integrates a diverse array of metrics to provide an extensive evaluation of the performance of various models and instructions across a broad range of multimodal reasoning tasks with vision-text contexts. MM-InstructEval enhances the research on the performance of MLLMs in complex multimodal reasoning tasks, facilitating a more thorough and holistic zero-shot evaluation of MLLMs. We firstly utilize the "Best Performance" metric to determine the upper performance limit of each model across various datasets. The "Mean Relative Gain" metric provides an analysis of the overall performance across different models and instructions, while the "Stability" metric evaluates their sensitivity to variations. Historically, the research has focused on evaluating models independently or solely assessing instructions, overlooking the interplay between models and instructions. To address this gap, we introduce the "Adaptability" metric, designed to quantify the degree of adaptability between models and instructions. Evaluations are conducted on 31 models (23 MLLMs) across 16 multimodal datasets, covering 6 tasks, with 10 distinct instructions. The extensive analysis enables us to derive novel insights., Comment: Under review, the new version of MM-BigBench: arXiv:2310.09036

We present the eclipsing Heartbeat Star KIC 9704906 with tidally excited oscillations (TEOs) and gamma Doradus/delta Scuti hybrid pulsations. The derived parameters show that it has an orbital period of $P$=8.7529108(1) days, a high eccentricity of $e$=0.467(3), and a high inclination of $i$=78$^{\circ}$.81(6). The mass ratio $q$=0.981(5), the relative radii (radius divided by semi-major axis) $r_1$=0.0639(2), and $r_2$=0.0715(4) indicate that the secondary component has a less mass and a larger radius, and may have evolved off the main sequence. The eight derived TEO candidates, $n$ = 3, 4, 5, 6, 7, 12, 40, and 44 harmonics, are consistent with or close to the dominant spherical harmonic $l=2$, $m=0$, or $\pm2$, assuming that the spin and orbital axes are aligned, and the pulsations are adiabatic and standing waves. We also identify ten independent frequency candidates, but one of them, $\mathit{f_{7}}$, is more like a modulation of a quasi-periodic signal and the orbits. According to the g-mode frequencies, we find that the rotation period of one component is 11.52(29) days. Although the masses and radii cannot be further constrained due to the lack of sufficient high-precision spectra, the fascinating phenomena in the Fourier spectra are evident and valuable in this system.

Spiking Neural Networks (SNNs) aim to bridge the gap between neuroscience and machine learning by emulating the structure of the human nervous system. However, like convolutional neural networks, SNNs are vulnerable to adversarial attacks. To tackle the challenge, we propose a biologically inspired methodology to enhance the robustness of SNNs, drawing insights from the visual masking effect and filtering theory. First, an end-to-end SNN-based image purification model is proposed to defend against adversarial attacks, including a noise extraction network and a non-blind denoising network. The former network extracts noise features from noisy images, while the latter component employs a residual U-Net structure to reconstruct high-quality noisy images and generate clean images. Simultaneously, a multi-level firing SNN based on Squeeze-and-Excitation Network is introduced to improve the robustness of the classifier. Crucially, the proposed image purification network serves as a pre-processing module, avoiding modifications to classifiers. Unlike adversarial training, our method is highly flexible and can be seamlessly integrated with other defense strategies. Experimental results on various datasets demonstrate that the proposed methodology outperforms state-of-the-art baselines in terms of defense effectiveness, training time, and resource consumption., Comment: 8 pages, 5 figures, ECAI2024 under review

Pretrained language models underpin several AI applications, but their high computational cost for training limits accessibility. Initiatives such as BLOOM and StarCoder aim to democratize access to pretrained models for collaborative community development. However, such existing models face challenges: limited multilingual capabilities, continual pretraining causing catastrophic forgetting, whereas pretraining from scratch is computationally expensive, and compliance with AI safety and development laws. This paper presents Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code. Continually pretrained from StarCoderPlus on 435 billion additional tokens, Aurora-M surpasses 2 trillion tokens in total training token count. It is the first open-source multilingual model fine-tuned on human-reviewed safety instructions, thus aligning its development not only with conventional red-teaming considerations, but also with the specific concerns articulated in the Biden-Harris Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Aurora-M is rigorously evaluated across various tasks and languages, demonstrating robustness against catastrophic forgetting and outperforming alternatives in multilingual settings, particularly in safety evaluations. To promote responsible open-source LLM development, Aurora-M and its variants are released at https://huggingface.co/collections/aurora-m/aurora-m-models-65fdfdff62471e09812f5407 ., Comment: Preprint

3-dB couplers, which are commonly used in photonic integrated circuits for on-chip information processing, precision measurement, and quantum computing, face challenges in achieving robust performance due to their limited 3-dB bandwidths and sensitivity to fabrication errors. To address this, we introduce topological physics to nanophotonics, developing a framework for topological 3-dB couplers. These couplers exhibit broad working wavelength range and robustness against fabrication dimensional errors. By leveraging valley-Hall topology and mirror symmetry, the photonic-crystal-slab couplers achieve ideal 3-dB splitting characterized by a wavelength-insensitive scattering matrix. Tolerance analysis confirms the superiority on broad bandwidth of 48 nm and robust splitting against dimensional errors of 20 nm. We further propose a topological interferometer for on-chip distance measurement, which also exhibits robustness against dimensional errors. This extension of topological principles to the fields of interferometers, may open up new possibilities for constructing robust wavelength division multiplexing, temperature-drift-insensitive sensing, and optical coherence tomography applications., Comment: 20 pages, 4 figures

In the pursuit of advancing holographic display technology, we face a unique yet persistent roadblock: the inflexibility of learned holography in adapting to various hardware configurations. This is due to the variances in the complex optical components and system settings in existing holographic displays. Although the emerging learned approaches have enabled rapid and high-quality hologram generation, any alteration in display hardware still requires a retraining of the model. Our work introduces a configurable learned model that interactively computes 3D holograms from RGB-only 2D images for a variety of holographic displays. The model can be conditioned to predefined hardware parameters of existing holographic displays such as working wavelengths, pixel pitch, propagation distance, and peak brightness without having to retrain. In addition, our model accommodates various hologram types, including conventional single-color and emerging multi-color holograms that simultaneously use multiple color primaries in holographic displays. Notably, we enabled our hologram computations to rely on identifying the correlation between depth estimation and 3D hologram synthesis tasks within the learning domain for the first time in the literature. We employ knowledge distillation via a student-teacher learning strategy to streamline our model for interactive performance. Achieving up to a 2x speed improvement compared to state-of-the-art models while consistently generating high-quality 3D holograms with different hardware configurations., Comment: 14 pages, 5 figures

We present a novel application of evolutionary algorithms to automate the creation of powerful foundation models. While model merging has emerged as a promising approach for LLM development due to its cost-effectiveness, it currently relies on human intuition and domain knowledge, limiting its potential. Here, we propose an evolutionary approach that overcomes this limitation by automatically discovering effective combinations of diverse open-source models, harnessing their collective intelligence without requiring extensive additional training data or compute. Our approach operates in both parameter space and data flow space, allowing for optimization beyond just the weights of the individual models. This approach even facilitates cross-domain merging, generating models like a Japanese LLM with Math reasoning capabilities. Surprisingly, our Japanese Math LLM achieved state-of-the-art performance on a variety of established Japanese LLM benchmarks, even surpassing models with significantly more parameters, despite not being explicitly trained for such tasks. Furthermore, a culturally-aware Japanese VLM generated through our approach demonstrates its effectiveness in describing Japanese culture-specific content, outperforming previous Japanese VLMs. This work not only contributes new state-of-the-art models back to the open-source community, but also introduces a new paradigm for automated model composition, paving the way for exploring alternative, efficient approaches to foundation model development.

Diffusion models offer unprecedented image generation capabilities given just a text prompt. While emerging control mechanisms have enabled users to specify the desired spatial arrangements of the generated content, they cannot predict or control where viewers will pay more attention due to the complexity of human vision. Recognizing the critical necessity of attention-controllable image generation in practical applications, we present a saliency-guided framework to incorporate the data priors of human visual attention into the generation process. Given a desired viewer attention distribution, our control module conditions a diffusion model to generate images that attract viewers' attention toward desired areas. To assess the efficacy of our approach, we performed an eye-tracked user study and a large-scale model-based saliency analysis. The results evidence that both the cross-user eye gaze distributions and the saliency model predictions align with the desired attention distributions. Lastly, we outline several applications, including interactive design of saliency guidance, attention suppression in unwanted regions, and adaptive generation for varied display/viewing conditions.

Quantization is a crucial technique for deploying deep learning models on resource-constrained devices, such as embedded FPGAs. Prior efforts mostly focus on quantizing matrix multiplications, leaving other layers like BatchNorm or shortcuts in floating-point form, even though fixed-point arithmetic is more efficient on FPGAs. A common practice is to fine-tune a pre-trained model to fixed-point for FPGA deployment, but potentially degrading accuracy. This work presents QFX, a novel trainable fixed-point quantization approach that automatically learns the binary-point position during model training. Additionally, we introduce a multiplier-free quantization strategy within QFX to minimize DSP usage. QFX is implemented as a PyTorch-based library that efficiently emulates fixed-point arithmetic, supported by FPGA HLS, in a differentiable manner during backpropagation. With minimal effort, models trained with QFX can readily be deployed through HLS, producing the same numerical results as their software counterparts. Our evaluation shows that compared to post-training quantization, QFX can quantize models trained with element-wise layers quantized to fewer bits and achieve higher accuracy on both CIFAR-10 and ImageNet datasets. We further demonstrate the efficacy of multiplier-free quantization using a state-of-the-art binarized neural network accelerator designed for an embedded FPGA (AMD Xilinx Ultra96 v2). We plan to release QFX in open-source format.

Document-level Relation Triplet Extraction (DocRTE) is a fundamental task in information systems that aims to simultaneously extract entities with semantic relations from a document. Existing methods heavily rely on a substantial amount of fully labeled data. However, collecting and annotating data for newly emerging relations is time-consuming and labor-intensive. Recent advanced Large Language Models (LLMs), such as ChatGPT and LLaMA, exhibit impressive long-text generation capabilities, inspiring us to explore an alternative approach for obtaining auto-labeled documents with new relations. In this paper, we propose a Zero-shot Document-level Relation Triplet Extraction (ZeroDocRTE) framework, which generates labeled data by retrieval and denoising knowledge from LLMs, called GenRDK. Specifically, we propose a chain-of-retrieval prompt to guide ChatGPT to generate labeled long-text data step by step. To improve the quality of synthetic data, we propose a denoising strategy based on the consistency of cross-document knowledge. Leveraging our denoised synthetic data, we proceed to fine-tune the LLaMA2-13B-Chat for extracting document-level relation triplets. We perform experiments for both zero-shot document-level relation and triplet extraction on two public datasets. The experimental results illustrate that our GenRDK framework outperforms strong baselines., Comment: Accepted by WWW 2024

Only bosonic molecular species have been directly laser cooled to date, primarily due to an abundance of bosonic isotopes in nature and to their simpler hyperfine structure. Fermionic molecules provide new opportunities for ultracold chemistry, quantum simulation, and precision measurements. Here we report direct laser cooling of a fermionic molecular isotopologue, calcium monodeuteride (CaD). With a nuclear spin I = 1, only 5 hyperfine states need to be addressed for rotational closure in optical cycling. These hyperfine states are unresolved for typical experimental linewidths. We present a method for efficiently producing alkaline-earth metal hydrides and deuterides. We demonstrate rotational closure and show magnetically assisted Sisyphus cooling in one dimension for a beam of CaD molecules. Our results indicate that the experimental complexity for laser cooling CaD is similar to that of calcium monohydride (CaH). Laser cooling of CaD is a promising first step for production of ultracold and trapped atomic deuterium., Comment: 8 pages, 5 figures

Tidally excited oscillations (TEOs) in Heartbeat Stars (HBSs) are an essential probe of the internal properties of the systems, but their potential has yet to be fully exploited. Based on the orbital parameters of TEO candidates from our previous works, we identify the pulsation phases and amplitudes of TEOs in fourteen Kepler HBSs. Most pulsation phases of most systems can be explained by the dominant being $l=2$, $m=0$, or $\pm2$ spherical harmonic, assuming that the spin and orbital axes are aligned, and the pulsations are adiabatic and standing waves. The largest deviation ($>6\sigma$) occurs in KIC 8459354, which can be explained by the spin-orbit misalignment, and KIC 5877364 has a similar scenario. For KIC 11122789, almost half of the harmonics show large deviations; we cautiously suggest that these harmonics may not be considered TEO candidates. A similar scenario also exists in KIC 6290740. This phases and mode identification approach can also be used inversely to verify the TEO candidates derived by the Fourier analysis. Furthermore, the harmonics with large deviations ($>2\sigma$) in KIC 4377638, KIC 5090937, and KIC 11403032 can be expected to be travelling waves rather than standing waves. In addition, we also suggest that the apsidal motion could cause large deviations in TEO phases from theoretical values., Comment: published on MNRAS

TV Col is a long-period eclipsing intermediate polar (IPs) prototype star for the negative superhump (NSH) system. We investigate the eclipse minima, eclipse depth, and NSH amplitude based on TESS photometry. Using the same analytical method as SDSS J081256.85+191157.8, we find periodic variations of the O-C for eclipse minima and NSH amplitudes with periods of 3.939(25) d and 3.907(30) d, respectively. The periodic variation of the NSH amplitude of TV Col confirms that periodic NSH amplitude changes in response to the tilted disk precession may be universal, which is another evidence that the origin of the NSHs is related to the tilted disk precession. We suggest that the NSH amplitude variation may be similar to the superorbital signal, coming from the periodic change in visual brightness of the energy released by streams touching the tilted disk with tilted disk precession. Finally, we find for the first time that the eclipse depth exhibits bi-periodic variations with periods of P1 = 3.905(11) d and P2 = 1.953(4) d, respectively. P2 is about half of P1 and the disk precession period (P1~ Pprec ~ 2 * P2). We suggest that P1 may come from the periodic change in the brightness of the eclipse center due to tilted disk precession, while P2 may come from two accretion curtains precessing together with the tilted disk, but more verification and discussion are necessary. The discovery of bi-periodic variations in eclipse depth provides a new window for studying IPs and tilted disk precession., Comment: 18 pages, 7 figures, 3 tables; Accepted for publication in The Astrophysical Journal

Motivated by the first image of a black hole captured by the Event Horizon Telescope (EHT), there has been a surge of research using observations of black hole shadows to test theories of gravity. In this paper, we carry out a study related to the shadow of Kerr black holes surrounded by a cloud of strings in Rastall gravity, which deviates from the Kerr black hole due to the presence of the string parameter $a_0$ and the parameter $\beta$. The horizons, ergospheres, and photon region of the black hole are shown. Moreover, we explore the shadow and observations of the black hole, which are closely linked to the parameters $a_0$ and $\beta$. By treating \text{M87*} as a Kerr black hole surrounded by a cloud of strings under Rastall gravity, we constrain the black hole parameters using the EHT observations. For a given $\beta$, the circularity deviation of the black hole obeys $\Delta C\lesssim0.1$ in all regions. The angular diameter $\theta_{d}=42\pm3\mu as$ provides the upper bound of parameters $a$ and $a_0$ for fixed $\beta$. The shadow axis ratio satisfies the observation data of EHT ($1

Recent observational data from the Event Horizon Telescope (EHT) collaboration provide convincing realistic evidence for the existence of black hole rotation. From a phenomenological perspective, a recently proposed stable rotating regular (SRR) black hole circumvents the theoretical flaws of the Kerr solution. For the purpose of obtaining observational signatures of this black hole, we study its gravitational lensing effect. In the strong field limit, we calculate the deflection angle of light, the radius of the photon sphere, and other observables. The observables include the relativistic image position, separation, magnification, and time delays between different images. Then, by modeling M87* and Sgr A* as the SRR black hole, we compute their observables and evaluate the deviation of the observables from the Kerr case. In the weak field limit, we calculate the light deflection angle of M87* and Sgr A* via the Gauss-Bonnet theorem (GBT). With the growth of deviation parameter $e$, the gravitational lensing effect in the weak field limit intensifies monotonically, and the gravitational lensing effect in the strong field limit changes dramatically only at high spins. Our research may contribute to distinguish between SRR black holes from Kerr black holes under higher-precision astronomical observations., Comment: 24 pages, 10 figures, 4 tables

In this article, we introduce inhomogeneous variable Triebel-Lizorkin spaces, $F_{p(\cdot),q(\cdot)}^{\alpha(\cdot),H}(\mathbb R^n)$, associated with the Hermite operator $H:=-\Delta+|x|^2$, where $\Delta$ is the Laplace operator on $\mathbb R^n$, and mainly establish the molecular characterization of this space. As applications, we obtain some regularity results to fractional Hermite equations $$(-\Delta+|x|^2)^\sigma u=f,\quad (-\Delta+|x|^2+I)^\sigma u=f,$$ and the boundedness of spectral multiplier associated to the operator $H$ on the variable Triebel-Lizorkin space $F_{p(\cdot),q(\cdot)}^{\alpha(\cdot),H}(\mathbb R^n)$. Furthermore, we explain the relationship between $F_{p(\cdot),q(\cdot)}^{\alpha(\cdot),H}(\mathbb R^n)$ and the variable Triebel-Lizorkin spaces $F_{p(\cdot),q(\cdot)}^{\alpha(\cdot)}(\mathbb R^n)$ (introduced in Diening t al. J. Funct. Anal. 256(2009), 1731-1768.) via the atomic decomposition., Comment: 26 pages; This paper was submitted on Setember 29 of 2022 to a journal

The popularity of multimodal large language models (MLLMs) has triggered a recent surge in research efforts dedicated to evaluating these models. Nevertheless, existing evaluation studies of MLLMs primarily focus on the comprehension and reasoning of unimodal (vision) content, neglecting performance evaluations in the domain of multimodal (vision-language) content understanding. Beyond multimodal reasoning, tasks related to multimodal content comprehension necessitate a profound understanding of multimodal contexts, achieved through the multimodal interaction to obtain a final answer. In this paper, we introduce a comprehensive assessment framework called MM-BigBench, which incorporates a diverse range of metrics to offer an extensive evaluation of the performance of various models and instructions across a wide spectrum of diverse multimodal content comprehension tasks. Consequently, our work complements research on the performance of MLLMs in multimodal comprehension tasks, achieving a more comprehensive and holistic evaluation of MLLMs. To begin, we employ the Best Performance metric to ascertain each model's performance upper bound on different datasets. Subsequently, the Mean Relative Gain metric offers an assessment of the overall performance of various models and instructions, while the Stability metric measures their sensitivity. Furthermore, previous research centers on evaluating models independently or solely assessing instructions, neglecting the adaptability between models and instructions. We propose the Adaptability metric to quantify the adaptability between models and instructions. Our paper evaluates a total of 20 language models (14 MLLMs) on 14 multimodal datasets spanning 6 tasks, with 10 instructions for each task, and derives novel insights. Our code will be released at https://github.com/declare-lab/MM-BigBench., Comment: Underview

Balance laws of coupled bulk-surface-bulk fluid flows are considered. We carefully investigate how do the interface conditions influence the total mass, total momentum and total energy of the system.

Virtual Reality (VR) has the potential of becoming the next ubiquitous computing platform. Continued progress in the burgeoning field of VR depends critically on an efficient computing substrate. In particular, DRAM access energy is known to contribute to a significant portion of system energy. Today's framebuffer compression system alleviates the DRAM traffic by using a numerically lossless compression algorithm. Being numerically lossless, however, is unnecessary to preserve perceptual quality for humans. This paper proposes a perceptually lossless, but numerically lossy, system to compress DRAM traffic. Our idea builds on top of long-established psychophysical studies that show that humans cannot discriminate colors that are close to each other. The discrimination ability becomes even weaker (i.e., more colors are perceptually indistinguishable) in our peripheral vision. Leveraging the color discrimination (in)ability, we propose an algorithm that adjusts pixel colors to minimize the bit encoding cost without introducing visible artifacts. The algorithm is coupled with lightweight architectural support that, in real-time, reduces the DRAM traffic by 66.9\% and outperforms existing framebuffer compression mechanisms by up to 20.4\%. Psychophysical studies on human participants show that our system introduce little to no perceptual fidelity degradation., Comment: ASPLOS '24

Speed and consistency of target-shifting play a crucial role in human ability to perform complex tasks. Shifting our gaze between objects of interest quickly and consistently requires changes both in depth and direction. Gaze changes in depth are driven by slow, inconsistent vergence movements which rotate the eyes in opposite directions, while changes in direction are driven by ballistic, consistent movements called saccades, which rotate the eyes in the same direction. In the natural world, most of our eye movements are a combination of both types. While scientific consensus on the nature of saccades exists, vergence and combined movements remain less understood and agreed upon. We eschew the lack of scientific consensus in favor of proposing an operationalized computational model which predicts the speed of any type of gaze movement during target-shifting in 3D. To this end, we conduct a psychophysical study in a stereo VR environment to collect more than 12,000 gaze movement trials, analyze the temporal distribution of the observed gaze movements, and fit a probabilistic model to the data. We perform a series of objective measurements and user studies to validate the model. The results demonstrate its predictive accuracy, generalization, as well as applications for optimizing visual performance by altering content placement. Lastly, we leverage the model to measure differences in human target-changing time relative to the natural world, as well as suggest scene-aware projection depth. By incorporating the complexities and randomness of human oculomotor control, we hope this research will support new behavior-aware metrics for VR/AR display design, interface layout, and gaze-contingent rendering.

In this paper, we study the effects of the rotation parameter $a$, the twist parameter $n$, the string cloud parameter $b$, the quintessence state parameter $\omega _{q}$ and the charge parameter $q$ on the horizons and ergosphere of rotating and twisting charged black holes with cloud of strings and quintessence, and obtain the equations of motion and effective potential of the particle on the equatorial plane of black hole. We find that a particle with the critical angular momentum $L = L_C$ falling from infinity reaches the event horizon($u^r$=0) and satisfies the circular orbit condition $V_{e f f}={V_{e f f}}'=0$. We derive the expression of the centre-of-mass (CM) energy of two particles with different masses from the equations of particle motion. We show that the CM energy can be arbitrarily large for extremal black holes when the particles reach the event horizon by adjusting the angular momentum of the incident particles. However, for non-extremal black holes, the CM energy of particles that reach the event horizon cannot diverge by adjusting the angular momentum of the incident particles., Comment: 17 pages, 7 figures, 2 Tables

Negative superhumps (NSHs) are signals a few percent shorter than the orbital period of a binary star and are considered to originate from the reverse precession of the tilted disk. Based on TESS photometry, we find nine new cataclysmic variable stars (CVs) with NSHs. Three (ASAS J1420, TZ Per, and V392 Hya) of these stars similar to AH Her still have NSHs during dwarf nova outbursts, and the NSH amplitude varies with the outburst. The variation in the radius of the accretion disk partially explains this phenomenon. However, it does not explain the rebound of the NSH amplitude after the peak of the outburst and the fact that the NSH amplitude of the quiescence is sometimes not the largest, and it is necessary to combine the disk instability model (DIM) and add other ingredients. Therefore, we suggest that the variation of NSH amplitude with outburst can be an essential basis for studying the origin of NSHs and improving the DIM. The six ( ASASSN-V J1137, ASASSN-V J0611, 2MASS J0715, LAMOST J0925, ASASSN-17qj, and ZTF18acakuxo) remaining stars have been poorly studied, and for the first time we determine their orbital periods, NSHs and Superorbital signal (SOR) periods. The NSH periods and amplitudes of ASASSN-V J1137 and ASASSN-17qj vary with the SOR, and based on the comparison of the observations with the theory, we suggest that a single change in tilted disk angle does not explain the observations of the SOR and that other ingredients need to be considered as well., Comment: 21 pages, 12 figures, accepted for publication in the Astrophysical Journal

AH Her is a Z Cam-type dwarf nova with an orbital period of ~ 0.258 d. Dwarf nova oscillations and long-period dwarf nova oscillations have been detected, but no quasi-periodic oscillations (QPOs) and negative superhumps (NSHs) have been found. We investigated the association between NSHs, QPOs, and outbursts of AH Her based on \textit{TESS} photometry. We find for the first time the NSHs with a period of 0.24497(1) d in AH Her, and trace the variation of the amplitude and period of NSHs with the outburst. The amplitude of the NSHs is most significant at quiescence, weakening as the outburst rises, becoming undetectable at the top, rebounding and weakening at the plateau, and strengthening again as the outburst declines. The variation of the accretion disk radius can explain the NSHs amplitude variation except for the plateau, so we suggest that the relationship between NSHs amplitude and outburst can be used as a window to study the accretion disk instability and the origin of NSHs. In addition, we find the periodic variations in the amplitude, maxima, and shape of the NSHs ranging from 2.33(2) d to 2.68(5) d, which may be related to the precession of the tilted disk. Finally, we find QPOs at the top of AH Her's long outburst with ~ 2800 s similar to HS 2325+8205, suggesting that the presence of QPOs at the top of Z Cam's long outburst may be a general phenomenon, Comment: 20 pages, 11 figures, accepted for publication in the Astrophysical Journal

Heartbeat stars (HBSs) with tidally excited oscillations (TEOs) are ideal laboratories for studying the effect of equilibrium and dynamical tides. However, studies of TEOs in Kepler HBSs are rare due to the need for better modeling of the equilibrium tide in light curves. We revisit the HBSs in our previous work and study the TEOs in these HBSs based on the derived orbital parameters that could express the equilibrium tide. We also compile a set of analytic procedures to examine the harmonic and anharmonic TEOs in their Fourier spectra. The TEOs of 21 HBSs have been newly analyzed and presented. Twelve of these HBSs show prominent TEOs (signal-to-noise ratio of the harmonics $S/N \ge 10$). The relation between the orbital eccentricities and the harmonic number of the TEOs shows a positive correlation. The relation between the orbital periods and the harmonic number also shows a positive correlation. Furthermore, the distribution of HBSs with TEOs in the Hertzsprung-Russell (H-R) diagram shows that TEOs are more visible in hot stars with surface effective temperatures $T$ $\gtrsim$ 6500 K. These samples may also be valuable targets for future studies of the effect of tidal action in eccentric orbits., Comment: Published on ApJ

Ergonomic efficiency is essential to the mass and prolonged adoption of VR/AR experiences. While VR/AR head-mounted displays unlock users' natural wide-range head movements during viewing, their neck muscle comfort is inevitably compromised by the added hardware weight. Unfortunately, little quantitative knowledge for understanding and addressing such an issue is available so far. Leveraging electromyography devices, we measure, model, and predict VR users' neck muscle contraction levels (MCL) while they move their heads to interact with the virtual environment. Specifically, by learning from collected physiological data, we develop a bio-physically inspired computational model to predict neck MCL under diverse head kinematic states. Beyond quantifying the cumulative MCL of completed head movements, our model can also predict potential MCL requirements with target head poses only. A series of objective evaluations and user studies demonstrate its prediction accuracy and generality, as well as its ability in reducing users' neck discomfort by optimizing the layout of visual targets. We hope this research will motivate new ergonomic-centered designs for VR/AR and interactive graphics applications. Source code is released at: https://github.com/NYU-ICL/xr-ergonomics-neck-comfort., Comment: ACM SIGGRAPH 2023 Conference Proceedings

Recently, Neural Radiance Field (NeRF) has shown great success in rendering novel-view images of a given scene by learning an implicit representation with only posed RGB images. NeRF and relevant neural field methods (e.g., neural surface representation) typically optimize a point-wise loss and make point-wise predictions, where one data point corresponds to one pixel. Unfortunately, this line of research failed to use the collective supervision of distant pixels, although it is known that pixels in an image or scene can provide rich structural information. To the best of our knowledge, we are the first to design a nonlocal multiplex training paradigm for NeRF and relevant neural field methods via a novel Stochastic Structural SIMilarity (S3IM) loss that processes multiple data points as a whole set instead of process multiple inputs independently. Our extensive experiments demonstrate the unreasonable effectiveness of S3IM in improving NeRF and neural surface representation for nearly free. The improvements of quality metrics can be particularly significant for those relatively difficult tasks: e.g., the test MSE loss unexpectedly drops by more than 90% for TensoRF and DVGO over eight novel view synthesis tasks; a 198% F-score gain and a 64% Chamfer $L_{1}$ distance reduction for NeuS over eight surface reconstruction tasks. Moreover, S3IM is consistently robust even with sparse inputs, corrupted images, and dynamic scenes., Comment: ICCV 2023 main conference. Code: https://github.com/Madaoer/S3IM. 14 pages, 5 figures, 17 tables

The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine tune the components of the AR assistant., Comment: 11 pages, 8 figures. This is the author's version of the article of the article that has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics

In this work, we study the thermal fluctuation, deflection angle and greybody factor of the high-dimensional Schwarzschild black hole in scalar-tensor-vector gravity (STVG). Based on the correction of black hole entropy due to thermal fluctuation, we calculate some thermodynamic quantities associated with the correction of black hole entropy. The influence of the first-order and second-order corrections, spacetime dimensionality and STVG parameters on these thermodynamics quantities are discussed in detail. Additionally, by utilizing the Gauss-Bonnet theorem, the deflection angle is obtained in the weak field limit and the effect of two parameters on the results is visualized. Finally, we calculate the bounds on greybody factors of a massless scalar field., Comment: 13 pages, 11 figures