Search

Eating disorders are complex mental health conditions that affect millions of people around the world. Effective interventions on social media platforms are crucial, yet testing strategies in situ can be risky. We present a novel LLM-driven experimental testbed for simulating and assessing intervention strategies in ED-related discussions. Our framework generates synthetic conversations across multiple platforms, models, and ED-related topics, allowing for controlled experimentation with diverse intervention approaches. We analyze the impact of various intervention strategies on conversation dynamics across four dimensions: intervention type, generative model, social media platform, and ED-related community/topic. We employ cognitive domain analysis metrics, including sentiment, emotions, etc., to evaluate the effectiveness of interventions. Our findings reveal that civility-focused interventions consistently improve positive sentiment and emotional tone across all dimensions, while insight-resetting approaches tend to increase negative emotions. We also uncover significant biases in LLM-generated conversations, with cognitive metrics varying notably between models (Claude-3 Haiku $>$ Mistral $>$ GPT-3.5-turbo $>$ LLaMA3) and even between versions of the same model. These variations highlight the importance of model selection in simulating realistic discussions related to ED. Our work provides valuable information on the complex dynamics of ED-related discussions and the effectiveness of various intervention strategies., Comment: 9 pages, 5 figures

We present the discovery of a low-density planet transiting TOI-5688 A b, a high-metallicity M2V star. This planet was discovered as part of the search for transiting giant planets ($R \gtrsim8$ M$_\oplus$) through the Searching for GEMS (Giant Exoplanets around M-dwarf Stars) survey. The planet TOI-5688 A b was discovered with the Transiting Exoplanet Survey Satellite (TESS), and characterized with ground-based transits from Red Buttes Observatory (RBO), the Table Mountain Observatory of Pomona College, and radial velocity (RV) measurements with the Habitable-Zone Planet Finder (HPF) on the 10 m Hobby Eberly Telescope (HET) and NEID on the WIYN 3.5 m telescope. From the joint fit of transit and RV data, the mass of the planet is $124\pm24$ M$_\oplus$ and the radius is $10.4\pm0.7$ R$_\oplus$. This planet has a density of $0.61^{+0.20}_{-0.15}$ g/cm${}^3$, and is on a $\sim2.95$ day orbit around its host star. The spectroscopic and photometric analysis of the host star TOI-5688 A shows that it is a high metallicity ([Fe/H] $ = 0.47\pm0.16$ dex) M2V star, favoring the core-accretion formation pathway as the likely formation scenario for this planet. In this paper, we analyze potential mechanisms of planet formation in the context of the formation of TOI-5688 A b. Additionally, observations with Gaia suggest the presence of a wide-separation binary companion, TOI-5688 B, which has a projected separation of $\sim5"$ (1110 AU) and is an M4V. This makes TOI-5688 A b part of a growing number of GEMS in wide-separation binary systems., Comment: 20 pages, 7 figures, Submitted to AJ, Comments are welcome

The Near Infrared Spectrograph (NIRSpec) on the James Webb Space Telescope affords the astronomical community an unprecedented space-based Multi-Object Spectroscopy (MOS) capability through the use of a programmable array of micro-electro-mechanical shutters. Launched in December 2021 and commissioned along with a suite of other observatory instruments throughout the first half of 2022, NIRSpec has been carrying out scientific observations since the completion of commissioning. These observations would not be possible without a rigorous program of engineering operations to actively monitor and maintain NIRSpec's hardware health and safety and enhance instrument efficiency and performance. Although MOS is only one of the observing modes available to users, the complexity and uniqueness of the Micro-Shutter Assembly (MSA) that enables it has presented a variety of engineering challenges, including the appearance of electrical shorts that produce contaminating glow in exposures. Despite these challenges, the NIRSpec Multi-Object Spectrograph continues to perform robustly with no discernible degradation or significant reduction in capability. This paper provides an overview of the NIRSpec micro-shutter subsystem's state of health and operability and presents some of the developments that have taken place in its operation since the completion of instrument commissioning., Comment: SPIE Astronomical Telescopes + Instrumentation: Ground-based and Airborne Instrumentation for Astronomy X (Yokohama 2024), paper number 13092-38

Transiting giant exoplanets around M-dwarf stars (GEMS) are rare, owing to the low-mass host stars. However, the all-sky coverage of TESS has enabled the detection of an increasingly large number of them to enable statistical surveys like the \textit{Searching for GEMS} survey. As part of this endeavour, we describe the observations of six transiting giant planets, which includes precise mass measurements for two GEMS (K2-419Ab, TOI-6034b) and statistical validation for four systems, which includes validation and mass upper limits for three of them (TOI-5218b, TOI-5616b, TOI-5634Ab), while the fourth one -- TOI-5414b is classified as a `likely planet'. Our observations include radial velocities from the Habitable-zone Planet Finder on the Hobby-Eberly Telescope, and MAROON-X on Gemini-North, along with photometry and high-contrast imaging from multiple ground-based facilities. In addition to TESS photometry, K2-419Ab was also observed and statistically validated as part of the K2 mission in Campaigns 5 and 18, which provides precise orbital and planetary constraints despite the faint host star and long orbital period of $\sim 20.4$ days. With an equilibrium temperature of only 380 K, K2-419Ab is one of the coolest known well-characterized transiting planets. TOI-6034 has a late F-type companion about 40\arcsec~away, making it the first GEMS host star to have an earlier main-sequence binary companion. These confirmations add to the existing small sample of confirmed transiting GEMS., Comment: Accepted in AJ

Indoor monocular depth estimation helps home automation, including robot navigation or AR/VR for surrounding perception. Most previous methods primarily experiment with the NYUv2 Dataset and concentrate on the overall performance in their evaluation. However, their robustness and generalization to diversely unseen types or categories for indoor spaces (spaces types) have yet to be discovered. Researchers may empirically find degraded performance in a released pretrained model on custom data or less-frequent types. This paper studies the common but easily overlooked factor-space type and realizes a model's performance variances across spaces. We present InSpaceType Dataset, a high-quality RGBD dataset for general indoor scenes, and benchmark 13 recent state-of-the-art methods on InSpaceType. Our examination shows that most of them suffer from performance imbalance between head and tailed types, and some top methods are even more severe. The work reveals and analyzes underlying bias in detail for transparency and robustness. We extend the analysis to a total of 4 datasets and discuss the best practice in synthetic data curation for training indoor monocular depth. Further, dataset ablation is conducted to find out the key factor in generalization. This work marks the first in-depth investigation of performance variances across space types and, more importantly, releases useful tools, including datasets and codes, to closely examine your pretrained depth models. Data and code: https://depthcomputation.github.io/DepthPublic/, Comment: BMVC 2024. This version supersedes 2309.13516

Neural network models for guitar amplifier emulation, while being effective, often demand high computational cost and lack interpretability. Drawing ideas from physical amplifier design, this paper aims to address these issues with a new differentiable digital signal processing (DDSP)-based model, called ``DDSP guitar amp,'' that models the four components of a guitar amp (i.e., preamp, tone stack, power amp, and output transformer) using specific DSP-inspired designs. With a set of time- and frequency-domain metrics, we demonstrate that DDSP guitar amp achieves performance comparable with that of black-box baselines while requiring less than 10\% of the computational operations per audio sample, thereby holding greater potential for usages in real-time applications., Comment: Preprint paper

In Multimodal Language Models (MLMs), the cost of manually annotating high-quality image-text pair data for fine-tuning and alignment is extremely high. While existing multimodal data augmentation frameworks propose ways to augment image-text pairs, they either suffer from semantic inconsistency between texts and images, or generate unrealistic images, causing knowledge gap with real world examples. To address these issues, we propose Attribute-based Multimodal Data Augmentation (ARMADA), a novel multimodal data augmentation method via knowledge-guided manipulation of visual attributes of the mentioned entities. Specifically, we extract entities and their visual attributes from the original text data, then search for alternative values for the visual attributes under the guidance of knowledge bases (KBs) and large language models (LLMs). We then utilize an image-editing model to edit the images with the extracted attributes. ARMADA is a novel multimodal data generation framework that: (i) extracts knowledge-grounded attributes from symbolic KBs for semantically consistent yet distinctive image-text pair generation, (ii) generates visually similar images of disparate categories using neighboring entities in the KB hierarchy, and (iii) uses the commonsense knowledge of LLMs to modulate auxiliary visual attributes such as backgrounds for more robust representation of original entities. Our empirical results over four downstream tasks demonstrate the efficacy of our framework to produce high-quality data and enhance the model performance. This also highlights the need to leverage external knowledge proxies for enhanced interpretability and real-world grounding.

Converting different modalities into generalized text, which then serves as input prompts for large language models (LLMs), is a common approach for aligning multimodal models, particularly when pairwise data is limited. Text-centric alignment method leverages the unique properties of text as a modality space, transforming diverse inputs into a unified textual representation, thereby enabling downstream models to effectively interpret various modal inputs. This study evaluates the quality and robustness of multimodal representations in the face of noise imperfections, dynamic input order permutations, and missing modalities, revealing that current text-centric alignment methods can compromise downstream robustness. To address this issue, we propose a new text-centric adversarial training approach that significantly enhances robustness compared to traditional robust training methods and pre-trained multimodal foundation models. Our findings underscore the potential of this approach to improve the robustness and adaptability of multimodal representations, offering a promising solution for dynamic and real-world applications., Comment: arXiv admin note: text overlap with arXiv:2407.05036

XR devices running chat-bots powered by Large Language Models (LLMs) have tremendous potential as always-on agents that can enable much better productivity scenarios. However, screen based chat-bots do not take advantage of the the full-suite of natural inputs available in XR, including inward facing sensor data, instead they over-rely on explicit voice or text prompts, sometimes paired with multi-modal data dropped as part of the query. We propose a solution that leverages an attention framework that derives context implicitly from user actions, eye-gaze, and contextual memory within the XR environment. This minimizes the need for engineered explicit prompts, fostering grounded and intuitive interactions that glean user insights for the chat-bot. Our user studies demonstrate the imminent feasibility and transformative potential of our approach to streamline user interaction in XR with chat-bots, while offering insights for the design of future XR-embodied LLM agents.

With the rapid development of large language models in recent years, there has been an increasing demand for domain-specific Agents that can cater to the unique needs of enterprises and organizations. Unlike general models, which strive for broad coverage, these specialized Agents rely on focused datasets tailored to their intended applications. This research proposes a pipeline that leverages the power of LLMs and the Retrieval-Augmented Generation related framework to construct high-quality instruction datasets for fine-tuning on specific domains using custom document collections. By ingesting domain-specific documents, the pipeline generates relevant and contextually appropriate instructions, thus effectively creating a comprehensive dataset for fine-tuning LLMs on the target domain. This approach overcomes the limitations of traditional dataset creation methods, which often rely on manual curation or web-scraping techniques that may introduce noise and irrelevant data. Notably, our pipeline offers a dynamic solution that can quickly adapt to updates or modifications in the domain-specific document collection, eliminating the need for complete retraining. Additionally, it addresses the challenge of data scarcity by enabling the generation of instruction datasets from a limited set of initial documents, rendering it suitable for unpopular or specialized domains where comprehensive datasets are scarce. As a case study, we apply this approach to the domain of psychiatry, a field requiring specialized knowledge and sensitive handling of patient information. The resulting fine-tuned LLM demonstrates showcases the viability of the proposed approach and underscores its potential for widespread adoption across various industries and domains where tailored, accurate, and contextually relevant language models are indispensable., Comment: 5 pages, SCA 2024: The 7th IEEE International Workshop on Smart Computing & Applications

We present a new analysis of Kepler-21, the brightest (V = 8.5) Kepler system with a known transiting exoplanet, Kepler-21 b. Kepler-21 b is a radius valley planet ($R = 1.6\pm 0.2 R_{\oplus}$) with an Earth-like composition (8.38$\pm$1.62 g/cc), though its mass and radius fall in the regime of possible "water worlds." We utilize new Keck/HIRES and WIYN/NEID radial velocity (RV) data in conjunction with Kepler and TESS photometry to perform a detailed study of activity mitigation between photometry and RVs. We additionally refine the system parameters, and we utilize Gaia astrometry to place constraints on a long-term RV trend. Our activity analysis affirms the quality of Kepler photometry for removing correlated noise from RVs, despite its temporal distance, though we reveal some cases where TESS may be superior. Using refined orbital parameters and updated composition curves, we rule out a ``water world" scenario for Kepler-21 b, and we identify a long period super-Jupiter planetary candidate, Kepler-21 (c).

TOI-6255~b (GJ 4256) is an Earth-sized planet (1.079$\pm0.065$ $R_\oplus$) with an orbital period of only 5.7 hours. With the newly commissioned Keck Planet Finder (KPF) and CARMENES spectrographs, we determined the planet's mass to be 1.44$\pm$0.14 $M_{\oplus}$. The planet is just outside the Roche limit, with $P_{\rm orb}/P_{\rm Roche}$ = 1.13 $\pm0.10$. The strong tidal force likely deforms the planet into a triaxial ellipsoid with a long axis that is $\sim$10\% longer than the short axis. Assuming a reduced stellar tidal quality factor $Q_\star^\prime \approx10^7$, we predict that tidal orbital decay will cause TOI-6255 to reach the Roche limit in roughly 400 Myr. Such tidal disruptions may produce the possible signatures of planet engulfment that have been on stars with anomalously high refractory elemental abundances compared to its conatal binary companion. TOI-6255 b is also a favorable target for searching for star-planet magnetic interactions, which might cause interior melting and hasten orbital decay. TOI-6255 b is a top target (Emission Spectroscopy Metric of about 24) for phase curve observations with the James Webb Space Telescope., Comment: 18 pages, 7 figures, 5 tables, accepted to AAS Journals. The first RV mass measurement from the Keck Planet Finder

The goal of future quantum networks is to enable new internet applications that are impossible to achieve using solely classical communication. Up to now, demonstrations of quantum network applications and functionalities on quantum processors have been performed in ad-hoc software that was specific to the experimental setup, programmed to perform one single task (the application experiment) directly into low-level control devices using expertise in experimental physics. Here, we report on the design and implementation of the first architecture capable of executing quantum network applications on quantum processors in platform-independent high-level software. We demonstrate the architecture's capability to execute applications in high-level software, by implementing it as a quantum network operating system -- QNodeOS -- and executing test programs including a delegated computation from a client to a server on two quantum network nodes based on nitrogen-vacancy (NV) centers in diamond. We show how our architecture allows us to maximize the use of quantum network hardware, by multitasking different applications on a quantum network for the first time. Our architecture can be used to execute programs on any quantum processor platform corresponding to our system model, which we illustrate by demonstrating an additional driver for QNodeOS for a trapped-ion quantum network node based on a single $^{40}\text{Ca}^+$ atom. Our architecture lays the groundwork for computer science research in the domain of quantum network programming, and paves the way for the development of software that can bring quantum network technology to society., Comment: 12 pages, 5 figures, supplementary materials (48 pages, 24 figures, 11 tables)

Removing audio effects from electric guitar recordings makes it easier for post-production and sound editing. An audio distortion recovery model not only improves the clarity of the guitar sounds but also opens up new opportunities for creative adjustments in mixing and mastering. While progress have been made in creating such models, previous efforts have largely focused on synthetic distortions that may be too simplistic to accurately capture the complexities seen in real-world recordings. In this paper, we tackle the task by using a dataset of guitar recordings rendered with commercial-grade audio effect VST plugins. Moreover, we introduce a novel two-stage methodology for audio distortion recovery. The idea is to firstly process the audio signal in the Mel-spectrogram domain in the first stage, and then use a neural vocoder to generate the pristine original guitar sound from the processed Mel-spectrogram in the second stage. We report a set of experiments demonstrating the effectiveness of our approach over existing methods, through both subjective and objective evaluation metrics., Comment: DAFx 2024

Faithful real-time facial animation is essential for avatar-mediated telepresence in Virtual Reality (VR). To emulate authentic communication, avatar animation needs to be efficient and accurate: able to capture both extreme and subtle expressions within a few milliseconds to sustain the rhythm of natural conversations. The oblique and incomplete views of the face, variability in the donning of headsets, and illumination variation due to the environment are some of the unique challenges in generalization to unseen faces. In this paper, we present a method that can animate a photorealistic avatar in realtime from head-mounted cameras (HMCs) on a consumer VR headset. We present a self-supervised learning approach, based on a cross-view reconstruction objective, that enables generalization to unseen users. We present a lightweight expression calibration mechanism that increases accuracy with minimal additional cost to run-time efficiency. We present an improved parameterization for precise ground-truth generation that provides robustness to environmental variation. The resulting system produces accurate facial animation for unseen users wearing VR headsets in realtime. We compare our approach to prior face-encoding methods demonstrating significant improvements in both quantitative metrics and qualitative results., Comment: SIGGRAPH 2024 (ACM Transactions on Graphics (TOG))

Recent advancements in Large Language Models (LLMs) and Prompt Engineering have made chatbot customization more accessible, significantly reducing barriers to tasks that previously required programming skills. However, prompt evaluation, especially at the dataset scale, remains complex due to the need to assess prompts across thousands of test instances within a dataset. Our study, based on a comprehensive literature review and pilot study, summarized five critical challenges in prompt evaluation. In response, we introduce a feature-oriented workflow for systematic prompt evaluation. In the context of text summarization, our workflow advocates evaluation with summary characteristics (feature metrics) such as complexity, formality, or naturalness, instead of using traditional quality metrics like ROUGE. This design choice enables a more user-friendly evaluation of prompts, as it guides users in sorting through the ambiguity inherent in natural language. To support this workflow, we introduce Awesum, a visual analytics system that facilitates identifying optimal prompt refinements for text summarization through interactive visualizations, featuring a novel Prompt Comparator design that employs a BubbleSet-inspired design enhanced by dimensionality reduction techniques. We evaluate the effectiveness and general applicability of the system with practitioners from various domains and found that (1) our design helps overcome the learning curve for non-technical people to conduct a systematic evaluation of summarization prompts, and (2) our feature-oriented workflow has the potential to generalize to other NLG and image-generation tasks. For future works, we advocate moving towards feature-oriented evaluation of LLM prompts and discuss unsolved challenges in terms of human-agent interaction.

A flexible multimode fiber is an exceptionally efficient tool for in vivo deep tissue imaging. Recent advances in compressive multimode fiber sensing allow for imaging with sub-diffraction spatial resolution and sub-Nyquist speed. At present, the technology is limited to imaging in a two-dimensional (2D) plane near the fiber distal facet while in real applications it is very important to visualize three-dimensional (3D) structures. Here we propose a new approach for fast sub-diffraction 3D imaging through a multimode fiber by using a single 2D scan, speckle illumination, and bucket detection. We experimentally demonstrate precise image plane location as well as 3D imaging of samples with various scattering coefficients. The full width at half maximum (FWHM) of the point spread function along the axial direction is 3 times smaller than the diffraction limit. Our study grants depth-resolving capacity to ultra-thin super-resolution fiber endoscopes for life science and medical applications., Comment: 8 pages,4 figures

Single-stranded RNA viruses co-assemble their capsid with the genome and variations in capsid structures can have significant functional relevance. In particular, viruses need to respond to a dehydrating environment to prevent genomic degradation and remain active upon rehydration. Theoretical work has predicted low-energy buckling transitions in icosahedral capsids which could protect the virus from further dehydration. However, there has been no direct experimental evidence, nor molecular mechanism, for such behaviour. Here we observe this transition using X-ray single particle imaging of MS2 bacteriophages after aerosolization. Using a combination of machine learning tools, we classify hundreds of thousands of single particle diffraction patterns to learn the structural landscape of the capsid morphology as a function of time spent in the aerosol phase. We found a previously unreported compact conformation as well as intermediate structures which suggest an incoherent buckling transition which does not preserve icosahedral symmetry. Finally, we propose a mechanism of this buckling, where a single 19-residue loop is destabilised, leading to the large observed morphology change. Our results provide experimental evidence for a mechanism by which viral capsids protect themselves from dehydration. In the process, these findings also demonstrate the power of single particle X-ray imaging and machine learning methods in studying biomolecular structural dynamics., Comment: 10 pages, 4 figures plus 9 pages supplementary information

Replicating analog device circuits through neural audio effect modeling has garnered increasing interest in recent years. Existing work has predominantly focused on a one-to-one emulation strategy, modeling specific devices individually. In this paper, we tackle the less-explored scenario of one-to-many emulation, utilizing conditioning mechanisms to emulate multiple guitar amplifiers through a single neural model. For condition representation, we use contrastive learning to build a tone embedding encoder that extracts style-related features of various amplifiers, leveraging a dataset of comprehensive amplifier settings. Targeting zero-shot application scenarios, we also examine various strategies for tone embedding representation, evaluating referenced tone embedding against two retrieval-based embedding methods for amplifiers unseen in the training time. Our findings showcase the efficacy and potential of the proposed methods in achieving versatile one-to-many amplifier modeling, contributing a foundational step towards zero-shot audio modeling applications., Comment: ISMIR 2024

Converting different modalities into general text, serving as input prompts for large language models (LLMs), is a common method to align multimodal models when there is limited pairwise data. This text-centric approach leverages the unique properties of text as a modality space, transforming diverse inputs into a unified textual representation. This enables downstream models to effectively interpret various modal inputs. This study assesses the quality and robustness of multimodal representations in the presence of missing entries, noise, or absent modalities, revealing that current text-centric alignment methods compromise downstream robustness. To address this issue, we propose a new text-centric approach that achieves superior robustness compared to previous methods across various modalities in different settings. Our findings highlight the potential of this approach to enhance the robustness and adaptability of multimodal representations, offering a promising solution for dynamic and real-world applications.

We present a comprehensive analysis of young stellar object (YSO) variability within the M17 Southwest Extension (M17 SWex), using 3.5 years of monitoring data from the JCMT Transient Survey at sub-millimeter (sub-mm) and 9 years from the NEOWISE mission at mid-infrared (mid-IR). Our study encompasses observations of 147 bright sub-mm peaks identified within our deep JCMT co-added map as well as 156 YSOs in NEOWISE W1 and 179 in W2 that were previously identified in Spitzer surveys. We find three robust sub-mm variables: two are candidate YSOs and one is a likely extragalactic source. At mid-IR wavelengths, our analysis reveals secular and stochastic variability in 47 YSOs, with the highest fraction of secular variability occurring at the earliest evolutionary stage. This is similar to what has previously been observed for low-mass YSO variability within the Gould Belt. However, we observe less overall variability in M17SWex at both the sub-mm and mid-IR. We suspect that this lower fraction is due to the greater distance to M17 SWex. Our findings showcase the utility of multi-wavelength observations to better capture the complex variability phenomena inherent to star formation processes and demonstrate the importance of years-long monitoring of a diverse selection of star-forming environments., Comment: Accepted for Publication in The Astronomical Journal

Expressive music synthesis (EMS) for violin performance is a challenging task due to the disagreement among music performers in the interpretation of expressive musical terms (EMTs), scarcity of labeled recordings, and limited generalization ability of the synthesis model. These challenges create trade-offs between model effectiveness, diversity of generated results, and controllability of the synthesis system, making it essential to conduct a comparative study on EMS model design. This paper explores two violin EMS approaches. The end-to-end approach is a modification of a state-of-the-art text-to-speech generator. The parameter-controlled approach is based on a simple parameter sampling process that can render note lengths and other parameters compatible with MIDI-DDSP. We study these two approaches (in total, three model variants) through objective and subjective experiments and discuss several key issues of EMS based on the results., Comment: 15 pages, 2 figures, 3 tables

Visual programs are executable code generated by large language models to address visual reasoning problems. They decompose complex questions into multiple reasoning steps and invoke specialized models for each step to solve the problems. However, these programs are prone to logic errors, with our preliminary evaluation showing that 58% of the total errors are caused by program logic errors. Debugging complex visual programs remains a major bottleneck for visual reasoning. To address this, we introduce VDebugger, a novel critic-refiner framework trained to localize and debug visual programs by tracking execution step by step. VDebugger identifies and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy. The training data is generated through an automated pipeline that injects errors into correct visual programs using a novel mask-best decoding technique. Evaluations on six datasets demonstrate VDebugger's effectiveness, showing performance improvements of up to 3.2% in downstream task accuracy. Further studies show VDebugger's ability to generalize to unseen tasks, bringing a notable improvement of 2.3% on the unseen COVR task. Code, data and models are made publicly available at https://github.com/shirley-wu/vdebugger/, Comment: update reference

The integration of pre-trained text-based large language models (LLM) with speech input has enabled instruction-following capabilities for diverse speech tasks. This integration requires the use of a speech encoder, a speech adapter, and an LLM, trained on diverse tasks. We propose the use of discrete speech units (DSU), rather than continuous-valued speech encoder outputs, that are converted to the LLM token embedding space using the speech adapter. We generate DSU using a self-supervised speech encoder followed by k-means clustering. The proposed model shows robust performance on speech inputs from seen/unseen domains and instruction-following capability in spoken question answering. We also explore various types of DSU extracted from different layers of the self-supervised speech encoder, as well as Mel frequency Cepstral Coefficients (MFCC). Our findings suggest that the ASR task and datasets are not crucial in instruction-tuning for spoken question answering tasks.

We develop a lattice model which exhibits topological transitions from $Z_2$ topological insulators to mirror symmetry-protected topological crystalline insulators by introducing additional spin-orbit coupling terms. The topological phase is characterized by the mirror winding number, defined within the mirror symmetry invariant subspace, which ensures the protection of gapless edge states and zero-energy corner states under specific boundary conditions. Additionally, we propose a feasible scheme using ultracold atoms confined in a stacked hexagonal optical lattice with Raman fields to realize the two-dimensional topological crystalline insulators. Detection of the mirror winding number in these systems can be achieved by implementing a simple quench sequence and observing the evolution of the time-of-flight patterns., Comment: 11 pages, 5 figures

Recent progresses in Josephson-junction-based superconducting circuits have propelled quantum information processing forward. However, the lack of a metastable state in most superconducting artificial atoms hinders the development of photonic quantum memory in this platform. Here, we use a single superconducting qubit-resonator system to realize a desired $\Lambda$-type artificial atom, and to demonstrate slow light with a group velocity of 3.6 km/s and the microwave storage with a memory time extending to several hundred nanoseconds via electromagnetically induced transparency. Our results highlight the potential of achieving microwave quantum memory, promising substantial advancements in quantum information processing within superconducting circuits., Comment: 10 pages, 6 figures

Soft grippers, with their inherent compliance and adaptability, show advantages for delicate and versatile manipulation tasks in robotics. This paper presents a novel approach to underactuated control of multiple soft actuators, explicitly focusing on the coordination of soft fingers within a soft gripper. Utilizing a single syringe pump as the actuation mechanism, we address the challenge of coordinating multiple degrees of freedom of a compliant system. The theoretical framework applies concepts from stable inversion theory, adapting them to the unique dynamics of the underactuated soft gripper. Through meticulous mechatronic system design and controller synthesis, we demonstrate the efficacy and applicability of our approach in achieving precise and coordinated manipulation tasks in simulation and experimentation. Our findings not only contribute to the advancement of soft robot control but also offer practical insights into the design and control of underactuated systems for real-world applications., Comment: 11 pages, 7 figures

Interlayer rotation in van der Waals (vdW) materials offers great potential for manipulating phonon dynamics and heat flow in advanced electronics with ever higher compactness and power density. However, despite extensive theoretical efforts in recent years, experimental measurements remain scarce especially due to the critical challenges of preparing single-crystalline twisted interfaces and probing interfacial thermal transport with sufficient resolution. Here, we exploited the intrinsic twisted interfaces in highly oriented pyrolytic graphite (HOPG). By developing novel experimental schemes based on microfabricated mesas, we managed to achieve simultaneous mechanical characterizations and thermal measurements. In particular, we pushed the HOPG mesas with a microprobe to identify and rotate single-crystalline intrinsic interfaces owing to their slippery nature as is well known in structural superlubricity. Remarkably, we observed over 30-fold suppression of thermal conductance for the slippery interfaces by using epitaxial graphite as a control. Nonetheless, the interfacial conductance remains around 600 $\mathrm{MWm^{-2}K^{-1}}$ which surpasses the highest values for artificially stacked vdW structures by more than five times. Further, atomic simulations revealed the predominant role of the transverse acoustic phonons. Together, our findings highlight a general physical picture that directly correlates interfacial thermal transport with sliding resistance, and lay the foundation for twist-enabled thermal management which are particularly beneficial to twistronics and slidetronics.

There exist three approaches for multilingual and crosslingual automatic speech recognition (MCL-ASR) - supervised pre-training with phonetic or graphemic transcription, and self-supervised pre-training. We find that pre-training with phonetic supervision has been underappreciated so far for MCL-ASR, while conceptually it is more advantageous for information sharing between different languages. This paper explores the approach of pre-training with weakly phonetic supervision towards data-efficient MCL-ASR, which is called Whistle. We relax the requirement of gold-standard human-validated phonetic transcripts, and obtain International Phonetic Alphabet (IPA) based transcription by leveraging the LanguageNet grapheme-to-phoneme (G2P) models. We construct a common experimental setup based on the CommonVoice dataset, called CV-Lang10, with 10 seen languages and 2 unseen languages. A set of experiments are conducted on CV-Lang10 to compare, as fair as possible, the three approaches under the common setup for MCL-ASR. Experiments demonstrate the advantages of phoneme-based models (Whistle) for MCL-ASR, in terms of speech recognition for seen languages, crosslingual performance for unseen languages with different amounts of few-shot data, overcoming catastrophic forgetting, and training efficiency.It is found that when training data is more limited, phoneme supervision can achieve better results compared to subword supervision and self-supervision, thereby providing higher data-efficiency. To support reproducibility and promote future research along this direction, we will release the code, models and data for the whole pipeline of Whistle at https://github.com/thu-spmi/CAT upon publication.

This paper provides an overview of the Fake-EmoReact 2021 Challenge, held at the 9th SocialNLP Workshop, in conjunction with NAACL 2021. The challenge requires predicting the authenticity of tweets using reply context and augmented GIF categories from EmotionGIF dataset. We offer the Fake-EmoReact dataset with more than 453k as the experimental materials, where every tweet is labeled with authenticity. Twenty-four teams registered to participate in this challenge, and 5 submitted their results successfully in the evaluation phase. The best team achieves 93.9 on Fake-EmoReact 2021 dataset using F1 score. In addition, we show the definition of share task, data collection, and the teams' performance that joined this challenge and their approaches.

Autoregressive next-token prediction is a standard pretraining method for large-scale language models, but its application to vision tasks is hindered by the non-sequential nature of image data, leading to cumulative errors. Most vision models employ masked autoencoder (MAE) based pretraining, which faces scalability issues. To address these challenges, we introduce \textbf{TokenUnify}, a novel pretraining method that integrates random token prediction, next-token prediction, and next-all token prediction. We provide theoretical evidence demonstrating that TokenUnify mitigates cumulative errors in visual autoregression. Cooperated with TokenUnify, we have assembled a large-scale electron microscopy (EM) image dataset with ultra-high resolution, ideal for creating spatially correlated long sequences. This dataset includes over 120 million annotated voxels, making it the largest neuron segmentation dataset to date and providing a unified benchmark for experimental validation. Leveraging the Mamba network inherently suited for long-sequence modeling on this dataset, TokenUnify not only reduces the computational complexity but also leads to a significant 45\% improvement in segmentation performance on downstream EM neuron segmentation tasks compared to existing methods. Furthermore, TokenUnify demonstrates superior scalability over MAE and traditional autoregressive methods, effectively bridging the gap between pretraining strategies for language and vision models. Code is available at \url{https://github.com/ydchen0806/TokenUnify}.

In the past years, the amount of research on active matter has grown extremely rapidly, a fact that is reflected in particular by the existence of more than 600 review articles on this topic. Moreover, the field has become very diverse, ranging from theoretical studies of the statistical mechanics of active particles to applied work on medical applications of microrobots and from biological systems to artificial swimmers. This makes it very difficult to get an overview over the field as a whole. Here, we provide such an overview in the form of a metareview article that surveys the existing review articles on active matter. Thereby, this article provides an introduction to the various subdisciplines of active matter science and constitutes a useful starting point for finding literature about a specific topic., Comment: 28 pages, 2 figures, 646 references

In this paper, we introduce a new quantum circuit synthesis (QCS) framework, Qsyn, for developers to research, develop, test, experiment, and then contribute their QCS algorithms and tools to the framework. Our framework is more developer-friendly than other modern QCS frameworks in three aspects: (1) We design a rich command-line interface so that developers can easily design various testing scenarios and flexibly conduct experiments on their algorithms. (2) We offer detailed access to many data representations on different abstract levels of quantum circuits so that developers can optimize their algorithms to the extreme. (3) We define a rigid developing flow and environment so that developers can ensure their development qualities with the best modern software engineering practices. We illustrate the friendliness of our framework with a showcase of developing a T-Count Optimization algorithm and demonstrate our performance superiority with fair comparisons to other modern QCS frameworks., Comment: Table II has errors in parts of its data and needs to be revised

Existing subject-driven text-to-image generation models suffer from tedious fine-tuning steps and struggle to maintain both text-image alignment and subject fidelity. For generating compositional subjects, it often encounters problems such as object missing and attribute mixing, where some subjects in the input prompt are not generated or their attributes are incorrectly combined. To address these limitations, we propose a subject-driven generation framework and introduce training-free guidance to intervene in the generative process during inference time. This approach strengthens the attention map, allowing for precise attribute binding and feature injection for each subject. Notably, our method exhibits exceptional zero-shot generation ability, especially in the challenging task of compositional generation. Furthermore, we propose a novel metric GroundingScore to evaluate subject alignment thoroughly. The obtained quantitative results serve as compelling evidence showcasing the effectiveness of our proposed method. The code will be released soon., Comment: 26 pages, 13 figures

We report on the development of an implementable physics-data hybrid dynamic model for an articulated manipulator to plan and operate in various scenarios. Meanwhile, the physics-based and data-driven dynamic models are studied in this research to select the best model for planning. The physics-based model is constructed using the Lagrangian method, and the loss terms include inertia loss, viscous loss, and friction loss. As for the data-driven model, three methods are explored, including DNN, LSTM, and XGBoost. Our modeling results demonstrate that, after comprehensive hyperparameter optimization, the XGBoost architecture outperforms DNN and LSTM in accurately representing manipulator dynamics. The hybrid model with physics-based and data-driven terms has the best performance among all models based on the RMSE criteria, and it only needs about 24k of training data. In addition, we developed a virtual force sensor of a manipulator using the observed external torque derived from the dynamic model and designed a motion planner through the physics-data hybrid dynamic model. The external torque contributes to forces and torque on the end effector, facilitating interaction with the surroundings, while the internal torque governs manipulator motion dynamics and compensates for internal losses. By estimating external torque via the difference between measured joint torque and internal losses, we implement a sensorless control strategy which is demonstrated through a peg-in-hole task. Lastly, a learning-based motion planner based on the hybrid dynamic model assists in planning time-efficient trajectories for the manipulator. This comprehensive approach underscores the efficacy of integrating physics-based and data-driven models for advanced manipulator control and planning in industrial environments., Comment: 26 pages, 16 figures

We work the lattice fermions and non-Hermitian formulation in the 2D GNY model and demonstrate the numerical implementation for two flavors by the Hybrid Monte Carlo. Our approach has a notable advantage in dealing with chiral symmetry on a lattice by avoiding the Nielsen-Ninomiya theorem, due to the non-symmetrized finite-difference operator. We restore the hypercubic symmetry by averaging over all possible orientations with the proper continuum limit. Our study is the first simulation for the interacting fermion formulated in a non-hermitian way. We compare the numerical solution with the one-loop resummation. The resummation results matches with the numerical solution in $\langle\phi\rangle$, $\langle\phi^2\rangle$, $\langle\mathrm{Tr}(\bar{\psi}_1\psi_1+\bar{\psi}_2\psi_2)/2\rangle$, and $\langle\mathrm{Tr}(\bar{\psi}_1\psi_1+\bar{\psi}_2\psi_2)\phi/2\rangle$. We also used the one-loop resummation to provide the RG flow and asymptotic safety in the 2D GNY model., Comment: 26 pages, 10 figures, minor changes

Nonreciprocal forces violating Newton's third law are common in a plethora of nonequilibrium situations ranging from predator-prey systems to the swarming of birds and effective colloidal interactions under flow. While many recent studies have focused on two species with nonreciprocal coupling, much less is examined for the basic single-component system breaking the actio and reactio equality of force within the same species. Here, we systematically derive the fundamental field theory of single-species nonreciprocal interactions from microscopic dynamics, leading to a generic framework termed Active Model N (N denoting nonreciprocity). We explore the rich dynamics of pattern formation in this intrinsic nonreciprocal system and the emergence of self-traveling states with persistent variation and flowing of active branched patterns. One particular new characteristic pattern is an interwoven self-knitting "yarn" structure with a unique feature of simultaneous development of micro- and bulk phase separations. The growth dynamics of a "ball-of-wool" active droplet towards these self-knitted yarn or branched states exhibits a crossover between different scaling behaviors. The mechanism underlying this distinct class of active phase separation is attributed to the interplay between force nonreciprocity and competition. Our predictions can be applied to various biological and artificial active matter systems controlled by single-species nonreciprocity., Comment: 25 pages, 7 figures

Strong laser pulses can control superconductivity, inducing non-equilibrium transient pairing by leveraging strong-light matter interaction. Here we demonstrate theoretically that equilibrium ground-state phonon-mediated superconductive pairing can be affected through the vacuum fluctuating electromagnetic field in a cavity. Using the recently developed ab initio quantum electrodynamical density-functional theory approximation, we specifically investigate the phonon-mediated superconductive behavior of MgB$_2$ under different cavity setups and find that in the strong light-matter coupling regime its superconducting transition temperature can be, in principles, enhanced by $\approx 73\%$ ($\approx 40\%$) in an in-plane (out-of-plane) polarized cavity. However, in a realistic cavity, we expect the T$_{\rm{c}}$ of MgB$_2$ can increase, at most, by $5$ K via photon vacuum fluctuations. The results highlight that strong light-matter coupling in extended systems can profoundly alter material properties in a non-perturbative way by modifying their electronic structure and phononic dispersion at the same time. Our findings indicate a pathway to the experimental realization of light-controlled superconductivity in solid-state materials at equilibrium via cavity-material engineering., Comment: 30 pages, 3 figures

Image dehazing poses significant challenges in environmental perception. Recent research mainly focus on deep learning-based methods with single modality, while they may result in severe information loss especially in dense-haze scenarios. The infrared image exhibits robustness to the haze, however, existing methods have primarily treated the infrared modality as auxiliary information, failing to fully explore its rich information in dehazing. To address this challenge, the key insight of this study is to design a visible-infrared fusion network for image dehazing. In particular, we propose a multi-scale Deep Structure Feature Extraction (DSFE) module, which incorporates the Channel-Pixel Attention Block (CPAB) to restore more spatial and marginal information within the deep structural features. Additionally, we introduce an inconsistency weighted fusion strategy to merge the two modalities by leveraging the more reliable information. To validate this, we construct a visible-infrared multimodal dataset called AirSim-VID based on the AirSim simulation platform. Extensive experiments performed on challenging real and simulated image datasets demonstrate that VIFNet can outperform many state-of-the-art competing methods. The code and dataset are available at https://github.com/mengyu212/VIFNet_dehazing.

A key challenge towards future quantum internet technology is connecting quantum processors at metropolitan scale. Here, we report on heralded entanglement between two independently operated quantum network nodes separated by 10km. The two nodes hosting diamond spin qubits are linked with a midpoint station via 25km of deployed optical fiber. We minimize the effects of fiber photon loss by quantum frequency conversion of the qubit-native photons to the telecom L-band and by embedding the link in an extensible phase-stabilized architecture enabling the use of the loss-resilient single-photon entangling protocol. By capitalizing on the full heralding capabilities of the network link in combination with real-time feedback logic on the long-lived qubits, we demonstrate the delivery of a predefined entangled state on the nodes irrespective of the heralding detection pattern. Addressing key scaling challenges and being compatible with different qubit systems, our architecture establishes a generic platform for exploring metropolitan-scale quantum networks., Comment: 10 pages, 4 figures, supplementary materials

Detecting problematic research articles timely is a vital task. This study explores whether Twitter mentions of retracted articles can signal potential problems with the articles prior to retraction, thereby playing a role in predicting future retraction of problematic articles. A dataset comprising 3,505 retracted articles and their associated Twitter mentions is analyzed, alongside 3,505 non-retracted articles with similar characteristics obtained using the Coarsened Exact Matching method. The effectiveness of Twitter mentions in predicting article retraction is evaluated by four prediction methods, including manual labelling, keyword identification, machine learning models, and ChatGPT. Manual labelling results indicate that there are indeed retracted articles with their Twitter mentions containing recognizable evidence signaling problems before retraction, although they represent only a limited share of all retracted articles with Twitter mention data (approximately 16%). Using the manual labelling results as the baseline, ChatGPT demonstrates superior performance compared to other methods, implying its potential in assisting human judgment for predicting article retraction. This study uncovers both the potential and limitation of social media events as an early warning system for article retraction, shedding light on a potential application of generative artificial intelligence in promoting research integrity.

We introduce RakutenAI-7B, a suite of Japanese-oriented large language models that achieve the best performance on the Japanese LM Harness benchmarks among the open 7B models. Along with the foundation model, we release instruction- and chat-tuned models, RakutenAI-7B-instruct and RakutenAI-7B-chat respectively, under the Apache 2.0 license.

The Liouvillian skin effect and the non-Hermitian skin effect have both been used to explain the localization of eigenmodes near system boundaries, though the former is arguably more accurate in some regimes due to its incorporation of quantum jumps. However, these frameworks predominantly focus on weak Markovian interactions, neglecting the potentially crucial role of memory effects. To address this, we investigate, utilizing the powerful hierarchical equations of motion method, how a non-Markovian environment can modify the Liouvillian skin effect. We demonstrate that a non-Markovian environment can induce not only a ``thick skin effect", where the skin mode broadens and shifts into the bulk, but also skin-mode coherence, leading to the coherence-delocalization and oscillatory relaxation with a characteristic linear scaling with system size. Remarkably, both the skin-mode and steady-state coherence exhibit resistance to decoherence from additional environmental noise. These findings highlight the profound impact of system-bath correlations on relaxation and localization, revealing unique phenomena beyond conventional Markovian approximations., Comment: 15 pages, 9 figures

This research's primary motivation of this study is to address the high hardware and computational demands typically associated with LLMs.Therefore,our goal is to find a balance between model lightness and performance,striving to maximize performance while using a comparatively lightweight model. Hyacinth6B was developed with this objective in mind,aiming to fully leverage the core capabilities of LLMs without incurring substantial resource costs, effectively pushing the boundaries of smaller model's performance. The training approach involves parameter efficient finetuning using the LoRA method., Comment: 14pages