Search

The fine-tuning of pre-trained language models (PLMs) has been shown to be effective across various domains. By using domain-specific supervised data, the general-purpose representation derived from PLMs can be transformed into a domain-specific representation. However, these methods often fail to generalize to out-of-domain (OOD) data due to their reliance on non-causal representations, often described as spurious features. Existing methods either make use of adjustments with strong assumptions about lack of hidden common causes, or mitigate the effect of spurious features using multi-domain data. In this work, we investigate how fine-tuned pre-trained language models aid generalizability from single-domain scenarios under mild assumptions, targeting more general and practical real-world scenarios. We show that a robust representation can be derived through a so-called causal front-door adjustment, based on a decomposition assumption, using fine-tuned representations as a source of data augmentation. Comprehensive experiments in both synthetic and real-world settings demonstrate the superior generalizability of the proposed method compared to existing approaches. Our work thus sheds light on the domain generalization problem by introducing links between fine-tuning and causal mechanisms into representation learning.

Backdoor attacks pose a significant threat to Deep Neural Networks (DNNs) as they allow attackers to manipulate model predictions with backdoor triggers. To address these security vulnerabilities, various backdoor purification methods have been proposed to purify compromised models. Typically, these purified models exhibit low Attack Success Rates (ASR), rendering them resistant to backdoored inputs. However, Does achieving a low ASR through current safety purification methods truly eliminate learned backdoor features from the pretraining phase? In this paper, we provide an affirmative answer to this question by thoroughly investigating the Post-Purification Robustness of current backdoor purification methods. We find that current safety purification methods are vulnerable to the rapid re-learning of backdoor behavior, even when further fine-tuning of purified models is performed using a very small number of poisoned samples. Based on this, we further propose the practical Query-based Reactivation Attack (QRA) which could effectively reactivate the backdoor by merely querying purified models. We find the failure to achieve satisfactory post-purification robustness stems from the insufficient deviation of purified models from the backdoored model along the backdoor-connected path. To improve the post-purification robustness, we propose a straightforward tuning defense, Path-Aware Minimization (PAM), which promotes deviation along backdoor-connected paths with extra model updates. Extensive experiments demonstrate that PAM significantly improves post-purification robustness while maintaining a good clean accuracy and low ASR. Our work provides a new perspective on understanding the effectiveness of backdoor safety tuning and highlights the importance of faithfully assessing the model's safety., Comment: NeurIPS 2024 Spotlight paper. The first two authors contributed equally

In multi-agent learning, the predominant approach focuses on generalization, often neglecting the optimization of individual agents. This emphasis on generalization limits the ability of agents to utilize their unique strengths, resulting in inefficiencies. This paper introduces Comparative Advantage Maximization (CAM), a method designed to enhance individual agent specialization in multiagent systems. CAM employs a two-phase process, combining centralized population training with individual specialization through comparative advantage maximization. CAM achieved a 13.2% improvement in individual agent performance and a 14.9% increase in behavioral diversity compared to state-of-the-art systems. The success of CAM highlights the importance of individual agent specialization, suggesting new directions for multi-agent system development., Comment: 19 pages

We present a multi-wavelength analysis of MCG+11-11-032, a nearby AGN with the unique classification of both a binary and a dual AGN candidate. With new Chandra observations we aim to resolve any dual AGN system via imaging data, and search for signs of a binary AGN via analysis of the X-ray spectrum. Analyzing the Chandra spectrum, we find no evidence of previously suggested double-peaked Fe K$\alpha$ lines; the spectrum is instead best fit by an absorbed powerlaw with a single Fe K$\alpha$ line, as well as an additional line centered at $\approx$7.5 keV. The Chandra observation reveals faint, soft, and extended X-ray emission, possibly linked to low-level nuclear outflows. Further analysis shows evidence for a compact, hard source -- MCG+11-11-032 X2 -- located 3.27'' from the primary AGN. Modeling MCG+11-11-032 X2 as a compact source, we find that it is relatively luminous ($L_{\text{2$-$10 keV}} = 1.52_{-0.48}^{+0.96}\times 10^{41}$ erg s$^{-1}$), and the location is coincident with an compact and off-nuclear source resolved in Hubble Space Telescope infrared (F105W) and ultraviolet (F621M, F547M) bands. Pairing our X-ray results with a 144 MHz radio detection at the host galaxy location, we observe X-ray and radio properties similar to those of ESO 243-49 HLX-1, suggesting that MCG+11-11-032 X2 may be a hyper-luminous X-ray source. This detection with Chandra highlights the importance of a high-resolution X-ray imager, and how previous binary AGN candidates detected with large-aperture instruments benefit from high-resolution follow-up. Future spatially resolved optical spectra, and deeper X-ray observations, can better constrain the origin of MCG+11-11-032 X2., Comment: 16 pages, 7 figures, submitted to ApJ

Pseudorandom values are often generated as 64-bit binary words. These random words need to be converted into ranged values without statistical bias. We present an efficient algorithm to generate multiple independent uniformly-random bounded integers from a single uniformly-random binary word, without any bias. In the common case, our method uses one multiplication and no division operations per value produced. In practice, our algorithm can more than double the speed of unbiased random shuffling for small to moderately large arrays., Comment: software: https://github.com/lemire/batched_random

F-type star--planet systems represent an intriguing case for habitability studies. Although F-type stars spend considerably less time on the main-sequence than G, K, and M-type stars, they still offer a unique set of features, allowing for the principal possibility of exolife. Examples of the latter include the increased widths of stellar habitable zones as well as the presence of enhanced UV flux, which in moderation may have added to the origin of life in the Universe. In this study, we pursue a detailed statistical analysis of the currently known planet-hosting F-type stars by making use of the NASA Exoplanet Archive. After disregarding systems with little or no information on the planet(s), we identify 206 systems of interest. We also evaluate whether the stars are on the main-sequence based on various criteria. In one approach, we use the stellar evolution code MESA. Depending on the adopted criterion, about 60 to 80 stars have been identified as main-sequence stars. In 18 systems, the planet spends at least part of its orbit within the stellar habitable zone. In one case, i.e., HD 111998, commonly known as 38 Vir, the planet is situated in the habitable zone at all times. Our work may serve as a basis for future studies, including studies on the existence of Earth-mass planets in F-type systems, as well as investigations of possibly habitable exomoons hosted by exo-Jupiters as the lowest-mass habitable zone planet currently identified has a mass estimate of 143 Earth masses., Comment: Accepted by Astrophysical Journal Supplement Series (ApJS); 26 pages, including 10 figures and 4 tables

Traditional Chinese medicine (TCM) relies on specific combinations of herbs in prescriptions to treat symptoms and signs, a practice that spans thousands of years. Predicting TCM prescriptions presents a fascinating technical challenge with practical implications. However, this task faces limitations due to the scarcity of high-quality clinical datasets and the intricate relationship between symptoms and herbs. To address these issues, we introduce DigestDS, a new dataset containing practical medical records from experienced experts in digestive system diseases. We also propose a method, TCM-FTP (TCM Fine-Tuning Pre-trained), to leverage pre-trained large language models (LLMs) through supervised fine-tuning on DigestDS. Additionally, we enhance computational efficiency using a low-rank adaptation technique. TCM-FTP also incorporates data augmentation by permuting herbs within prescriptions, capitalizing on their order-agnostic properties. Impressively, TCM-FTP achieves an F1-score of 0.8031, surpassing previous methods significantly. Furthermore, it demonstrates remarkable accuracy in dosage prediction, achieving a normalized mean square error of 0.0604. In contrast, LLMs without fine-tuning perform poorly. Although LLMs have shown capabilities on a wide range of tasks, this work illustrates the importance of fine-tuning for TCM prescription prediction, and we have proposed an effective way to do that.

To facilitate new studies of galaxy merger driven fueling of active galactic nuclei (AGNs), we present a catalog of 387 AGNs that we have identified in the final population of over $10,000$ $z<0.15$ galaxies observed by the SDSS-IV integral field spectroscopy survey Mapping Nearby Galaxies at Apache Point Observatory (MaNGA). We selected the AGNs via mid-infrared WISE colors, Swift/BAT ultra hard X-ray detections, NVSS and FIRST radio observations, and broad emission lines in SDSS spectra. By combining the MaNGA AGN catalog with a new SDSS catalog of galaxy mergers that were identified based on a suite of hydrodynamical simulations of merging galaxies, we study the link between galaxy mergers and nuclear activity for AGNs above a limiting bolometric luminosity of $10^{44.4}$ erg s$^{-1}$. We find an excess of AGNs in mergers, relative to non-mergers, for galaxies with stellar mass $\sim10^{11}$ $M_{\odot}$, where the AGN excess is somewhat stronger in major mergers than in minor mergers. Further, when we combine minor and major mergers and sort by merger stage, we find that the highest AGN excess occurs in post-coalescence mergers in the highest mass galaxies. However, we find no evidence of a correlation between galaxy mergers and AGN luminosity or accretion rate. In summary, while galaxy mergers overall do appear to trigger or enhance AGN activity more than non-mergers, they do not seem to induce higher levels of accretion or higher luminosities. We provide the MaNGA AGN Catalog and the MaNGA Galaxy Merger Catalog for the community here., Comment: The MaNGA AGN Catalog (https://content.cld.iop.org/journals/0004-637X/963/1/53/revision1/apjad1a15t1_mrt.txt) and MaNGA Galaxy Merger Catalog (https://content.cld.iop.org/journals/0004-637X/963/1/53/revision1/apjad1a15t2_mrt.txt) are available for download from the ApJ article. The MaNGA AGN Catalog is also available as an SDSS value added catalog (https://data.sdss.org/sas/dr17/env/MANGA_AGN/)

Ultra-hot Jupiters are likely doomed by tidal forces to undergo orbital decay and eventual disruption by their stars, but the timescale over which this process unfolds is unknown. We present results from a long-term project to monitor ultra-hot Jupiters transits. We recovered WASP-12 b's orbital decay rate of dP/dt = -29.8 +/- 1.6 ms yr-1, in agreement with prior work. Five other systems initially had promising non-linear transit ephemerides. However, a closer examination of two -- WASP-19 b and CoRoT-2 b, both with prior tentative detections -- revealed several independent errors with the literature timing data; after correction neither planet shows signs of orbital decay. Meanwhile, a potential decreasing period for TrES-1 b, dP/dt = -16 +/- 5 ms yr-1, corresponds to a tidal quality factor Q*' = 160 and likely does not result from orbital decay, if driven by dissipation within the host star. Nominal period increases in two systems, WASP-121 b and WASP-46 b, rest on a small handful of points. Only 1/43 planets (WASP-12 b) in our sample is experiencing detectable orbital decay. For nearly half (20/42) we can rule out dP/dt as high as observed for WASP-12 b. Thus while many ultra-hot Jupiters could still be experiencing rapid decay that we cannot yet detect, a sizeable sub-population of UHJs are decaying at least an order of magnitude more slowly than WASP-12 b. Our reanalysis of Kepler-1658 b with no new data finds that it remains a promising orbital decay candidate. Finally, we recommend that the scientific community take steps to avoid spurious detections through better management of the multi-decade-spanning datasets needed to search for and study planetary orbital decay., Comment: Accepted to PSJ; revised discussion of WASP-19 b literature data

We investigate the dynamical tide in a gravitational wave (GW)-driven coalescing binary involving a neutron star (NS). The NS is assumed to spin rapidly, with its spin axis anti-aligned with the orbit. Such an NS may exist if the binary forms dynamically in a dense environment, and it can lead to a strong tide because the f-mode can be resonantly excited during the inspiral. We present a new analytical solution for the f-mode resonance by decomposing the tide into a resummed equilibrium component and a dynamical component that is excited only around resonance. This solution simplifies numerical implementations by avoiding the subtraction of two diverging terms. It also extends the solution's validity to frequencies beyond mode resonance. When the dynamical tide back reacts on the orbit, the commonly adopted effective Love number is insufficient because it does not capture the tidal torque on the orbit that dominates the back reaction during mode resonance. An additional dressing factor originating from the imaginary part of the Love number is introduced to model the torque. The dissipative interaction between the NS and the orbital mass multipoles is computed including the dynamical tide. Orbital phase shifts caused by the $l=3$ and $l=2$ f-modes can reach 0.5 and 10 radians at their respective resonances if the NS has a spin rate of 850 Hz. Because of the large impact of the dynamical tide, a linearized analytical description becomes insufficient. After mode excitation, the orbit cannot remain quasi-circular, and the eccentricity excited by the dynamical tide can approach $e\simeq 0.1$, leading to non-monotonic frequency evolution which breaks the stationary phase approximation commonly adopted by frequency-domain waveform constructions. The GW radiation from the excited f-mode alone can be detected with a signal-to-noise ratio exceeding unity with the next-generation detectors., Comment: 26 pages, 8 figures, accepted by PRD

Test-time domain adaptation effectively adjusts the source domain model to accommodate unseen domain shifts in a target domain during inference. However, the model performance can be significantly impaired by continuous distribution changes in the target domain and non-independent and identically distributed (non-i.i.d.) test samples often encountered in practical scenarios. While existing memory bank methodologies use memory to store samples and mitigate non-i.i.d. effects, they do not inherently prevent potential model degradation. To address this issue, we propose a resilient practical test-time adaptation (ResiTTA) method focused on parameter resilience and data quality. Specifically, we develop a resilient batch normalization with estimation on normalization statistics and soft alignments to mitigate overfitting and model degradation. We use an entropy-driven memory bank that accounts for timeliness, the persistence of over-confident samples, and sample uncertainty for high-quality data in adaptation. Our framework periodically adapts the source domain model using a teacher-student model through a self-training loss on the memory samples, incorporating soft alignment losses on batch normalization. We empirically validate ResiTTA across various benchmark datasets, demonstrating state-of-the-art performance.

With two central galaxies engaged in a major merger and a remarkable chain of 19 young stellar superclusters wound around them in projection, the galaxy cluster SDSS J1531+3414 ($z=0.335$) offers an excellent laboratory to study the interplay between mergers, AGN feedback, and star formation. New Chandra X-ray imaging reveals rapidly cooling hot ($T\sim 10^6$ K) intracluster gas, with two "wings" forming a concave density discontinuity near the edge of the cool core. LOFAR $144$ MHz observations uncover diffuse radio emission strikingly aligned with the "wings," suggesting that the "wings" are actually the opening to a giant X-ray supercavity. The steep radio emission is likely an ancient relic of one of the most energetic AGN outbursts observed, with $4pV > 10^{61}$ erg. To the north of the supercavity, GMOS detects warm ($T\sim 10^4$ K) ionized gas that enshrouds the stellar superclusters but is redshifted up to $+ 800$ km s$^{-1}$ with respect to the southern central galaxy. ALMA detects a similarly redshifted $\sim 10^{10}$ M$_\odot$ reservoir of cold ($T\sim 10^2$ K) molecular gas, but it is offset from the young stars by $\sim 1{-}3$ kpc. We propose that the multiphase gas originated from low-entropy gas entrained by the X-ray supercavity, attribute the offset between the young stars and the molecular gas to turbulent intracluster gas motions, and suggest that tidal interactions stimulated the "beads on a string" star formation morphology., Comment: Accepted by ApJ, 36 pages, 23 figures

Out-of-distribution (OOD) generalization is a complicated problem due to the idiosyncrasies of possible distribution shifts between training and test domains. Most benchmarks employ diverse datasets to address this issue; however, the degree of the distribution shift between the training domains and the test domains of each dataset remains largely fixed. This may lead to biased conclusions that either underestimate or overestimate the actual OOD performance of a model. Our study delves into a more nuanced evaluation setting that covers a broad range of shift degrees. We show that the robustness of models can be quite brittle and inconsistent under different degrees of distribution shifts, and therefore one should be more cautious when drawing conclusions from evaluations under a limited range of degrees. In addition, we observe that large-scale pre-trained models, such as CLIP, are sensitive to even minute distribution shifts of novel downstream tasks. This indicates that while pre-trained representations may help improve downstream in-distribution performance, they could have minimal or even adverse effects on generalization in certain OOD scenarios of the downstream task if not used properly. In light of these findings, we encourage future research to conduct evaluations across a broader range of shift degrees whenever possible.

Training large language models (LLMs) with open-domain instruction data has yielded remarkable success in aligning to end tasks and human preferences. Extensive research has highlighted the importance of the quality and diversity of instruction data. However, the impact of data complexity, as a crucial metric, remains relatively unexplored from three aspects: (1)where the sustainability of performance improvements with increasing complexity is uncertain; (2)whether the improvement brought by complexity merely comes from introducing more training tokens; and (3)where the potential benefits of incorporating instructions from easy to difficult are not yet fully understood. In this paper, we propose Tree-Instruct to systematically enhance the instruction complexity in a controllable manner. By adding a specified number of nodes to instructions' semantic trees, this approach not only yields new instruction data from the modified tree but also allows us to control the difficulty level of modified instructions. Our preliminary experiments reveal the following insights: (1)Increasing complexity consistently leads to sustained performance improvements of LLMs. (2)Under the same token budget, a few complex instructions outperform diverse yet simple instructions. (3)Curriculum instruction tuning might not yield the anticipated results; focusing on increasing complexity appears to be the key., Comment: LREC-Coling 2024