Search

To be safely and successfully deployed, LLMs must simultaneously satisfy truthfulness and utility goals. Yet, often these two goals compete (e.g., an AI agent assisting a used car salesman selling a car with flaws), partly due to ambiguous or misleading user instructions. We propose AI-LieDar, a framework to study how LLM-based agents navigate scenarios with utility-truthfulness conflicts in a multi-turn interactive setting. We design a set of realistic scenarios where language agents are instructed to achieve goals that are in conflict with being truthful during a multi-turn conversation with simulated human agents. To evaluate the truthfulness at large scale, we develop a truthfulness detector inspired by psychological literature to assess the agents' responses. Our experiment demonstrates that all models are truthful less than 50% of the time, although truthfulness and goal achievement (utility) rates vary across models. We further test the steerability of LLMs towards truthfulness, finding that models follow malicious instructions to deceive, and even truth-steered models can still lie. These findings reveal the complex nature of truthfulness in LLMs and underscore the importance of further research to ensure the safe and reliable deployment of LLMs and AI agents.

The Nab experiment will measure the electron-neutrino correlation and Fierz interference term in free neutron beta decay to test the Standard Model and probe Beyond the Standard Model Physics. Using National Instrument's PXIe-5171 Reconfigurable Oscilloscope module, we have developed a data acquisition system that is not only capable of meeting Nab's specifications, but flexible enough to be adapted in situ as the experimental environment dictates. The L1 and L2 trigger logic can be reconfigured to optimize the system for coincidence event detection at runtime through configuration files and LabVIEW controls. This system is capable of identifying L1 triggers at at least $1$ MHz, while reading out a peak signal rate of approximately $2$ GB/s. During commissioning, the system ran at a sustained readout rate of $400$ MB/s of signal data originating from roughly $6$ kHz L2 triggers, well within the peak performance of the system., Comment: 11 pages, 6 figures

Social media enables activists to directly communicate with the public and provides a space for movement leaders, participants, bystanders, and opponents to collectively construct and contest narratives. Focusing on Twitter messages from social movements surrounding three issues in 2018-2019 (guns, immigration, and LGBTQ rights), we create a codebook, annotated dataset, and computational models to detect diagnostic (problem identification and attribution), prognostic (proposed solutions and tactics), and motivational (calls to action) framing strategies. We conduct an in-depth unsupervised linguistic analysis of each framing strategy, and uncover cross-movement similarities in associations between framing and linguistic features such as pronouns and deontic modal verbs. Finally, we compare framing strategies across issues and other social, cultural, and interactional contexts. For example, we show that diagnostic framing is more common in replies than original broadcast posts, and that social movement organizations focus much more on prognostic and motivational framing than journalists and ordinary citizens., Comment: Published in ICWSM Special Issue of the Journal of Quantitative Description: Digital Media

Manually annotating data for computational social science tasks can be costly, time-consuming, and emotionally draining. While recent work suggests that LLMs can perform such annotation tasks in zero-shot settings, little is known about how prompt design impacts LLMs' compliance and accuracy. We conduct a large-scale multi-prompt experiment to test how model selection (ChatGPT, PaLM2, and Falcon7b) and prompt design features (definition inclusion, output type, explanation, and prompt length) impact the compliance and accuracy of LLM-generated annotations on four CSS tasks (toxicity, sentiment, rumor stance, and news frames). Our results show that LLM compliance and accuracy are highly prompt-dependent. For instance, prompting for numerical scores instead of labels reduces all LLMs' compliance and accuracy. The overall best prompting setup is task-dependent, and minor prompt changes can cause large changes in the distribution of generated labels. By showing that prompt design significantly impacts the quality and distribution of LLM-generated annotations, this work serves as both a warning and practical guide for researchers and practitioners., Comment: under review

Exposure to large language model output is rapidly increasing. How will seeing AI-generated ideas affect human ideas? We conducted an experiment (800+ participants, 40+ countries) where participants viewed creative ideas that were from ChatGPT or prior experimental participants and then brainstormed their own idea. We varied the number of AI-generated examples (none, low, or high exposure) and if the examples were labeled as 'AI' (disclosure). Our dynamic experiment design -- ideas from prior participants in an experimental condition are used as stimuli for future participants in the same experimental condition -- speaks to the interdependent process of cultural creation: creative ideas are built upon prior ideas. Hence, we capture the compounding effects of having LLMs 'in the culture loop'. We find that high AI exposure (but not low AI exposure) did not affect the creativity of individual ideas but did increase the average amount and rate of change of collective idea diversity. AI made ideas different, not better. There were no main effects of disclosure. We also found that self-reported creative people were less influenced by knowing an idea was from AI and that participants may knowingly adopt AI ideas when the task is difficult. Our findings suggest that introducing AI ideas may increase collective diversity but not individual creativity.

Fundamental neutron physics, combining precision measurements and theory, probes particle physics at short range with reach well beyond the highest energies probed by the LHC. Significant US efforts are underway that will probe BSM CP violation with orders of magnitude more sensitivity, provide new data on the Cabibbo anomaly, more precisely measure the neutron lifetime and decay, and explore hadronic parity violation. World-leading results from the US Fundamental Neutron Physics community since the last Long Range Plan, include the world's most precise measurement of the neutron lifetime from UCN$\tau$, the final results on the beta-asymmetry from UCNA and new results on hadronic parity violation from the NPDGamma and n-${^3}$He runs at the FNPB (Fundamental Neutron Physics Beamline), precision measurement of the radiative neutron decay mode and n-${}^4$He at NIST. US leadership and discovery potential are ensured by the development of new high-impact experiments including BL3, Nab, LANL nEDM and nEDM@SNS. On the theory side, the last few years have seen results for the neutron EDM from the QCD $\theta$ term, a factor of two reduction in the uncertainty for inner radiative corrections in beta-decay which impacts CKM unitarity, and progress on {\it ab initio} calculations of nuclear structure for medium-mass and heavy nuclei which can eventually improve the connection between nuclear and nucleon EDMs. In order to maintain this exciting program and capitalize on past investments while also pursuing new ideas and building US leadership in new areas, the Fundamental Neutron Physics community has identified a number of priorities and opportunities for our sub-field covering the time-frame of the last Long Range Plan (LRP) under development. This white paper elaborates on these priorities., Comment: arXiv admin note: text overlap with arXiv:2304.03451

The generalized Oberwolfach problem asks for a factorization of the complete graph $K_v$ into prescribed $2$-factors and at most a $1$-factor. When all $2$-factors are pairwise isomorphic and $v$ is odd, we have the classic Oberwolfach problem, which was originally stated as a seating problem: given $v$ attendees at a conference with $t$ circular tables such that the $i$th table seats $a_i$ people and ${\sum_{i=1}^t a_i = v}$, find a seating arrangement over the $\frac{v-1}{2}$ days of the conference, so that every person sits next to each other person exactly once. In this paper we introduce the related {\em minisymposium problem}, which requires a solution to the generalized Oberwolfach problem on $v$ vertices that contains a subsystem on $m$ vertices. That is, the decomposition restricted to the required $m$ vertices is a solution to the generalized Oberwolfach problem on $m$ vertices. In the seating context above, the larger conference contains a minisymposium of $m$ participants, and we also require that pairs of these $m$ participants be seated next to each other for $\left\lfloor\frac{m-1}{2}\right\rfloor$ of the days. When the cycles are as long as possible, i.e.\ $v$, $m$ and $v-m$, a flexible method of Hilton and Johnson provides a solution. We use this result to provide further solutions when $v \equiv m \equiv 2 \pmod 4$ and all cycle lengths are even. In addition, we provide extensive results in the case where all cycle lengths are equal to $k$, solving all cases when $m\mid v$, except possibly when $k$ is odd and $v$ is even., Comment: 25 pages

Dogwhistles are coded expressions that simultaneously convey one meaning to a broad audience and a second one, often hateful or provocative, to a narrow in-group; they are deployed to evade both political repercussions and algorithmic content moderation. For example, in the sentence 'we need to end the cosmopolitan experiment,' the word 'cosmopolitan' likely means 'worldly' to many, but secretly means 'Jewish' to a select few. We present the first large-scale computational investigation of dogwhistles. We develop a typology of dogwhistles, curate the largest-to-date glossary of over 300 dogwhistles with rich contextual information and examples, and analyze their usage in historical U.S. politicians' speeches. We then assess whether a large language model (GPT-3) can identify dogwhistles and their meanings, and find that GPT-3's performance varies widely across types of dogwhistles and targeted groups. Finally, we show that harmful content containing dogwhistles avoids toxicity detection, highlighting online risks of such coded language. This work sheds light on the theoretical and applied importance of dogwhistles in both NLP and computational social science, and provides resources for future research in modeling dogwhistles and mitigating their online harms., Comment: ACL 2023, see https://dogwhistles.allen.ai/ for the glossary and other materials

Social media enables the rapid spread of many kinds of information, from memes to social movements. However, little is known about how information crosses linguistic boundaries. We apply causal inference techniques on the European Twitter network to quantify multilingual users' structural role and communication influence in cross-lingual information exchange. Overall, multilinguals play an essential role; posting in multiple languages increases betweenness centrality by 13%, and having a multilingual network neighbor increases monolinguals' odds of sharing domains and hashtags from another language 16-fold and 4-fold, respectively. We further show that multilinguals have a greater impact on diffusing information less accessible to their monolingual compatriots, such as information from far-away countries and content about regional politics, nascent social movements, and job opportunities. By highlighting information exchange across borders, this work sheds light on a crucial component of how information and ideas spread around the world., Comment: ICWSM 2023 (please cite accordingly); see https://github.com/juliamendelsohn/bridging-nations for data, models, and code

We obtain an inequality for the kissing number in 16 dimensions. We do this by generalising a sum-product bound of Solymosi and Wong for quaternions to a semialgebra in dimension 16. In particular, we obtain the inequality $$k_{16}\geq \frac{\sum_{x \in \mathcal{R}}\left|\mathcal{S}_{x}\right|}{\left|\bigcup_{x \in \mathcal{R}} \mathcal{S}_{x}\right|}-1,$$ where $k_{16}$ is the 16-dimensional kissing number, and $S_x$ and $\mathcal{R}$ are sets defined below. Along the way we also obtain a sum-product bound for subsets of the octonions which are closed under taking inverses, using a similar strategy to that used for the quaternions. We use the fact that the kissing number in eight dimensions is 240 to achieve the result. Namely, we obtain the bound $$\operatorname{max}(|\mathcal{A}+\mathcal{A}|,|\mathcal{A}\mathcal{A}|)\geq \frac{|\mathcal{A}|^{4/3}}{(1928\cdot\lceil \operatorname{log}|\mathcal{A}|\rceil)^{1/3}},$$ where $\mathcal{A}$ is a finite set of octonions such that if $x\in\mathcal{A}$, $x^{-1}\in\mathcal{A}$ also., Comment: 11 pages

It is shown that the commuting probability of a finite ring cannot be a fraction with square-free denominator, resolving a conjecture of Buckley and MacHale., Comment: 5 pages, published in the Journal of Algebra and its Applications

The Nab experiment at Oak Ridge National Laboratory, USA, aims to measure the beta-antineutrino angular correlation following neutron $\beta$ decay to an anticipated precision of approximately 0.1\%. The proton momentum is reconstructed through proton time-of-flight measurements, and potential systematic biases in the timing reconstruction due to detector effects must be controlled at the nanosecond level. We present a thorough and detailed semiconductor and quasiparticle transport simulation effort to provide precise pulse shapes, and report on relevant systematic effects and potential measurement schemes.

Let $K$ be a totally real number field of degree $n$ over $\mathbb{Q}$, with discriminant and regulator $\Delta_K, R_K$ respectively. In this paper, using a similar method to van Woerden, we prove that the number of classes of perfect unary forms, up to equivalence and scaling, can be bounded above by $O( \Delta_K \exp(2n \log(n)+f(n,R_K)))$, where $f(n,R_K)$ is a finite value, satisfying $f(n,R_K)=\frac{\sqrt{n-1}}{2}R_K^{\frac{1}{n-1}}+\frac{4}{n-1}\log(\sqrt{|\Delta_K|})^2$ if $n \leq 11$. Moreover, if $K$ is a unit reducible field, the number of classes of perfect unary forms is bound above by $O( \Delta_K \exp(2n \log(n)))$., Comment: 8 pages

NLP research on public opinion manipulation campaigns has primarily focused on detecting overt strategies such as fake news and disinformation. However, information manipulation in the ongoing Russia-Ukraine war exemplifies how governments and media also employ more nuanced strategies. We release a new dataset, VoynaSlov, containing 38M+ posts from Russian media outlets on Twitter and VKontakte, as well as public activity and responses, immediately preceding and during the 2022 Russia-Ukraine war. We apply standard and recently-developed NLP models on VoynaSlov to examine agenda setting, framing, and priming, several strategies underlying information manipulation, and reveal variation across media outlet control, social media platform, and time. Our examination of these media effects and extensive discussion of current approaches' limitations encourage further development of NLP models for understanding information manipulation in emerging crises, as well as other real-world and interdisciplinary tasks., Comment: Findings of EMNLP 2022

We fit the mass and radial profile of the Orphan-Chenab Stream's (OCS) dwarf galaxy progenitor by using turnoff stars in the Sloan Digital Sky Survey (SDSS) and the Dark Energy Camera (DEC) to constrain N-body simulations of the OCS progenitor falling into the Milky Way on the 1.5 PetaFLOPS MilkyWay@home distributed supercomputer. We infer the internal structure of the OCS's progenitor under the assumption that it was a spherically symmetric dwarf galaxy comprised of a stellar system embedded in an extended dark matter halo. We optimize the evolution time, the baryonic and dark matter scale radii, and the baryonic and dark matter masses of the progenitor using a differential evolution algorithm. The likelihood score for each set of parameters is determined by comparing the simulated tidal stream to the angular distribution of OCS stars observed in the sky. We fit the total mass of the OCS's progenitor to ($2.0\pm0.3$) $\times 10^7 M_\odot$ with a mass-to-light ratio of $\gamma=73.5\pm10.6$ and ($1.1\pm0.2$)$\times10^6M_{\odot}$ within 300 pc of its center. Within the progenitor's half-light radius, we estimate total a mass of ($4.0\pm1.0$)$\times10^5M_{\odot}$. We also fit the current sky position of the progenitor's remnant to be $(\alpha,\delta)=((166.0\pm0.9)^\circ,(-11.1\pm2.5)^\circ)$ and show that it is gravitationally unbound at the present time. The measured progenitor mass is on the low end of previous measurements, and if confirmed lowers the mass range of ultrafaint dwarf galaxies. Our optimization assumes a fixed Milky Way potential, OCS orbit, and radial profile for the progenitor, ignoring the impact of the Large Magellanic Cloud., Comment: 33 pages, 53 figures, to be published by ApJ