Search

XR devices running chat-bots powered by Large Language Models (LLMs) have tremendous potential as always-on agents that can enable much better productivity scenarios. However, screen based chat-bots do not take advantage of the the full-suite of natural inputs available in XR, including inward facing sensor data, instead they over-rely on explicit voice or text prompts, sometimes paired with multi-modal data dropped as part of the query. We propose a solution that leverages an attention framework that derives context implicitly from user actions, eye-gaze, and contextual memory within the XR environment. This minimizes the need for engineered explicit prompts, fostering grounded and intuitive interactions that glean user insights for the chat-bot. Our user studies demonstrate the imminent feasibility and transformative potential of our approach to streamline user interaction in XR with chat-bots, while offering insights for the design of future XR-embodied LLM agents.

Intelligent assistance involves not only understanding but also action. Existing ego-centric video datasets contain rich annotations of the videos, but not of actions that an intelligent assistant could perform in the moment. To address this gap, we release PARSE-Ego4D, a new set of personal action recommendation annotations for the Ego4D dataset. We take a multi-stage approach to generating and evaluating these annotations. First, we used a prompt-engineered large language model (LLM) to generate context-aware action suggestions and identified over 18,000 action suggestions. While these synthetic action suggestions are valuable, the inherent limitations of LLMs necessitate human evaluation. To ensure high-quality and user-centered recommendations, we conducted a large-scale human annotation study that provides grounding in human preferences for all of PARSE-Ego4D. We analyze the inter-rater agreement and evaluate subjective preferences of participants. Based on our synthetic dataset and complete human annotations, we propose several new tasks for action suggestions based on ego-centric videos. We encourage novel solutions that improve latency and energy requirements. The annotations in PARSE-Ego4D will support researchers and developers who are working on building action recommendation systems for augmented and virtual reality systems.

Virtual, Mixed, and Augmented Reality (XR) technologies hold immense potential for transforming productivity beyond PC. Therefore there is a critical need for improved text input solutions for XR. However, achieving efficient text input in these environments remains a significant challenge. This paper examines the current landscape of XR text input techniques, focusing on the importance of keyboards (both physical and virtual) as essential tools. We discuss the unique challenges and opportunities presented by XR, synthesizing key trends from existing solutions.

Seamless integration of physical objects as interactive digital entities remains a challenge for spatial computing. This paper explores Artificial Object Intelligence (AOI) in the context of XR, an interaction paradigm that aims to blur the lines between digital and physical by equipping real-world objects with the ability to interact as if they were digital, where every object has the potential to serve as a portal to digital functionalities. Our approach utilizes real-time object segmentation and classification, combined with the power of Multimodal Large Language Models (MLLMs), to facilitate these interactions without the need for object pre-registration. We implement the AOI concept in the form of XR-Objects, an open-source prototype system that provides a platform for users to engage with their physical environment in contextually relevant ways using object-based context menus. This system enables analog objects to not only convey information but also to initiate digital actions, such as querying for details or executing tasks. Our contributions are threefold: (1) we define the AOI concept and detail its advantages over traditional AI assistants, (2) detail the XR-Objects system's open-source design and implementation, and (3) show its versatility through various use cases and a user study., Comment: To appear in the Proceedings of the 2024 ACM Symposium on User Interface Software and Technology (UIST)

M-theory compactified on a $G_2$ manifold with resolved $E_8$ singularity is a promising candidate for a unified theory. The experimentally observed masses of quarks and charged leptons put a restriction on the moduli of the $G_2$ manifold. These moduli in turn uniquely determine the Dirac interactions of the neutrinos. In the paper, we explicitly compute the Dirac terms for neutrino mass matrix using the moduli from a localized model with resolved $E_8$ singularities on a $G_2$ manifold. This is a novel approach as the Dirac terms are not assumed but derived from the structure of quarks' and charged leptons' masses. Using known mass splittings and mixing angles of neutrinos, we show the acceptable region for Majorana terms. We also analyse the theoretical region for Majorana terms induced from the expectation values of right handed neutrinos through the Kolda-Martin mechanism. The intersection of the two regions indicates a restriction on neutrino masses. In particular, the lightest neutrino must have small but non-zero mass. Moreover, this also puts constraints on possible Majorana contributions from K\"ahler potential and superpotential, which can be traced down to a restriction on the geometry.We conclude that the masses of the two heavier light neutrinos are about $0.05 \text{ eV}$ and $0.009 \text{ eV}$ ($0.05 \text{ eV} $ and $0.05 \text{ eV} $)) for normal (inverted) hierarchy. In both hierarchies, we predict the light neutrinos are mostly Dirac type. Hence neutrino-less double-beta decay will be small. This is a testable result in a near future. Some bounds on heavy neutrinos are also derived.

Reheating is a process where the energy density of a dominant component of the universe other than radiation, such as a matter component, is transferred into radiation. It is usually assumed that the temperature of the universe decreases due to cosmic expansion even during the reheating process, in which case the maximal temperature of the universe is much higher than the reheat temperature. We point out that the temperature of the universe during reheating may in fact increase in well-motivated scenarios. We derive the necessary conditions for the temperature to increase during reheating and discuss concrete examples involving a scalar field. We comment on implications for particle physics and cosmology due to an increasing temperature during reheating., Comment: 24 pages, 7 figures

M-theory compactified on a $G_2$ manifold with resolved $E_8$ singularities realizes 4d $\mathcal{N} = 1$ supersymmetric gauge theories coupled to gravity with three families of Standard Model fermions. Beginning with one $E_8$ singularity, three fermion families emerge when $E_8$ is broken by geometric engineering deformations to a smaller subgroup with equal rank. In this paper, we use the local geometry of the theory to explain the origin of the three families and their mass hierarchy. We linearize the blowing-up of 2-cycles associated with resolving $E_8$ singularities. After imposing explicit constraints on the effectively stabilized moduli, we arrive at Yukawa couplings for the quarks and leptons. We fit the high scale Yukawa couplings approximately which results in the quark masses agreeing reasonably well with the observations, implying that the experimental hierarchy of the masses is achievable within this framework. The hierarchy separation of the top quark from the charm and up is a stringy effect, while the spitting of the charm and up also depends on the Higgs sector. The Higgs sector cannot be reduced to having a single vev; all three vevs must be non-zero.Three extra $U(1)$'s survive to the low scale but are not massless, so Z' states are motivated to occur in the spectrum, but may be massive.

Several theoretical motivations point to ultralight QCD axions with large decay constants $f_a \simeq \mathcal{O}(10^{16}-10^{17})$ GeV, to which experimental proposals are dedicated. This regime is known to face the problem of overproduction of axion dark matter from the misalignment mechanism unless the misalignment angle $\theta_{\rm mis}$ is as small as $\mathcal{O}(10^{-3}-10^{-4})$, which is generally considered a fine-tuning problem. We investigate a dynamical explanation for a small $\theta_{\rm mis}$. The axion mass arises from strong dynamics and may be sufficiently enhanced by early dynamics so as to overcome Hubble friction and drive the field value to the bottom of the potential long before the QCD phase transition. Together with an approximate CP symmetry in the theory, this minimum is very closely related to today's value and thus $\theta_{\rm mis}$ can automatically be well under unity. Owing to such efficient relaxation, the isocurvature perturbations are essentially damped. As an existence proof, using supersymmetric theories we illustrate that the Higgs coupling with the inflaton energy can successfully achieve this axion damping in a consistent inflationary cosmology., Comment: 16 pages, 2 figures; v2: minor changes and references added, matches journal version

The QCD axion serves as a well-motivated dark matter candidate and the misalignment mechanism is known to reproduce the observed abundance with a decay constant $f_a \simeq \mathcal{O}(10^{12})$ GeV for a misalignment angle $\theta_{\rm mis} \simeq \mathcal{O}(1)$. While $f_a \ll 10^{12}$ GeV is of great experimental interest, the misalignment mechanism requires the axion to be very close to the hilltop, i.e. $\theta_{\rm mis} \simeq \pi$. This particular choice of $\theta_{\rm mis}$ has been understood as fine-tuning the initial condition. We offer a dynamical explanation for $\theta_{\rm mis} \simeq \pi$ in a class of models. The axion dynamically relaxes to the minimum of the potential by virtue of an enhanced mass in the early universe. This minimum is subsequently converted to a hilltop because the CP phase of the theory shifts by $\pi$ when one contribution becomes subdominant to another with an opposite sign. We demonstrate explicit and viable examples in supersymmetric models where the higher dimensional Higgs coupling with the inflaton naturally achieves both criteria. Associated phenomenology includes a strikingly sharp prediction of $3 \times 10^9$ GeV $\lesssim f_a \lesssim 10^{10}$ GeV and the absence of isocurvature perturbation., Comment: 25 pages, 3 figures; v2: discussions and references added, matches journal version

We examine the experimental signature of a UV complete Supersymmetry (SUSY) theory, the $G_2$-MSSM. This model predicts that only some superpartners will be produced in possibly detectable amounts at LHC: $p p \rightarrow \tilde{g}\tilde{g}$, $p p \rightarrow \tilde{\chi}^{\pm}_1 \tilde{\chi}^{\mp}_1$, and $p p \rightarrow \tilde{\chi}^{0}_2 \tilde{\chi}^{\pm}_1$. We exclude spectra with $m_{\tilde{g}} \approx1.5$ TeV. While spectra with $m_{\tilde{g}} \approx1.7$ TeV and $m_{\tilde{g}} \approx1.9$ TeV are currently allowed (contrary to what is often claimed), data in hand could exclude these spectra. This is not in tension with reported exclusion limits due to the difference in decay topologies between simplified models and a UV motivated ($G_2$-MSSM) model., Comment: 6 pages, 4 tables