Search

Training stability of large language models(LLMs) is an important research topic. Reproducing training instabilities can be costly, so we use a small language model with 830M parameters and experiment with higher learning rates to force models to diverge. One of the sources of training instability is the growth of logits in attention layers. We extend the focus of the previous work and look not only at the magnitude of the logits but at all outputs of linear layers in the Transformer block. We observe that with a high learning rate the L2 norm of all linear layer outputs can grow with each training step and the model diverges. Specifically we observe that QKV, Proj and FC2 layers have the largest growth of the output magnitude. This prompts us to explore several options: 1) apply layer normalization not only after QK layers but also after Proj and FC2 layers too; 2) apply layer normalization after the QKV layer (and remove pre normalization). 3) apply QK layer normalization together with softmax capping. We show that with the last two methods we can increase learning rate by 1.5x (without model divergence) in comparison to an approach based on QK layer normalization only. Also we observe significant perplexity improvements for all three methods in comparison to the baseline model.

Information graphics are pivotal in effective information dissemination and storytelling. However, creating such graphics is extremely challenging for non-professionals, since the design process requires multifaceted skills and comprehensive knowledge. Thus, despite the many available authoring tools, a significant gap remains in enabling non-experts to produce compelling information graphics seamlessly, especially from scratch. Recent breakthroughs show that Large Language Models (LLMs), especially when tool-augmented, can autonomously engage with external tools, making them promising candidates for enabling innovative graphic design applications. In this work, we propose a LLM-centric interface with the agent GraphiMind for automatic generation, recommendation, and composition of information graphics design resources, based on user intent expressed through natural language. Our GraphiMind integrates a Textual Conversational Interface, powered by tool-augmented LLM, with a traditional Graphical Manipulation Interface, streamlining the entire design process from raw resource curation to composition and refinement. Extensive evaluations highlight our tool's proficiency in simplifying the design process, opening avenues for its use by non-professional users. Moreover, we spotlight the potential of LLMs in reshaping the domain of information graphics design, offering a blend of automation, versatility, and user-centric interactivity.

The disruption to tourism and travel caused by the COVID-related health crisis highlighted the potential of virtual tourism to provide a universally accessible way to engage in cultural experiences. 360-degree virtual tours, showing a realistic representation of the physical location, enable virtual tourists to experience cultural heritage sites and engage with their collections from the comfort and safety of their home. However, there is no clear standard for the design of such tours and the experience of visitors may vary widely from platform to platform. We first conducted a comprehensive analysis of 40 existing virtual tours, constructing a descriptive framework for understanding the key components and characteristics of virtual tours. Next, we conducted a remote usability study to gain deeper insights into the actual experiences and challenges faced by VT users. Our investigation revealed a significant disparity between users' mental models of virtual tours and the actual system behavior. We discuss these issues and provide concrete recommendations for the creation of better, user-friendly 360-degree virtual tours.

This short paper describes an early look at a project aimed at aiding students to understand various viewpoints through the advocacy of a museum visit. It aims to both improve historical reasoning and allow for more open-mindedness. This is done by combining a process of citizen curation with the use of de-polarization techniques. [For the complete proceedings, see ED639262.]

We evaluate the large-$N$ behavior of the superconformal indices of toric quiver gauge theories, and use it to find the entropy functions of the dual electrically charged rotating $\mathrm{AdS}_5$ black holes. To this end, we employ the recently proposed Bethe Ansatz method, and find a certain set of solutions to the Bethe Ansatz Equations of toric theories. This, in turn, allows us to compute the large-$N$ behavior of the index for these theories, including the infinite families $Y^{pq}$, $X^{pq}$ and $L^{pqr}$ of quiver gauge theories. Our results are in perfect agreement with the predictions made recently using the Cardy-like limit of the superconformal index. We also explore the index structure in the space of chemical potentials and describe the pattern of Stokes lines arising in the conifold theory case., Comment: 48 pages, 9 figures

We numerically compute the renormalized expectation value $\langle\hat{\Phi}^{2}\rangle_{ren}$ of a minimally-coupled massless quantum scalar field in the interior of a four-dimensional Reissner-Nordstrom black hole, in both the Hartle-Hawking and Unruh states. To this end we use a recently developed mode-sum renormalization scheme based on covariant point splitting. In both quantum states, $\langle\hat{\Phi}^{2}\rangle_{ren}$ is found to approach a \emph{finite} value at the inner horizon (IH). The final approach to the IH asymptotic value is marked by an inverse-power tail $r_{*}^{-n}$, where $r_{*}$ is the Regge-Wheeler "tortoise coordinate", and with $n=2$ for the Hartle-Hawking state and $n=3$ for the Unruh state. We also report here the results of an analytical computation of these inverse-power tails of $\langle\hat{\Phi}^{2}\rangle_{ren}$ near the IH. Our numerical results show very good agreement with this analytical derivation (for both the power index and the tail amplitude), in both quantum states. Finally, from this asymptotic behavior of $\langle\hat{\Phi}^{2}\rangle_{ren}$ we analytically compute the leading-order asymptotic behavior of the trace $\langle\hat{T}_{\mu}^{\mu}\rangle_{ren}$ of the renormalized stress-energy tensor at the IH. In both quantum states this quantity is found to diverge like $b(r-r_{-})^{-1}r_{*}^{-n-2}$ (with $n$ specified above, and with a known parameter $b$). To the best of our knowledge, this is the first fully-quantitative derivation of the asymptotic behavior of these renormalized quantities at the inner horizon of a four-dimensional Reissner-Nordstrom black hole., Comment: 7 pages, 4 figures

The full computation of the renormalized expectation values $\langle\Phi^{2}\rangle_{ren}$ and $\langle\hat{T}_{\mu\nu}\rangle_{ren}$ in 4D black hole interiors has been a long standing challenge, which has impeded the investigation of quantum effects on the internal structure of black holes for decades. Employing a recently developed mode sum renormalization scheme to numerically implement the point-splitting method, we report here the first computation of $\langle\Phi^{2}\rangle_{ren}$ in Unruh state in the region inside the event horizon of a 4D Schwarzschild black hole. We further present its Hartle-Hawking counterpart, which we calculated using the same method, and obtain a fairly good agreement with previous results attained using an entirely different method by Candelas and Jensen in 1986. Our results further agree upon approaching the event horizon when compared with previous results calculated outside the black hole. Finally, the results we obtained for Hartle-Hawking state at the event horizon agree with previous analytical results published by Candelas in 1980. This work sets the stage for further explorations of $\langle\Phi^{2}\rangle_{ren}$ and $\langle\hat{T}_{\mu\nu}\rangle_{ren}$ in 4D black hole interiors., Comment: 14 pages, 6 figures

Recently a very interesting three-dimensional $\mathcal{N}=2$ supersymmetric theory with $SU(3)$ global symmetry was discussed by several authors. We denote this model by $T_x$. This was conjectured to have two dual descriptions, one with explicit supersymmetry and emergent flavor symmetry and the other with explicit flavor symmetry and emergent supersymmetry. We discuss a third description of the model which has both flavor symmetry and supersymmetry manifest. We then investigate models which can be constructed by using $T_x$ as a building block gauging the global symmetry and paying special attention to the global structure of the gauge group. We conjecture several cases of $\mathcal{N}=2$ mirror dualities involving such constructions with the dual being either a simple $\mathcal{N}=2$ Wess-Zumino model or a discrete gauging thereof., Comment: 21 pages, 7 figures, harvmac; v2: references added, typos fixed

We derive explicit expressions for the two-point function of a massless scalar field in the interior region of a Reissner-Nordstrom black hole, in both the Unruh and Hartle-Hawking quantum states. The two-point function is expressed in terms of the standard $lm\omega$ modes of the scalar field (those associated with a spherical harmonic $Y_{lm}$ and a temporal mode $e^{-i\omega t}$), which can be conveniently obtained by solving an ordinary differential equation, the radial equation. These explicit expressions are the internal analogs of the well known results in the external region (originally derived by Christensen and Fulling), in which the two-point function outside the black hole is written in terms of the external $lm\omega$ modes of the field. They allow the computation of $<\Phi^{2}>_{ren}$ and the renormalized stress-energy tensor inside the black hole, after the radial equation has been solved (usually numerically). In the second part of the paper, we provide an explicit expression for the trace of the renormalized stress-energy tensor of a minimally-coupled massless scalar field (which is non-conformal), relating it to the d'Alembertian of $<\Phi^{2}>_{ren}$. This expression proves itself useful in various calculations of the renormalized stress-energy tensor., Comment: 14 pages, 1 figure; Minor modifications; Accepted for publication in PRD

The electromagnetic self-force equation of motion is known to be afflicted by the so-called runaway problem. A similar problem arises in the semiclassical Einstein's field equation and plagues the self-consistent semiclassical evolution of spacetime. Motivated to overcome the latter challenge, we first address the former (which is conceptually simpler), and present a pragmatic finite-difference method designed to numerically integrate the self-force equation of motion while curing the runaway problem. We restrict our attention here to a charged point-like mass in a one-dimensional motion, under a prescribed time-dependent external force $F_{ext}(t)$. We demonstrate the implementation of our method using two different examples of external force: a Gaussian and a Sin^4 function. In each of these examples we compare our numerical results with those obtained by two other methods (a Dirac-type solution and a reduction-of-order solution). Both external-force examples demonstrate a complete suppression of the undesired runaway mode, along with an accurate account of the radiation-reaction effect at the physically relevant time scale, thereby illustrating the effectiveness of our method in curing the self-force runaway problem., Comment: 12 pages, 7 figures; A significant extension of this manuscript was published in PRD by A. Lanir and O. Sela