Search

Enhancing user engagement through personalization in conversational agents has gained significance, especially with the advent of large language models that generate fluent responses. Personalized dialogue generation, however, is multifaceted and varies in its definition -- ranging from instilling a persona in the agent to capturing users' explicit and implicit cues. This paper seeks to systemically survey the recent landscape of personalized dialogue generation, including the datasets employed, methodologies developed, and evaluation metrics applied. Covering 22 datasets, we highlight benchmark datasets and newer ones enriched with additional features. We further analyze 17 seminal works from top conferences between 2021-2023 and identify five distinct types of problems. We also shed light on recent progress by LLMs in personalized dialogue generation. Our evaluation section offers a comprehensive summary of assessment facets and metrics utilized in these works. In conclusion, we discuss prevailing challenges and envision prospect directions for future research in personalized dialogue generation., Comment: Presented in LREC-COLING 2024

A leading candidate for the heating source of chondrules and igneous rims is shock waves. This mechanism generates high relative velocities between chondrules and dust particles. We have investigated the possibility of the chondrule destruction in collisions with dust particles behind a shock wave using a semianalytical treatment. We find that the chondrules are destroyed during melting in collisions. We derive the conditions for the destruction of chondrules and show that the typical size of the observed chondrules satisfies the condition. We suggest that the chondrule formation and rim accretion are different events if they are heated by shock waves., Comment: accepted for publication in ApJ

User-generated data sources have gained significance in uncovering Adverse Drug Reactions (ADRs), with an increasing number of discussions occurring in the digital world. However, the existing clinical corpora predominantly revolve around scientific articles in English. This work presents a multilingual corpus of texts concerning ADRs gathered from diverse sources, including patient fora, social media, and clinical reports in German, French, and Japanese. Our corpus contains annotations covering 12 entity types, four attribute types, and 13 relation types. It contributes to the development of real-world multilingual language models for healthcare. We provide statistics to highlight certain challenges associated with the corpus and conduct preliminary experiments resulting in strong baselines for extracting entities and relations between these entities, both within and across languages., Comment: Accepted at LREC-COLING 2024

Antiferromagnetic (AF) order of U$_{2}$Pt$_{6}$Ga$_{15}$ with the ordering temperature $T_{\rm N}$ = 26 K was investigated by resonant X-ray scattering and neutron diffraction on single crystals. This compound possesses a unique crystal structure in which uranium ions form honeycomb layers and then stacks along the $c$-axis with slight offset, which gives rise to a stacking disorder. The AF order can be described with the propagation vector of $q = (1/6, 1/6, 0)$ in the hexagonal notation. The ordered magnetic moments orient perpendicular to the honeycomb layers, indicating a collinear spin structure consistent with Ising-like anisotropy. The magnetic reflections are found to be broadened along $c^*$ indicating that the stacking disorder results in anisotropic correlation lengths. The semi-quantitative analysis of neutron diffraction intensity, combined with group theory considerations based on the crystallographic symmetry, suggests a zig-zag type magnetic structure for the AF ground state, in which the AF coupling runs perpendicular to the stacking offset, characterized as $q = (1, 0, 0)_{\rm orth}$. The realization of the zig-zag magnetic structure implies the presence of frustrating intralayer exchange interactions involving both ferromagnetic (FM) first-neighbor and AF second and third-neighbor interactions in this compound., Comment: 17 pages, 10 figures, accepted for Phys. Rev. B

We propose an unsupervised approach to paraphrasing multiword expressions (MWEs) in context. Our model employs only monolingual corpus data and pre-trained language models (without fine-tuning), and does not make use of any external resources such as dictionaries. We evaluate our method on the SemEval 2022 idiomatic semantic text similarity task, and show that it outperforms all unsupervised systems and rivals supervised systems., Comment: 13 pages; accepted for Findings of ACL 2023

Shock wave heating is a leading candidate for the mechanisms of chondrule formation. This mechanism forms chondrules when the shock velocity is in a certain range. If the shock velocity is lower than this range, dust particles smaller than chondrule precursors melt, while chondrule precursors do not. We focus on the low-velocity shock waves as the igneous rim accretion events. Using a semi-analytical treatment of the shock-wave heating model, we found that the accretion of molten dust particles occurs when they are supercooling. The accreted igneous rims have two layers, which are the layers of the accreted supercooled droplets and crystallized dust particles. We suggest that chondrules experience multiple rim-forming shock events., Comment: accepted for publication in ApJ

During the patient's hospitalization, the physician must record daily observations of the patient and summarize them into a brief document called "discharge summary" when the patient is discharged. Automated generation of discharge summary can greatly relieve the physicians' burden, and has been addressed recently in the research community. Most previous studies of discharge summary generation using the sequence-to-sequence architecture focus on only inpatient notes for input. However, electric health records (EHR) also have rich structured metadata (e.g., hospital, physician, disease, length of stay, etc.) that might be useful. This paper investigates the effectiveness of medical meta-information for summarization tasks. We obtain four types of meta-information from the EHR systems and encode each meta-information into a sequence-to-sequence model. Using Japanese EHRs, meta-information encoded models increased ROUGE-1 by up to 4.45 points and BERTScore by 3.77 points over the vanilla Longformer. Also, we found that the encoded meta-information improves the precisions of its related terms in the outputs. Our results showed the benefit of the use of medical meta-information.

We report on the discovery of an Earth-sized transiting planet ($R_p=1.015\pm0.051\,R_\oplus$) in a $P=4.02$ day orbit around K2-415 (EPIC 211414619), an M5V star at 22 pc. The planet candidate was first identified by analyzing the light curve data by the K2 mission, and is here shown to exist in the most recent data from TESS. Combining the light curves with the data secured by our follow-up observations including high-resolution imaging and near infrared spectroscopy with IRD, we rule out false positive scenarios, finding a low false positive probability of $2\times 10^{-4}$. Based on IRD's radial velocities of K2-415, which were sparsely taken over three years, we obtain the planet mass of $3.0\pm 2.7\,M_\oplus$ ($M_p<7.5\,M_\oplus$ at $95\,\%$ confidence) for K2-415b. Being one of the lowest mass stars ($\approx 0.16\,M_\odot$) known to host an Earth-sized transiting planet, K2-415 will be an interesting target for further follow-up observations, including additional radial velocity monitoring and transit spectroscopy., Comment: 20 pages, 12 figures, accepted for publication in AJ

Discriminativeness is a desirable feature of image captions: captions should describe the characteristic details of input images. However, recent high-performing captioning models, which are trained with reinforcement learning (RL), tend to generate overly generic captions despite their high performance in various other criteria. First, we investigate the cause of the unexpectedly low discriminativeness and show that RL has a deeply rooted side effect of limiting the output words to high-frequency words. The limited vocabulary is a severe bottleneck for discriminativeness as it is difficult for a model to describe the details beyond its vocabulary. Then, based on this identification of the bottleneck, we drastically recast discriminative image captioning as a much simpler task of encouraging low-frequency word generation. Hinted by long-tail classification and debiasing methods, we propose methods that easily switch off-the-shelf RL models to discriminativeness-aware models with only a single-epoch fine-tuning on the part of the parameters. Extensive experiments demonstrate that our methods significantly enhance the discriminativeness of off-the-shelf RL models and even outperform previous discriminativeness-aware methods with much smaller computational costs. Detailed analysis and human evaluation also verify that our methods boost the discriminativeness without sacrificing the overall quality of captions., Comment: WACV 2023 (19 pages, 9 figures; updated appendix)

Automated summarization of clinical texts can reduce the burden of medical professionals. "Discharge summaries" are one promising application of the summarization, because they can be generated from daily inpatient records. Our preliminary experiment suggests that 20-31% of the descriptions in discharge summaries overlap with the content of the inpatient records. However, it remains unclear how the summaries should be generated from the unstructured source. To decompose the physician's summarization process, this study aimed to identify the optimal granularity in summarization. We first defined three types of summarization units with different granularities to compare the performance of the discharge summary generation: whole sentences, clinical segments, and clauses. We defined clinical segments in this study, aiming to express the smallest medically meaningful concepts. To obtain the clinical segments, it was necessary to automatically split the texts in the first stage of the pipeline. Accordingly, we compared rule-based methods and a machine learning method, and the latter outperformed the formers with an F1 score of 0.846 in the splitting task. Next, we experimentally measured the accuracy of extractive summarization using the three types of units, based on the ROUGE-1 metric, on a multi-institutional national archive of health records in Japan. The measured accuracies of extractive summarization using whole sentences, clinical segments, and clauses were 31.91, 36.15, and 25.18, respectively. We found that the clinical segments yielded higher accuracy than sentences and clauses. This result indicates that summarization of inpatient records demands finer granularity than sentence-oriented processing. Although we used only Japanese health records, it can be interpreted as follows: physicians extract "concepts of medical significance" from patient records and recombine them ...

We propose a new unsupervised method for lexical substitution using pre-trained language models. Compared to previous approaches that use the generative capability of language models to predict substitutes, our method retrieves substitutes based on the similarity of contextualised and decontextualised word embeddings, i.e. the average contextual representation of a word in multiple contexts. We conduct experiments in English and Italian, and show that our method substantially outperforms strong baselines and establishes a new state-of-the-art without any explicit supervision or fine-tuning. We further show that our method performs particularly well at predicting low-frequency substitutes, and also generates a diverse list of substitute candidates, reducing morphophonetic or morphosyntactic biases induced by article-noun agreement., Comment: 14 pages, accepted for COLING 2022

We report the near-infrared radial-velocity (RV) discovery of a super-Earth planet on a 10.77-day orbit around the M4.5 dwarf Ross 508 ($J_\mathrm{mag}=9.1$). Using precision RVs from the Subaru Telescope IRD (InfraRed Doppler) instrument, we derive a semi-amplitude of $3.92^{+0.60}_{-0.58}$ ${\rm m\,s}^{-1}$, corresponding to a planet with a minimum mass $m \sin i = 4.00^{+0.53}_{-0.55}\ M_{\oplus}$. We find no evidence of significant signals at the detected period in spectroscopic stellar activity indicators or MEarth photometry. The planet, Ross 508 b, has a semimajor-axis of $0.05366^{+0.00056}_{-0.00049}$ au. This gives an orbit-averaged insolation of $\approx$1.4 times the Earth's value, placing Ross 508 b near the inner edge of its star's habitable zone. We have explored the possibility that the planet has a high eccentricity and its host is accompanied by an additional unconfirmed companion on a wide orbit. Our discovery demonstrates that the near-infrared RV search can play a crucial role to find a low-mass planet around cool M dwarfs like Ross 508., Comment: Accepted for publication in PASJ (May 23, 2022)

The Kepler transit survey with follow-up spectroscopic observations has discovered numerous super-Earth sized planets and revealed intriguing features of their sizes, orbital periods, and their relations between adjacent planets. For the first time, we investigate the size evolution of planets via both giant impacts and photoevaporation to compare with these observed features. We calculate the size of a protoplanet, which is the sum of its core and envelope sizes, by analytical models. $N$-body simulations are performed to evolve planet sizes during the giant impact phase with envelope stripping via impact shocks. We consider the initial radial profile of the core mass and the initial envelope mass fractions as parameters. Inner planets can lose their whole envelopes via giant impacts, while outer planets can keep their initial envelopes since they do not experience giant impacts. Photoevaporation is simulated to evolve planet sizes afterward. Our results suggest that the period-radius distribution of the observed planets would be reproduced if we perform simulations in which the initial radial profile of the core mass follows a wide range of power-law distributions and the initial envelope mass fractions are $\sim0.1$. Moreover, our model shows that the adjacent planetary pairs have similar sizes and regular spacings, with slight differences from detailed observational results such as the radius gap., Comment: accepted for publication in ApJ

We present a novel mechanism for the outward transport of crystalline dust particles: the outward radial drift of pebbles. The dust ring structure is frequently observed in protoplanetary disks. One of the plausible mechanisms of the formation of dust rings is the accumulation of pebbles around the pressure maximum, which is formed by the mass loss due to magnetically driven disk winds. In evolving protoplanetary disks due to magnetically driven disk winds, dust particles can migrate outwardly from the crystallization front to the pressure maximum by radial drift. We found that the outward radial drift process can transport crystalline dust particles efficiently when the radial drift timescale is shorter than the advection timescale. Our model predicts that the crystallinity of silicate dust particles could be as high as 100% inside the dust ring position., Comment: 15 pages, 14 figures. Accepted for publication in ApJ

Chondrules are often surrounded by fine-grained rims or igneous rims. The properties of these rims reflect their formation histories. While the formation of fine-grained rims is modeled by the accretion of dust grains onto chondrules, the accretion should be followed by the growth of dust grains due to the shorter growth timescale than the accretion. In this paper, we investigate the formation of rims, taking into account the growth of porous dust aggregates. We estimate the rim thickness as a function of the chondrule fraction at a time when dust aggregate accretion onto chondrules is switched to collisions between these chondrules. Our estimations are consistent with the measured thicknesses of fine-grained rims in ordinary chondrites. However, those of igneous rims are thicker than our estimations. The thickness of igneous rims would be enlarged in remelting events., Comment: Accepted for publication in Icarus

Unsupervised image captioning is a challenging task that aims at generating captions without the supervision of image-sentence pairs, but only with images and sentences drawn from different sources and object labels detected from the images. In previous work, pseudo-captions, i.e., sentences that contain the detected object labels, were assigned to a given image. The focus of the previous work was on the alignment of input images and pseudo-captions at the sentence level. However, pseudo-captions contain many words that are irrelevant to a given image. In this work, we investigate the effect of removing mismatched words from image-sentence alignment to determine how they make this task difficult. We propose a simple gating mechanism that is trained to align image features with only the most reliable words in pseudo-captions: the detected object labels. The experimental results show that our proposed method outperforms the previous methods without introducing complex sentence-level learning objectives. Combined with the sentence-level alignment method of previous work, our method further improves its performance. These results confirm the importance of careful alignment in word-level details., Comment: EACL 2021 (11 pages, 3 figures; added references)

Recent observations have revealed the existence of multiple-planet systems composed of Earth-mass planets around late M dwarfs. Most of their orbits are close to commensurabilities, which suggests that planets were commonly trapped in resonant chains in their formation around low-mass stars. We investigate the formation of multiple-planet systems in resonant chains around low-mass stars. A time-evolution model of the multiple-planet formation via pebble accretion in the early phase of the disk evolution is constructed based on the formation model for the TRAPPIST-1 system by Ormel et al. (2017). Our simulations show that knowing the protoplanet appearance timescale is important for determining the number of planets and their trapped resonances: as the protoplanet appearance timescale increases, fewer planets are formed, which are trapped in more widely separated resonances. We find that there is a range of the protoplanet appearance timescale for forming the stable multiple-planet systems in resonant chains. This range depends on the stellar mass and disk size. We suggest that the protoplanet appearance timescale is a key parameter for studying the formation of multiple-planet systems with planets in resonant chains around low-mass stars. The composition of the planets in our model is also discussed., Comment: Accepted for Publication in ApJ

We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus (e.g. a few hundred sentence pairs). Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously translates and reconstructs an input sentence. Through sharing model parameters among different languages, our model jointly trains the word embeddings in a common cross-lingual space. We also propose to combine word and subword embeddings to make use of orthographic similarities across different languages. We base our experiments on real-world data from endangered languages, namely Yongning Na, Shipibo-Konibo, and Griko. Our experiments on bilingual lexicon induction and word alignment tasks show that our model outperforms existing methods by a large margin for most language pairs. These results demonstrate that, contrary to common belief, an encoder-decoder translation model is beneficial for learning cross-lingual representations even in extremely low-resource conditions. Furthermore, our model also works well on high-resource conditions, achieving state-of-the-art performance on a German-English word-alignment task., Comment: 16 pages, accepted at the 1st Workshop on Multilingual Representation Learning

Entity representations are useful in natural language tasks involving entities. In this paper, we propose new pretrained contextualized representations of words and entities based on the bidirectional transformer. The proposed model treats words and entities in a given text as independent tokens, and outputs contextualized representations of them. Our model is trained using a new pretraining task based on the masked language model of BERT. The task involves predicting randomly masked words and entities in a large entity-annotated corpus retrieved from Wikipedia. We also propose an entity-aware self-attention mechanism that is an extension of the self-attention mechanism of the transformer, and considers the types of tokens (words or entities) when computing attention scores. The proposed model achieves impressive empirical performance on a wide range of entity-related tasks. In particular, it obtains state-of-the-art results on five well-known datasets: Open Entity (entity typing), TACRED (relation classification), CoNLL-2003 (named entity recognition), ReCoRD (cloze-style question answering), and SQuAD 1.1 (extractive question answering). Our source code and pretrained representations are available at https://github.com/studio-ousia/luke., Comment: EMNLP 2020

Earth-sized planets were observed in close-in orbits around M dwarfs. While more and more planets are expected to be uncovered around M dwarfs, theories of their formation and dynamical evolution are still in their infancy. We investigate the giant impact growth of protoplanets, which includes strong scattering around low-mass stars. The aim is to clarify whether strong scattering around low-mass stars affects the orbital and mass distributions of the planets. We perform $N$-body simulation of protoplanets by systematically surveying the parameter space of the stellar mass and surface density of protoplanets. We find that protoplanets are often ejected after twice or three times close-scattering around late M dwarfs. The ejection sets the upper limit of the largest planet mass. Adopting the surface density scaling linearly with the stellar mass, we find that as the stellar mass decreases less massive planets are formed in orbits with higher eccentricities and inclinations. Under this scaling, we also find that a few close-in protoplanets are generally ejected. The ejection of protoplanets plays an important role in the mass distribution of super-Earths around late M dwarfs. The mass relation of observed close-in super-Earths and their central star mass is well reproduced by ejection., Comment: accepted for publication in A&A

Recent exoplanet observations reported a large number of multiple-planet systems, in which some of the planets are in a chain of resonances. The fraction of resonant systems to non-resonant systems provides clues about their formation history. We investigated the orbital stability of planets in resonant chains by considering the long-term evolution of planetary mass and stellar mass and using orbital calculations. We found that while resonant chains were stable, they can be destabilized by a change of $\sim$10% in planetary mass. Such a mass evolution can occur by atmospheric escape due to photoevaporation. We also found that resonant chains can be broken by a stellar mass loss of $\lesssim1$%, which would be explained by stellar winds or coronal mass ejections. The long-term mass change of planets and stars plays an important role in the orbital evolutions of planetary systems including super-Earths., Comment: accepted for publication in The Astrophysical Journal

We propose a new length-controllable abstractive summarization model. Recent state-of-the-art abstractive summarization models based on encoder-decoder models generate only one summary per source text. However, controllable summarization, especially of the length, is an important aspect for practical applications. Previous studies on length-controllable abstractive summarization incorporate length embeddings in the decoder module for controlling the summary length. Although the length embeddings can control where to stop decoding, they do not decide which information should be included in the summary within the length constraint. Unlike the previous models, our length-controllable abstractive summarization model incorporates a word-level extractive module in the encoder-decoder model instead of length embeddings. Our model generates a summary in two steps. First, our word-level extractor extracts a sequence of important words (we call it the "prototype text") from the source text according to the word-level importance scores and the length constraint. Second, the prototype text is used as additional input to the encoder-decoder model, which generates a summary by jointly encoding and copying words from both the prototype text and source text. Since the prototype text is a guide to both the content and length of the summary, our model can generate an informative and length-controlled summary. Experiments with the CNN/Daily Mail dataset and the NEWSROOM dataset show that our model outperformed previous models in length-controlled settings.

Major components of chondrites are chondrules and matrix. Measurements of the volatile abundance in Semarkona chondrules suggest that chondrules formed in a dense clump that had a higher solid density than the gas density in the solar nebula. We investigate collisions between chondrules and matrix in the surface region of dense clumps using fluffy aggregate growth models. Our simulations show that the collisional growth of aggregates composed of chondrules and matrix takes place in the clumps well before they experience gravitational collapse. The internal structure of chondrite parent bodies (CPBs) can be thereby determined by aggregate growth. We find that the aggregate growth generates two scales within CPBs. The first scale is involved with the small scale distribution of chondrules and determined by the early growth stage, where chondrules accrete aggregates composed of matrix grains. This accretion can reproduce the thickness of the matrix layer around chondrules found in chondrites. The other scale is related to the large scale distribution of chondrules. Its properties (e.g., the abundance of chondrules and the overall size) depend on the gas motion within the clump, which is parameterized in this work. Our work thus suggests that the internal structure of CPBs may provide important clues about their formation conditions and mechanisms., Comment: 24 pages, 17 figures, accepted for publication in ApJ

Understanding chondrule formation provides invaluable clues about the origin of the solar system. Recent studies suggest that planetesimal collisions and the resulting impact melts are promising for forming chondrules. Given that the dynamics of planetesimals is a key in impact-based chondrule formation scenarios, we here perform direct $N$-body simulations to examine how the presence of Jupiter affects the properties of chondrule-forming collisions. Our results show that the absence/presence of Jupiter considerably changes the properties of high velocity collisions whose impact velocities are higher than 2.5 km s$^{-1}$; high velocity collisions occur due to impacts between protoplanets and planetesimals for the case without Jupiter; for the case with Jupiter, eccentricities of planetesimals are pumped up by the secular and resonant perturbations from Jupiter. We also categorize the resulting planetesimal collisions and find that most of high velocity collisions are classified as grazing ones for both cases. To examine the effect of Jupiter on chondrule formation directly, we adopt the impact jetting scenario and compute the resulting abundance of chondrules. Our results show that for the case without Jupiter, chondrule formation proceeds in the inside-out manner, following the growth of protoplanets. If Jupiter is present, the location and timing of chondrule formation are determined by Jupiter's eccentricity, which is treated as a free parameter in our simulations. Thus, the existence of Jupiter is the key parameter for specifying when and where chondrule formation occurs for impact-based scenarios., Comment: 28 pages, 15 figures, 2 tables, ApJ accepted

We propose a global entity disambiguation (ED) model based on BERT. To capture global contextual information for ED, our model treats not only words but also entities as input tokens, and solves the task by sequentially resolving mentions to their referent entities and using resolved entities as inputs at each step. We train the model using a large entity-annotated corpus obtained from Wikipedia. We achieve new state-of-the-art results on five standard ED datasets: AIDA-CoNLL, MSNBC, AQUAINT, ACE2004, and WNED-WIKI. The source code and model checkpoint are available at https://github.com/studio-ousia/luke., Comment: NAACL 2022

Molecule property prediction is a fundamental problem for computer-aided drug discovery and materials science. Quantum-chemical simulations such as density functional theory (DFT) have been widely used for calculating the molecule properties, however, because of the heavy computational cost, it is difficult to search a huge number of potential chemical compounds. Machine learning methods for molecular modeling are attractive alternatives, however, the development of expressive, accurate, and scalable graph neural networks for learning molecular representations is still challenging. In this work, we propose a simple and powerful graph neural networks for molecular property prediction. We model a molecular as a directed complete graph in which each atom has a spatial position, and introduce a recursive neural network with simple gating function. We also feed input embeddings for every layers as skip connections to accelerate the training. Experimental results show that our model achieves the state-of-the-art performance on the standard benchmark dataset for molecular property prediction.

Biomedical Named Entity Recognition (BioNER) is a crucial step for analyzing Biomedical texts, which aims at extracting biomedical named entities from a given text. Different supervised machine learning algorithms have been applied for BioNER by various researchers. The main requirement of these approaches is an annotated dataset used for learning the parameters of machine learning algorithms. Segment Representation (SR) models comprise of different tag sets used for representing the annotated data, such as IOB2, IOE2 and IOBES. In this paper, we propose an extension of IOBES model to improve the performance of BioNER. The proposed SR model, FROBES, improves the representation of multi-word entities. We used Bidirectional Long Short-Term Memory (BiLSTM) network; an instance of Recurrent Neural Networks (RNN), to design a baseline system for BioNER and evaluated the new SR model on two datasets, i2b2/VA 2010 challenge dataset and JNLPBA 2004 shared task dataset. The proposed SR model outperforms other models for multi-word entities with length greater than two. Further, the outputs of different SR models have been combined using majority voting ensemble method which outperforms the baseline models performance., Comment: 13 pages, 2 figures, International Conference on Cognitive Informatics and Soft Computing (ICCISC-2017)