Search

The Perseus cluster is the brightest X-ray cluster in the sky and is known as a cool-core galaxy cluster. Being a very nearby cluster, it has been extensively studied. This has provided a comprehensive view of the physical processes that operate in the intracluster medium (ICM), including feedback from the AGN 3C84 and measurements of ICM turbulence. Additionally, the Perseus cluster contains a central radio mini-halo. This diffuse radio source traces cosmic ray electrons (re-)accelerated in-situ in the ICM. Here we report on LOFAR high-band antenna 120-168 MHz observations of the Perseus cluster that probe a range of four orders of magnitude in angular scales. In our 0.3 arcsec resolution image, we find that the northern extension of the 3C84 lobe consists of several narrow 1.5-3 kpc parallel strands of emission. In addition, we detect steep-spectrum filaments associated with a previous outburst of the central AGN radio emission filling two known X-ray ghost cavities. At 7 arcsec resolution, our images show a complex structured radio mini-halo, with several edges and filaments. At resolutions of 26 arcsec and 80 arcsec, we discover diffuse radio emission with a 1.1 Mpc extent. We classify this emission as a giant radio halo and its properties are distinct from the inner mini-halo. We also detect two diffuse sources at projected cluster centric radii of 0.7 and 1.0 Mpc. Finally, we observe a 0.9 Mpc long trail of radio emission from the cluster member galaxy IC310, connecting it with the giant radio halo. Together with other recent studies of relaxed clusters, our LOFAR observations indicate that cluster-wide radio emission could be (more) common in cool-core clusters. In the case of the Perseus cluster, a past off-axis merger event that preserved the cool core might have generated enough turbulence to produce an extended radio halo observable at low frequencies., Comment: accepted for publication in A&A, 18 pages, 19 figures

Today's most advanced multimodal models remain proprietary. The strongest open-weight models rely heavily on synthetic data from proprietary VLMs to achieve good performance, effectively distilling these closed models into open ones. As a result, the community is still missing foundational knowledge about how to build performant VLMs from scratch. We present Molmo, a new family of VLMs that are state-of-the-art in their class of openness. Our key innovation is a novel, highly detailed image caption dataset collected entirely from human annotators using speech-based descriptions. To enable a wide array of user interactions, we also introduce a diverse dataset mixture for fine-tuning that includes in-the-wild Q&A and innovative 2D pointing data. The success of our approach relies on careful choices for the model architecture details, a well-tuned training pipeline, and, most critically, the quality of our newly collected datasets, all of which will be released. The best-in-class 72B model within the Molmo family not only outperforms others in the class of open weight and data models but also compares favorably against proprietary systems like GPT-4o, Claude 3.5, and Gemini 1.5 on both academic benchmarks and human evaluation. We will be releasing all of our model weights, captioning and fine-tuning data, and source code in the near future. Select model weights, inference code, and demo are available at https://molmo.allenai.org.

We introduce OLMoE, a fully open, state-of-the-art language model leveraging sparse Mixture-of-Experts (MoE). OLMoE-1B-7B has 7 billion (B) parameters but uses only 1B per input token. We pretrain it on 5 trillion tokens and further adapt it to create OLMoE-1B-7B-Instruct. Our models outperform all available models with similar active parameters, even surpassing larger ones like Llama2-13B-Chat and DeepSeekMoE-16B. We present various experiments on MoE training, analyze routing in our model showing high specialization, and open-source all aspects of our work: model weights, training data, code, and logs., Comment: 61 pages (24 main), 36 figures, 14 tables

We introduce DataComp for Language Models (DCLM), a testbed for controlled dataset experiments with the goal of improving language models. As part of DCLM, we provide a standardized corpus of 240T tokens extracted from Common Crawl, effective pretraining recipes based on the OpenLM framework, and a broad suite of 53 downstream evaluations. Participants in the DCLM benchmark can experiment with data curation strategies such as deduplication, filtering, and data mixing at model scales ranging from 412M to 7B parameters. As a baseline for DCLM, we conduct extensive experiments and find that model-based filtering is key to assembling a high-quality training set. The resulting dataset, DCLM-Baseline enables training a 7B parameter language model from scratch to 64% 5-shot accuracy on MMLU with 2.6T training tokens. Compared to MAP-Neo, the previous state-of-the-art in open-data language models, DCLM-Baseline represents a 6.6 percentage point improvement on MMLU while being trained with 40% less compute. Our baseline model is also comparable to Mistral-7B-v0.3 and Llama 3 8B on MMLU (63% & 66%), and performs similarly on an average of 53 natural language understanding tasks while being trained with 6.6x less compute than Llama 3 8B. Our results highlight the importance of dataset design for training language models and offer a starting point for further research on data curation., Comment: Project page: https://www.datacomp.ai/dclm/

The largely unexplored decameter radio band (10-30 MHz) provides a unique window for studying a range of astronomical topics, such as auroral emission from exoplanets, inefficient cosmic ray acceleration mechanisms, fossil radio plasma, and free-free absorption. The scarcity of low-frequency studies is mainly due to the severe perturbing effects of the ionosphere. Here we present a calibration strategy that can correct for the ionosphere in the decameter band. We apply this to an observation from the Low Frequency Array (LOFAR) between 16 to 30 MHz . The resulting image covers 330 square degrees of sky at a resolution of 45", reaching a sensitivity of 12 mJy/beam. Residual ionospheric effects cause additional blurring ranging between 60 to 100". This represents an order of magnitude improvement in terms of sensitivity and resolution compared to previous decameter band observations. In the region we surveyed, we have identified four fossil plasma sources. These rare sources are believed to contain old, possibly re-energised, radio plasma originating from previous outbursts of active galactic nuclei. At least three of them are situated near the center of low-mass galaxy clusters. Notably, two of these sources display the steepest radio spectral index among all the sources detected at 23 MHz. This indicates that fossil plasma sources constitute the primary population of steep-spectrum sources at these frequencies, emphasising the large discovery potential of ground-based decameter observations., Comment: 19 pages, 9 figures, 1 table. Submitted version, full version is published by Nature Astronomy

Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have become closed off, gated behind proprietary interfaces, with important details of their training data, architectures, and development undisclosed. Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs. To this end, we have built OLMo, a competitive, truly Open Language Model, to enable the scientific study of language models. Unlike most prior efforts that have only released model weights and inference code, we release OLMo alongside open training data and training and evaluation code. We hope this release will empower the open research community and inspire a new wave of innovation.

Information about pretraining corpora used to train the current best-performing language models is seldom discussed: commercial models rarely detail their data, and even open models are often released without accompanying training data or recipes to reproduce them. As a result, it is challenging to conduct and advance scientific research on language modeling, such as understanding how training data impacts model capabilities and limitations. To facilitate scientific research on language model pretraining, we curate and release Dolma, a three-trillion-token English corpus, built from a diverse mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials. We extensively document Dolma, including its design principles, details about its construction, and a summary of its contents. We present analyses and experimental results on intermediate states of Dolma to share what we have learned about important data curation practices. Finally, we open-source our data curation toolkit to enable reproduction of our work as well as support further research in large-scale data curation., Comment: Accepted at ACL 2024; Dataset: https://hf.co/datasets/allenai/dolma; Code: https://github.com/allenai/dolma

Creativity is a critical skill that professional software engineers leverage to tackle difficult problems. In higher education, multiple efforts have been made to spark creative skills of engineering students. However, creativity is a vague concept that is open to interpretation. Furthermore, studies have shown that there is a gap in perception and implementation of creativity between industry and academia. To better understand the role of creativity in software engineering (SE), we interviewed 33 professionals via four focus groups and 10 SE students. Our results reveal 45 underlying topics related to creativity. When comparing the perception of students versus professionals, we discovered fundamental differences, grouped into five themes: the creative environment, application of techniques, creative collaboration, nature vs nurture, and the perceived value of creativity. As our aim is to use these findings to install and further encourage creative problem solving in higher education, we have included a list of implications for educational practice.

Language models (LMs) commonly report perplexity on monolithic data held out from training. Implicitly or explicitly, this data is composed of domains$\unicode{x2013}$varying distributions of language. Rather than assuming perplexity on one distribution extrapolates to others, Perplexity Analysis for Language Model Assessment (Paloma), measures LM fit to 585 text domains, ranging from nytimes.com to r/depression on Reddit. We invite submissions to our benchmark and organize results by comparability based on compliance with guidelines such as removal of benchmark contamination from pretraining. Submissions can also record parameter and training token count to make comparisons of Pareto efficiency for performance as a function of these measures of cost. We populate our benchmark with results from 6 baselines pretrained on popular corpora. In case studies, we demonstrate analyses that are possible with Paloma, such as finding that pretraining without data beyond Common Crawl leads to inconsistent fit to many domains., Comment: Project Page: https://paloma.allen.ai/

The success of large language models has shifted the evaluation paradigms in natural language processing (NLP). The community's interest has drifted towards comparing NLP models across many tasks, domains, and datasets, often at an extreme scale. This imposes new engineering challenges: efforts in constructing datasets and models have been fragmented, and their formats and interfaces are incompatible. As a result, it often takes extensive (re)implementation efforts to make fair and controlled comparisons at scale. Catwalk aims to address these issues. Catwalk provides a unified interface to a broad range of existing NLP datasets and models, ranging from both canonical supervised training and fine-tuning, to more modern paradigms like in-context learning. Its carefully-designed abstractions allow for easy extensions to many others. Catwalk substantially lowers the barriers to conducting controlled experiments at scale. For example, we finetuned and evaluated over 64 models on over 86 datasets with a single command, without writing any code. Maintained by the AllenNLP team at the Allen Institute for Artificial Intelligence (AI2), Catwalk is an ongoing open-source effort: https://github.com/allenai/catwalk., Comment: technical report, work in progress

Large text corpora are the backbone of language models. However, we have a limited understanding of the content of these corpora, including general statistics, quality, social factors, and inclusion of evaluation data (contamination). In this work, we propose What's In My Big Data? (WIMBD), a platform and a set of sixteen analyses that allow us to reveal and compare the contents of large text corpora. WIMBD builds on two basic capabilities -- count and search -- at scale, which allows us to analyze more than 35 terabytes on a standard compute node. We apply WIMBD to ten different corpora used to train popular language models, including C4, The Pile, and RedPajama. Our analysis uncovers several surprising and previously undocumented findings about these corpora, including the high prevalence of duplicate, synthetic, and low-quality content, personally identifiable information, toxic language, and benchmark contamination. For instance, we find that about 50% of the documents in RedPajama and LAION-2B-en are duplicates. In addition, several datasets used for benchmarking models trained on such corpora are contaminated with respect to important benchmarks, including the Winograd Schema Challenge and parts of GLUE and SuperGLUE. We open-source WIMBD's code and artifacts to provide a standard set of evaluations for new text-based corpora and to encourage more analyses and transparency around them., Comment: Published at ICLR 2024 spotlight

This study explores the influence of an interdisciplinary intervention on creative problem-solving skills. Literature deems such skills as vital for software engineering (SE) students in higher education. 39 SE students and graphic design (GD) students were randomly paired to work on an open-ended creative coding assignment in p5.js, an online JS-based Processing editor that makes it easy for novices to quickly and easily code visual webpages. Three categories were formed: the test group SE+GD (18 students), and control groups SE+SE (10) and GD+GD (11). A mixed methods approach was taken to gather and interpret results: Amabile's Consensual Assessment Technique provided a global creativity score for the finished product, the Creative Programming Problem Solving Test assessed three dimensions of the creative process (Ability, Mindset, Interaction), and 9 semi-structured follow-up interviews provided context and revealed underlying themes. The results indicate that, while the creativity of the end product initially takes a hit, the SE+GD groups' socio-interactive creativity levels increased. We also observed fixed mindsets towards creativity ("design students are more creative than we") that call for future work.

Feedback beyond the grade is an important part of the learning process. However, because of the large student groups, many teachers in higher education are faced with practicalities such as the limited time to prepare and communicate the feedback to individual students. We have set up an experiment, titled Pro-f-quiz, in which over two years 236 students participated and in which the feedback is communicated through an online quiz that activates the students to reflect upon their solution. The system can be set up in a very limited time compared to booking individual time slots. The results show that approximately 85% of the students appreciate the approach with 60% indicating that they reflect more intensively about their work than when the feedback is transmitted traditionally. Moreover, the grade of the students participating in the project was substantially higher than students not participating.

Large language models (LLMs) enable unparalleled few- and zero-shot reasoning capabilities but at a high computational footprint. A growing assortment of methods for compression promises to reduce the computational burden of LLMs in deployment, but so far, only quantization approaches have been demonstrated to be effective for LLM compression while maintaining zero-shot performance. A critical step in the compression process, the pretrain-then-finetune paradigm, has largely been overlooked when adapting existing pruning strategies to LLMs or proposing new ones. In this work, we show that embarrassingly simple layer pruning coupled with an extended language model pretraining as the finetuning phase produces state-of-the-art results against structured and even semi-structured compression of models at a 7B scale while being more inference efficient. We call this method LayerChop, where we deterministically remove layers from a model followed by task-agnostic finetuning of the remaining weights by continued self-supervised pretraining. At this scale, we also show how distillation, which has been super effective in task-agnostic compression of smaller BERT-style models, becomes inefficient against our simple pruning technique., Comment: 13 pages, 6 figures, 6 tables

Recently introduced language model prompting methods can achieve high accuracy in zero- and few-shot settings while requiring few to no learned task-specific parameters. Nevertheless, these methods still often trail behind full model finetuning. In this work, we investigate if a dedicated continued pretraining stage could improve "promptability", i.e., zero-shot performance with natural language prompts or few-shot performance with prompt tuning. We reveal settings where existing continued pretraining methods lack promptability. We also identify current methodological gaps, which we fill with thorough large-scale experiments. We demonstrate that a simple recipe, continued pretraining that incorporates a trainable prompt during multi-task learning, leads to improved promptability in both zero- and few-shot settings compared to existing methods, up to 31% relative. On the other hand, we find that continued pretraining using MAML-style meta-learning, a method that directly optimizes few-shot promptability, yields subpar performance. We validate our findings with two prompt tuning methods, and, based on our results, we provide concrete recommendations to optimize promptability for different use cases., Comment: EMNLP 2022

We developed a self-assessment tool for computing students in higher education to measure their Creative Problem Solving skills. Our survey encompasses 7 dimensions of creativity, based on existing validated scales and conducted focus groups. These are: technical knowledge, communication, constraints, critical thinking, curiosity, creative state of mind, and creative techniques. Principal axis factor analysis groups the dimensions into three overarching constructs: ability, mindset, and interaction. The results of a pilot study (n = 269) provide evidence for its psychometric qualities, making it a useful instrument for educational researchers to investigate students' creative skills.

The compact steep spectrum radio source, 3C 298, (redshift of 1.44) has the largest 178 MHz luminosity in the 3CR (revised Third Cambridge Catalogue) catalog; its radio lobes are among the most luminous in the Universe. The plasma state of the radio lobes is modeled with the aid of interferometric radio observations (in particular, the new Low Frequency Array observation and archival MERLIN data) and archival single-station data. It is estimated that the long-term time-averaged jet power required to fill these lobes with leptonic plasma is $\overline{Q} \approx 1.28 \pm 0.51 \times 10^{47} \rm{erg}\,\rm{s}^{-1}$, rivaling the largest time averaged jet powers from any quasar. Supporting this notion of extraordinary jet power is a 0.5 keV -10 keV luminosity of $\approx 5.2 \times 10^{46} \rm{erg}\, \rm{s}^{-1}$, comparable to luminous blazars, yet there is no other indication of strong relativistic beaming. We combine two new high signal to noise optical spectroscopic observations from the Hobby-Eberly Telescope with archival Hubble Space Telescope, Two Micron Survey and Galaxy Evolutionary Explorer data to compute a bolometric luminosity from the accretion flow of $L_{\rm{bol}} \approx 1.55 \pm 0.15 \times 10^{47} \rm{erg} \,\rm{s}^{-1}$. The ratio, $\overline{Q}/L_{\rm{bol}}\approx 1$, is the approximate upper limit for quasars. Characteristic of a large $\overline{Q}/L_{\rm{bol}}$, we find an extreme ultraviolet (EUV) spectrum that is very steep (the "EUV deficit" of powerful radio quasars relative to radio quiet quasars) and this weak ionizing continuum is likely a contributing factor to the relatively small equivalent widths of the broad emission lines in this quasar., Comment: To appear in ApJ

Acute Lymphoblastic Leukemia (ALL) is the most frequent hematologic malignancy in children and adolescents. A strong prognostic factor in ALL is given by the Minimal Residual Disease (MRD), which is a measure for the number of leukemic cells persistent in a patient. Manual MRD assessment from Multiparameter Flow Cytometry (FCM) data after treatment is time-consuming and subjective. In this work, we present an automated method to compute the MRD value directly from FCM data. We present a novel neural network approach based on the transformer architecture that learns to directly identify blast cells in a sample. We train our method in a supervised manner and evaluate it on publicly available ALL FCM data from three different clinical centers. Our method reaches a median F1 score of ~0.94 when evaluated on 519 B-ALL samples and shows better results than existing methods on 4 different datasets, Comment: The article has been published as an open access article in the Journal for Computers in Biology and Medicine: https://doi.org/10.1016/j.compbiomed.2022.105314

[abridged] The International LOFAR Telescope is an interferometer with stations spread across Europe. With baselines of up to ~2,000 km, LOFAR has the unique capability of achieving sub-arcsecond resolution at frequencies below 200 MHz, although this is technically and logistically challenging. Here we present a calibration strategy that builds on previous high-resolution work with LOFAR. We give an overview of the calibration strategy and discuss the special challenges inherent to enacting high-resolution imaging with LOFAR, and describe the pipeline, which is publicly available, in detail. We demonstrate the calibration strategy by using the pipeline on P205+55, a typical LOFAR Two-metre Sky Survey (LoTSS) pointing. We perform in-field delay calibration, solution referencing to other calibrators, self-calibration, and imaging of example directions of interest in the field. For this specific field and these ionospheric conditions, dispersive delay solutions can be transferred between calibrators up to ~1.5 degrees away, while phase solution transferral works well over 1 degree. We demonstrate a check of the astrometry and flux density scale. Imaging in 17 directions, the restoring beam is typically 0.3" x 0.2" although this varies slightly over the entire 5 square degree field of view. We achieve ~80 to 300 $\mu$Jy/bm image rms noise, which is dependent on the distance from the phase centre; typical values are ~90 $\mu$Jy/bm for the 8 hour observation with 48 MHz of bandwidth. Seventy percent of processed sources are detected, and from this we estimate that we should be able to image ~900 sources per LoTSS pointing. This equates to ~3 million sources in the northern sky, which LoTSS will entirely cover in the next several years. Future optimisation of the calibration strategy for efficient post-processing of LoTSS at high resolution (LoTSS-HR) makes this estimate a lower limit., Comment: Accepted to a special issue of A&A on sub-arcsecond imaging with LOFAR. 24 pages, 16 figures

Relatively little information is available about the Universe at ultra-low radio frequencies, i.e. below 50 MHz (ULF), although the ULF spectral window contains a wealth of unique diagnostics for studying galactic and extragalactic phenomena. Sub-arcsecond resolution imaging at these frequencies is extremely difficult, due to the long baselines (>1000 km) required and large ionospheric perturbations. We have conducted a pilot project to investigate the ULF performance and potential of the International LOFAR Telescope (ILT), a trans-European interferometric array with baselines up to ~2000 km and observing frequencies down to 10 MHz. We have successfully produced images with sub-arcsecond resolution for 6 radio sources at frequencies down to 30 MHz. This is more than an order of magnitude better resolution than pre-ILT observations at similar frequencies. The six targets that we have imaged (3C 196, 3C 225, 3C 273, 3C 295, 3C 298 and 3C 380) are bright radio sources with compact structures. By comparing our data of 3C 196 and 3C 273 with observations at higher frequencies, we investigate their spatially resolved radio spectral properties. Our success shows that at frequencies down to 30 MHz, sub-arcsecond imaging with the ILT is possible. Further analysis is needed to determine the feasibility of observations of fainter sources or sources with less compact emission., Comment: Accepted to a special issue of A&A on sub-arcsecond imaging with LOFAR. 11 pages, 8 figures, 3 tables

Since its discovery in 1963, 3C273 has become one of the most widely studied quasars with investigations spanning the electromagnetic spectrum. While much has been discovered about this historically notable source, its low-frequency emission is far less well understood. Observations in the MHz regime have traditionally lacked the resolution required to explore small-scale structures that are key to understanding the processes that result in the observed emission. In this paper we use the first sub-arcsecond images of 3C273 at MHz frequencies to investigate the morphology of the compact jet structures and the processes that result in the observed spectrum. Using the full complement of LOFAR's international stations, we produce $0.31 \times 0.21$ arcsec images of 3C273 at 150 MHz to determine the jet's kinetic power, place constraints on the bulk speed and inclination angle of the jets, and look for evidence of the elusive counter-jet at 150 MHz. Using ancillary data at GHz frequencies, we fit free-free absorption (FFA) and synchrotron self-absorption (SSA) models to determine their validity in explaining the observed spectra. The images presented display for the first time that robust, high-fidelity imaging of low-declination complex sources is now possible with the LOFAR international baselines. We show that the main small-scale structures of 3C273 match those seen at higher frequencies and that absorption is present in the observed emission. We determine the kinetic power of the jet to be in the range of $3.5 \times 10^{43}$ - $1.5 \times 10^{44}$ erg s$^{-1}$ which agrees with estimates made using higher frequency observations. We derive lower limits for the bulk speed and Lorentz factor of $\beta \gtrsim 0.55$ and $\Gamma \geq 1.2$ respectively. The counter-jet remains undetected at $150$ MHz, placing a limit on the peak brightness of $S_\mathrm{cj\_150} < 40$ mJy beam$^{-1}$., Comment: Accepted to a special issue of A&A on sub-arcsecond imaging with LOFAR. 13 pages, 10 figures, 2 tables

Large language models have led to remarkable progress on many NLP tasks, and researchers are turning to ever-larger text corpora to train them. Some of the largest corpora available are made by scraping significant portions of the internet, and are frequently introduced with only minimal documentation. In this work we provide some of the first documentation for the Colossal Clean Crawled Corpus (C4; Raffel et al., 2020), a dataset created by applying a set of filters to a single snapshot of Common Crawl. We begin by investigating where the data came from, and find a significant amount of text from unexpected sources like patents and US military websites. Then we explore the content of the text itself, and find machine-generated text (e.g., from machine translation systems) and evaluation examples from other benchmark NLP datasets. To understand the impact of the filters applied to create this dataset, we evaluate the text that was removed, and show that blocklist filtering disproportionately removes text from and about minority individuals. Finally, we conclude with some recommendations for how to created and document web-scale datasets from a scrape of the internet., Comment: EMNLP 2021 accepted paper camera ready version

In order to solve today's complex problems in the world of software development, technical knowledge is no longer enough. Previous studies investigating and identifying non-technical skills of software engineers show that creative skills also play an important role in tackling difficult problems. However, creativity is typically a very vague concept to which everyone gives their own interpretation. Also, there is little research that focuses specifically on creativity in the field of software engineering. To better understand the role of creativity in this field, we conducted four focus groups, inviting 33 experts from four nationally and internationally renowned companies in total. This resulted in 399 minutes of transcripts, further coded into 39 sub-themes grouped into seven categories: technical knowledge, communication, constraints, critical thinking, curiosity, creative state of mind, and creative techniques. This study identifies the added value of creativity, which creative techniques are used, how creativity can be recognized, the reasons for being creative, and what environment is needed to facilitate creative work. Our ultimate goal is to use these findings to instill and further encourage the creative urge among undergraduate students in higher education.

Peer assessment is a popular technique for a more fine-grained evaluation of individual students in group projects. Its effect on the evaluation is well studied. However, its effects on the learning abilities of students are often overlooked. In this paper, we explore self-evaluation, motivation, and study time of students in relation to peer assessment, as part of an ongoing project at our local Faculty of Engineering Technology. The aggregated measurements of two years so far show that: (1) students get much better at evaluating their own project on some, but not all, of the evaluation criteria after a peer assessment session, (2) students report in a follow-up survey that they are more motivated to work on their project, and (3) the relation between motivation and time spent on the project increases. These results suggest that peer grading could have positive long-term effects on the reflective, and therefore lifelong learning, skills of students. A better understanding of the evaluation criteria results in more accurate self and peer grades, emphasizing the importance of properly defining and communicating these criteria throughout the semester.

State-of-the-art models for multi-hop question answering typically augment large-scale language models like BERT with additional, intuitively useful capabilities such as named entity recognition, graph-based reasoning, and question decomposition. However, does their strong performance on popular multi-hop datasets really justify this added design complexity? Our results suggest that the answer may be no, because even our simple pipeline based on BERT, named Quark, performs surprisingly well. Specifically, on HotpotQA, Quark outperforms these models on both question answering and support identification (and achieves performance very close to a RoBERTa model). Our pipeline has three steps: 1) use BERT to identify potentially relevant sentences independently of each other; 2) feed the set of selected sentences as context into a standard BERT span prediction model to choose an answer; and 3) use the sentence selection model, now with the chosen answer, to produce supporting sentences. The strong performance of Quark resurfaces the importance of carefully exploring simple model designs before using popular benchmarks to justify the value of complex techniques.

We study the three-body scattering hypervolume $D$ of atoms whose scattering length $a$ is on the order of or smaller than the typical range $r_{\mathrm{vdW}}$ of the van der Waals attraction. We find that the real part of $D$ behaves universally in this weakly interacting regime ($|a|/r_{\mathrm{vdW}}\lesssim 1$) in the absence of trimer resonances. This universality originates from hard-spherelike collisions that dominate elastic three-body scattering. We use this result to make quantitative predictions for the thermodynamics and elementary excitations of an atomic Bose-Einstein condensate in the vicinity of a quantum tricritical point, including quantum droplets stabilized by effective three-body interactions., Comment: 10 pages, 8 figures

Important building blocks of software engineering concepts are without a doubt technical. During the last decade, research and practical interest for non-technicalities has grown, revealing the building blocks to be various skills and abilities beside pure technical knowledge. Multiple attempts to categorise these blocks have been made, but so far little international studies have been performed that identify skills by asking experts from both the industrial and academic world: which abilities are needed for a developer to excel in the software engineering industry? To answer this question, we performed a Delphi study, inviting 36 experts from 11 different countries world-wide, affiliated with 21 internationally renowned institutions. This study presents the 55 identified and ranked skills as classified in four major areas: communicative skills (empathy, actively listening, etc.), collaborative skills (sharing responsibility, learning from each other, etc.), problem solving skills (verifying assumptions, solution-oriented thinking, etc.), and personal skills (curiosity, being open to ideas, etc.), of which a comparison has been made between opinions of technical experts, business experts, and academics. We hope this work inspires educators and practitioners to adjust their training programs, mitigating the gap between the industry and the academic world., Comment: 9 pages. This is a pre-print of a paper that has been accepted for publication in SIGCSE, 2020

Higher education provides a solid theoretical and practical, but mostly technical, background for the aspiring software developer. Research, however, has shown that graduates still fall short of the expectations of industry. These deficiencies are not limited to technical shortcomings. The ever changing landscape of 'lean' enterprise software development requires engineers to be equipped with abilities beyond the technical. How can higher education help students become great software developers in this context? As a first step towards answering this question, we present the results of a systematic literature review, focusing on noncognitive abilities, better known as 'soft skills'. Our results identify self-reflection, conflict resolution, communication, and teamwork as the top four taught skills. Internships and capstone projects require more attention as a teaching aspect to facilitate the learning of multiple skills, including creativity. Interdisciplinary teaching and group composition are other important factors that influence learning. By providing novel insights on relationships between noncognitive abilities and teaching aspects, this work contributes to the continuous improvement of software engineering curricula. These findings may also serve as a springboard for further investigation of certain undervalued skills., Comment: 13 pages. Published in Proceedings of the 47th SEFI Conference 2019; 2019. (SEFI - European Society for Engineering Education; Brussels)

AI has achieved remarkable mastery over games such as Chess, Go, and Poker, and even Jeopardy, but the rich variety of standardized exams has remained a landmark challenge. Even in 2016, the best AI system achieved merely 59.3% on an 8th Grade science exam challenge. This paper reports unprecedented success on the Grade 8 New York Regents Science Exam, where for the first time a system scores more than 90% on the exam's non-diagram, multiple choice (NDMC) questions. In addition, our Aristo system, building upon the success of recent language models, exceeded 83% on the corresponding Grade 12 Science Exam NDMC questions. The results, on unseen test questions, are robust across different test years and different variations of this kind of test. They demonstrate that modern NLP methods can result in mastery on this task. While not a full solution to general question-answering (the questions are multiple choice, and the domain is restricted to 8th Grade science), it represents a significant milestone for the field., Comment: AI Magazine 41 (4) Winter 2020. New analysis sections added

We describe a deployed scalable system for organizing published scientific literature into a heterogeneous graph to facilitate algorithmic manipulation and discovery. The resulting literature graph consists of more than 280M nodes, representing papers, authors, entities and various interactions between them (e.g., authorships, citations, entity mentions). We reduce literature graph construction into familiar NLP tasks (e.g., entity extraction and linking), point out research challenges due to differences from standard formulations of these tasks, and report empirical results for each task. The methods described in this paper are used to enable semantic features in www.semanticscholar.org, Comment: To appear in NAACL 2018 industry track

Results of optical frequency transfer over a carrier-grade dense-wavelength-division-multiplexing (DWDM) optical fibre network are presented. The relation between soil temperature changes on a buried optical fibre and frequency changes of an optical carrier through the fibre is modelled. Soil temperatures, measured at various depths by the Royal Netherlands Meteorology Institute (KNMI) are compared with observed frequency variations through this model. A comparison of a nine-day record of optical frequency measurements through the 2 x 298 km fibre link with soil temperature data shows qualitative agreement. A soil temperature model is used to predict the link stability over longer periods (days-months-years). We show that one-way optical frequency dissemination is sufficiently stable to distribute and compare e.g. rubidium frequency standards over standard DWDM optical fibre networks., Comment: 13 pages, 13 figures, submitted

We consider a simple model of particle transport on the line defined by a dynamical map F satisfying F(x+1) = 1 + F(x) for all x in R and F(x) = ax + b for |x| < 0.5. Its two parameters a (`slope') and b (`bias') are respectively symmetric and antisymmetric under reflection x -> R(x) = -x. Restricting ourselves to the chaotic regime |a| > 1 and therein mainly to the part a>1 we study not only the `diffusion coefficient' D(a,b), but also the `current' J(a,b). An important tool for such a study are the exact expressions for J and D as obtained recently by one of the authors. These expressions allow for a quite efficient numerical implementation, which is important, because the functions encountered typically have a fractal character. The main results are presented in several plots of these functions J(a,b) and D(a,b) and in an over-all `chart' displaying, in the parameter plane, all possibly relevant information on the system including, e.g., the dynamical phase diagram as well as invariants such as the values of topological invariants (kneading numbers) which, according to the formulas, determine the singularity structure of J and D. Our most significant findings are: 1) `Nonlinear Response': The parameter dependence of these transport properties is, throughout the `ergodic' part of the parameter plane (i.e. outside the infinitely many Arnol'd tongues) fractally nonlinear. 2) `Negative Response': Inside certain regions with an apparently fractal boundary the current J and the bias b have opposite signs., Comment: corrected typos and minor reformulations; 28 pages (revtex) with 7 figures (postscript); accepted for publication in JSP

Our primary results in Jasper Ridge Biological Preserve indicate that high spectral resolution Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) data may provide a substantial advantage in vegetation, based on the chlorophyll red edge feature from 700-780 nm. The chlorophyll red edge was detected for green vegetation cover as low as 4.8 percent. The objective of our studies in Mono Lake area is to continue the experiments performed in Jasper Ridge and to examine the persistence of red edge feature of trace quantities of green vegetation for different plant communities with non-uniform soil backgrounds.

The beam-foil technique has been used to study the 2S(2)S-2p(2)P(0) doublet in S XIV. The results confirm the doublet splitting measured aboard Skylab during solar flare events. In addition, the oscillator strengths for the resonance transitions comprising this doublet have been measured and found to agree well with recent relativistic f-value calculations.