Search

Social media enables activists to directly communicate with the public and provides a space for movement leaders, participants, bystanders, and opponents to collectively construct and contest narratives. Focusing on Twitter messages from social movements surrounding three issues in 2018-2019 (guns, immigration, and LGBTQ rights), we create a codebook, annotated dataset, and computational models to detect diagnostic (problem identification and attribution), prognostic (proposed solutions and tactics), and motivational (calls to action) framing strategies. We conduct an in-depth unsupervised linguistic analysis of each framing strategy, and uncover cross-movement similarities in associations between framing and linguistic features such as pronouns and deontic modal verbs. Finally, we compare framing strategies across issues and other social, cultural, and interactional contexts. For example, we show that diagnostic framing is more common in replies than original broadcast posts, and that social movement organizations focus much more on prognostic and motivational framing than journalists and ordinary citizens., Comment: Published in ICWSM Special Issue of the Journal of Quantitative Description: Digital Media

In recent years, there has been a surge in malware attacks across critical infrastructures, requiring further research and development of appropriate response and remediation strategies in malware detection and classification. Several works have used machine learning models for malware classification into categories, and deep neural networks have shown promising results. However, these models have shown its vulnerabilities against intentionally crafted adversarial attacks, which yields misclassification of a malicious file. Our paper explores such adversarial vulnerabilities of neural network based malware classification system in the dynamic and online analysis environments. To evaluate our approach, we trained Feed Forward Neural Networks (FFNN) to classify malware categories based on features obtained from dynamic and online analysis environments. We use the state-of-the-art method, SHapley Additive exPlanations (SHAP), for the feature attribution for malware classification, to inform the adversarial attackers about the features with significant importance on classification decision. Using the explainability-informed features, we perform targeted misclassification adversarial white-box evasion attacks using the Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks against the trained classifier. Our results demonstrated high evasion rate for some instances of attacks, showing a clear vulnerability of a malware classifier for such attacks. We offer recommendations for a balanced approach and a benchmark for much-needed future research into evasion attacks against malware classifiers, and develop more robust and trustworthy solutions.

In recent years, there has been a significant surge in malware attacks, necessitating more advanced preventive measures and remedial strategies. While several successful AI-based malware classification approaches exist categorized into static, dynamic, or online analysis, most successful AI models lack easily interpretable decisions and explanations for their processes. Our paper aims to delve into explainable malware classification across various execution environments (such as dynamic and online), thoroughly analyzing their respective strengths, weaknesses, and commonalities. To evaluate our approach, we train Feed Forward Neural Networks (FFNN) and Convolutional Neural Networks (CNN) to classify malware based on features obtained from dynamic and online analysis environments. The feature attribution for malware classification is performed by explainability tools, SHAP, LIME and Permutation Importance. We perform a detailed evaluation of the calculated global and local explanations from the experiments, discuss limitations and, ultimately, offer recommendations for achieving a balanced approach.

Most events in the world receive at most brief coverage by the news media. Occasionally, however, an event will trigger a media storm, with voluminous and widespread coverage lasting for weeks instead of days. In this work, we develop and apply a pairwise article similarity model, allowing us to identify story clusters in corpora covering local and national online news, and thereby create a comprehensive corpus of media storms over a nearly two year period. Using this corpus, we investigate media storms at a new level of granularity, allowing us to validate claims about storm evolution and topical distribution, and provide empirical support for previously hypothesized patterns of influence of storms on media coverage and intermedia agenda setting., Comment: Findings of EMNLP 2023; 16 pages; 12 figures; 4 tables

The versatility of Large Language Models (LLMs) on natural language understanding tasks has made them popular for research in social sciences. To properly understand the properties and innate personas of LLMs, researchers have performed studies that involve using prompts in the form of questions that ask LLMs about particular opinions. In this study, we take a cautionary step back and examine whether the current format of prompting LLMs elicits responses in a consistent and robust manner. We first construct a dataset that contains 693 questions encompassing 39 different instruments of persona measurement on 115 persona axes. Additionally, we design a set of prompts containing minor variations and examine LLMs' capabilities to generate answers, as well as prompt variations to examine their consistency with respect to content-level variations such as switching the order of response options or negating the statement. Our experiments on 17 different LLMs reveal that even simple perturbations significantly downgrade a model's question-answering ability, and that most LLMs have low negation consistency. Our results suggest that the currently widespread practice of prompting is insufficient to accurately and reliably capture model perceptions, and we therefore discuss potential alternatives to improve these issues., Comment: Camera-ready version for NAACL 2024. First two authors contributed equally

Measuring semantic change has thus far remained a task where methods using contextual embeddings have struggled to improve upon simpler techniques relying only on static word vectors. Moreover, many of the previously proposed approaches suffer from downsides related to scalability and ease of interpretation. We present a simplified approach to measuring semantic change using contextual embeddings, relying only on the most probable substitutes for masked terms. Not only is this approach directly interpretable, it is also far more efficient in terms of storage, achieves superior average performance across the most frequently cited datasets for this task, and allows for more nuanced investigation of change than is possible with static word vectors.

Efforts are underway to magnetically confine electron--positron pair plasmas to study their unique behavior, which is characterized by significant changes in plasma time and length scales, supported waves, and unstable modes. However, use of conventional plasma diagnostics presents challenges with these low-density and annihilating matter-antimatter plasma. To address this problem, we propose to develop techniques based on the distinct emission provided by annihilation. This emission exhibits two spatial correlations: the distance attenuation of isotropic sources and the back-to-back propagation of momentum-preserving 2-$\gamma$ annihilation. We present the results of our analysis of the $\gamma$ emission rate and the spatial profile of the annihilation in a magnetized pair plasma from direct pair collisions, from the formation and decay of positronium, as well as from transport processes. In order to demonstrate the effectiveness of annihilation-based techniques, we tested them on annular $\gamma$ emission profiles produced by a $\beta^+$ radioisotope on a rotating turntable. Direct and positronium-mediated annihilation result in overlapping volumetric $\gamma$ sources, and the 2-$\gamma$ emission from these volumetric sources can be tomographically reconstructed from coincident counts in multiple detectors. Transport processes result in localized annihilation where field lines intersect walls, limiters, or internal magnets. These localized sources can be identified by the fractional $\gamma$ counts on spatially distributed detectors., Comment: 21 pages, 11 figures, 2 tables, contribution to the 13th International Workshop on Non-Neutral Plasmas

We present PatentsView-Evaluation, a Python package that enables researchers to evaluate the performance of inventor name disambiguation systems such as PatentsView.org. The package includes benchmark datasets and evaluation tools, and aims to advance research on inventor name disambiguation by providing access to high-quality evaluation data and improving evaluation standards., Comment: 3 pages, 2 figures

Aging GONG second generation cameras (Silicon Mountain Design(TM) cameras) were planned to be replaced after their long service of more than a decade. This prompted a market-wide search for a potential replacement detector to meet the GONG science requirements. This report provides some history of the search process, a comparison between CMOS and CCD type sensors and then a quantitative evaluation of potential candidates to arrive at final selection. Further, a feasibility study of the selected sensor for adaptation to GONG optical system was done and sensor characteristics were independently verified in the laboratory. This technical report gives description of these studies and tests., Comment: 13 pages, 12 figures

Cosine similarity of contextual embeddings is used in many NLP tasks (e.g., QA, IR, MT) and metrics (e.g., BERTScore). Here, we uncover systematic ways in which word similarities estimated by cosine over BERT embeddings are understated and trace this effect to training data frequency. We find that relative to human judgements, cosine similarity underestimates the similarity of frequent words with other instances of the same word or other words across contexts, even after controlling for polysemy and other factors. We conjecture that this underestimation of similarity for high frequency words is due to differences in the representational geometry of high and low frequency words and provide a formal argument for the two-dimensional case., Comment: Camera Ready for ACL 2022 (Main Conference)

Off-the-shelf models are widely used by computational social science researchers to measure properties of text, such as sentiment. However, without access to source data it is difficult to account for domain shift, which represents a threat to validity. Here, we treat domain adaptation as a modular process that involves separate model producers and model consumers, and show how they can independently cooperate to facilitate more accurate measurements of text. We introduce two lightweight techniques for this scenario, and demonstrate that they reliably increase out-of-domain accuracy on four multi-domain text classification datasets when used with linear and contextual embedding models. We conclude with recommendations for model producers and consumers, and release models and replication code to accompany this paper., Comment: Findings of ACL (2022)

Language models increasingly rely on massive web dumps for diverse text data. However, these sources are rife with undesirable content. As such, resources like Wikipedia, books, and newswire often serve as anchors for automatically selecting web text most suitable for language modeling, a process typically referred to as quality filtering. Using a new dataset of U.S. high school newspaper articles -- written by students from across the country -- we investigate whose language is preferred by the quality filter used for GPT-3. We find that newspapers from larger schools, located in wealthier, educated, and urban ZIP codes are more likely to be classified as high quality. We then demonstrate that the filter's measurement of quality is unaligned with other sensible metrics, such as factuality or literary acclaim. We argue that privileging any corpus as high quality entails a language ideology, and more care is needed to construct training corpora for language models, with better transparency and justification for the inclusion or exclusion of various texts.

Feedback is ubiquitous in both biological and engineered control systems. In biology, in addition to typical feedback between plant and controller, we observe feedback pathways within control systems, which we call internal feedback pathways (IFPs), that are often very complex. IFPs are most familiar in neural systems, our primary motivation, but they appear everywhere from bacterial signal transduction to the human immune system. In this paper, we describe these very different motivating examples and introduce the concepts necessary to explain their complex IFPs, particularly the severe speed-accuracy tradeoffs that constrain the hardware in biology. We also sketch some minimal theory for extremely simplified toy models that nevertheless highlight the importance of diversity-enabled sweet spots (DESS) in mitigating the impact of hardware tradeoffs. For more realistic models, standard modern and robust control theory can give some insights into previously cryptic IFPs, and the new System Level Synthesis theory expands this substantially. These additional theories explaining IFPs will be explored in more detail in several companion papers., Comment: This paper is a companion to arXiv:2109.11752 and arXiv:2109.11757. v3: updated title + author metadata to match PDF

Research in NLP is often supported by experimental results, and improved reporting of such results can lead to better understanding and more reproducible science. In this paper we analyze three statistical estimators for expected validation performance, a tool used for reporting performance (e.g., accuracy) as a function of computational budget (e.g., number of hyperparameter tuning experiments). Where previous work analyzing such estimators focused on the bias, we also examine the variance and mean squared error (MSE). In both synthetic and realistic scenarios, we evaluate three estimators and find the unbiased estimator has the highest variance, and the estimator with the smallest variance has the largest bias; the estimator with the smallest MSE strikes a balance between bias and variance, displaying a classic bias-variance tradeoff. We use expected validation performance to compare between different models, and analyze how frequently each estimator leads to drawing incorrect conclusions about which of two models performs best. We find that the two biased estimators lead to the fewest incorrect conclusions, which hints at the importance of minimizing variance and MSE.

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature., Comment: Authored by the Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Report page with citation guidelines: https://crfm.stanford.edu/report.html

Machine learning currently exerts an outsized influence on the world, increasingly affecting institutional practices and impacted communities. It is therefore critical that we question vague conceptions of the field as value-neutral or universally beneficial, and investigate what specific values the field is advancing. In this paper, we first introduce a method and annotation scheme for studying the values encoded in documents such as research papers. Applying the scheme, we analyze 100 highly cited machine learning papers published at premier machine learning conferences, ICML and NeurIPS. We annotate key features of papers which reveal their values: their justification for their choice of project, which attributes of their project they uplift, their consideration of potential negative consequences, and their institutional affiliations and funding sources. We find that few of the papers justify how their project connects to a societal need (15\%) and far fewer discuss negative potential (1\%). Through line-by-line content analysis, we identify 59 values that are uplifted in ML research, and, of these, we find that the papers most frequently justify and assess themselves based on Performance, Generalization, Quantitative evidence, Efficiency, Building on past work, and Novelty. We present extensive textual evidence and identify key themes in the definitions and operationalization of these values. Notably, we find systematic textual evidence that these top values are being defined and applied with assumptions and implications generally supporting the centralization of power.Finally, we find increasingly close ties between these highly cited papers and tech companies and elite universities., Comment: Data and code available at https://github.com/wagnew3/The-Values-Encoded-in-Machine-Learning-Research. arXiv admin note: text overlap with arXiv:2206.04179

The creation of an electron space charge in a dipole magnetic trap and the subsequent injection of positrons has been experimentally demonstrated. Positrons (5eV) were magnetically guided from their source and injected into the trapping field generated by a permanent magnet (0.6T at the poles) using a cross field E $\times$ B drift, requiring tailored electrostatic and magnetic fields. The electron cloud is created by thermionic emission from a tungsten filament. The maximum space charge potential of the electron cloud reaches -42V, which is consistent with an average electron density of ($4 \pm 2$) $\times 10^{12}$ $\text{m}^{-3}$ and a Debye length of ($2 \pm 1$) $\text{cm}$. We demonstrate that the presence of this space potential does not hamper efficient positron injection. Understanding the effects of the negative space charge on the injection and confinement of positrons represents an important intermediate step towards the production of a confined electron-positron pair plasma.

Citing opinions is a powerful yet understudied strategy in argumentation. For example, an environmental activist might say, "Leading scientists agree that global warming is a serious concern," framing a clause which affirms their own stance ("that global warming is serious") as an opinion endorsed ("[scientists] agree") by a reputable source ("leading"). In contrast, a global warming denier might frame the same clause as the opinion of an untrustworthy source with a predicate connoting doubt: "Mistaken scientists claim [...]." Our work studies opinion-framing in the global warming (GW) debate, an increasingly partisan issue that has received little attention in NLP. We introduce Global Warming Stance Dataset (GWSD), a dataset of stance-labeled GW sentences, and train a BERT classifier to study novel aspects of argumentation in how different sides of a debate represent their own and each other's opinions. From 56K news articles, we find that similar linguistic devices for self-affirming and opponent-doubting discourse are used across GW-accepting and skeptic media, though GW-skeptical media shows more opponent-doubt. We also find that authors often characterize sources as hypocritical, by ascribing opinions expressing the author's own view to source entities known to publicly endorse the opposing view. We release our stance dataset, model, and lexicons of framing devices for future work on opinion-framing and the automatic detection of GW stance., Comment: 9 pages, 6 figures

We consider the problem of using observational data to estimate the causal effects of linguistic properties. For example, does writing a complaint politely lead to a faster response time? How much will a positive product review increase sales? This paper addresses two technical challenges related to the problem before developing a practical method. First, we formalize the causal quantity of interest as the effect of a writer's intent, and establish the assumptions necessary to identify this from observational data. Second, in practice, we only have access to noisy proxies for the linguistic properties of interest -- e.g., predictions from classifiers and lexicons. We propose an estimator for this setting and prove that its bias is bounded when we perform an adjustment for the text. Based on these results, we introduce TextCause, an algorithm for estimating causal effects of linguistic properties. The method leverages (1) distant supervision to improve the quality of noisy proxies, and (2) a pre-trained language model (BERT) to adjust for the text. We show that the proposed method outperforms related approaches when estimating the effect of Amazon review sentiment on semi-simulated sales figures. Finally, we present an applied case study investigating the effects of complaint politeness on bureaucratic response times., Comment: To appear at NAACL 2021 (Annual Conference of the North American Chapter of the Association for Computational Linguistics)

Despite its importance to experimental design, statistical power (the probability that, given a real effect, an experiment will reject the null hypothesis) has largely been ignored by the NLP community. Underpowered experiments make it more difficult to discern the difference between statistical noise and meaningful model improvements, and increase the chances of exaggerated findings. By meta-analyzing a set of existing NLP papers and datasets, we characterize typical power for a variety of settings and conclude that underpowered experiments are common in the NLP literature. In particular, for several tasks in the popular GLUE benchmark, small test sets mean that most attempted comparisons to state of the art models will not be adequately powered. Similarly, based on reasonable assumptions, we find that the most typical experimental design for human rating studies will be underpowered to detect small model differences, of the sort that are frequently studied. For machine translation, we find that typical test sets of 2000 sentences have approximately 75% power to detect differences of 1 BLEU point. To improve the situation going forward, we give an overview of best practices for power analysis in NLP and release a series of notebooks to assist with future power analyses., Comment: To appear at EMNLP 2020

Recent work on fairness in machine learning has primarily emphasized how to define, quantify, and encourage "fair" outcomes. Less attention has been paid, however, to the ethical foundations which underlie such efforts. Among the ethical perspectives that should be taken into consideration is consequentialism, the position that, roughly speaking, outcomes are all that matter. Although consequentialism is not free from difficulties, and although it does not necessarily provide a tractable way of choosing actions (because of the combined problems of uncertainty, subjectivity, and aggregation), it nevertheless provides a powerful foundation from which to critique the existing literature on machine learning fairness. Moreover, it brings to the fore some of the tradeoffs involved, including the problem of who counts, the pros and cons of using a policy, and the relative value of the distant future. In this paper we provide a consequentialist critique of common definitions of fairness within machine learning, as well as a machine learning perspective on consequentialism. We conclude with a broader discussion of the issues of learning and randomization, which have important implications for the ethics of automated decision making systems., Comment: Updating to published version

Current higher education practices do not sufficiently address the needs of international adult learners. With higher education's emphasis on pedagogical assumptions, adult learners are both isolated and oppressed by higher education. This article focuses on the following: assumptions of pedagogy versus andragogy, oppression of adult learners through banking theory and hegemony, and budget constraints that create an inhospitable environment for international adult learners. The author offers suggestions to address current issues in higher education using experiential learning, an andragogical model, intentional professional development for professors, and deliberate classroom experiences. [For the full proceedings, see ED597456.]

Research in natural language processing proceeds, in part, by demonstrating that new models achieve superior performance (e.g., accuracy) on held-out test data, compared to previous results. In this paper, we demonstrate that test-set performance scores alone are insufficient for drawing accurate conclusions about which model performs best. We argue for reporting additional details, especially performance on validation data obtained during model development. We present a novel technique for doing so: expected validation performance of the best-found model as a function of computation budget (i.e., the number of hyperparameter search trials or the overall training time). Using our approach, we find multiple recent model comparisons where authors would have reached a different conclusion if they had used more (or less) computation. Our approach also allows us to estimate the amount of computation required to obtain a given accuracy; applying it to several recently published results yields massive variation across papers, from hours to weeks. We conclude with a set of best practices for reporting experimental results which allow for robust future comparisons, and provide code to allow researchers to use our technique.

We introduce VAMPIRE, a lightweight pretraining framework for effective text classification when data and computing resources are limited. We pretrain a unigram document model as a variational autoencoder on in-domain, unlabeled data and use its internal states as features in a downstream classifier. Empirically, we show the relative strength of VAMPIRE against computationally expensive contextual embeddings and other popular semi-supervised baselines under low resource settings. We also find that fine-tuning to in-domain data is crucial to achieving decent performance from contextual embeddings when working with limited supervision. We accompany this paper with code to pretrain and use VAMPIRE embeddings in downstream tasks., Comment: ACL 2019

Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, concerns have been raised about the calibration, robustness, and interpretability of these models. In this paper we propose a simple way to modify any conventional deep architecture to automatically provide more transparent explanations for classification decisions, as well as an intuitive notion of the credibility of each prediction. Specifically, we draw on ideas from nonparametric kernel regression, and propose to predict labels based on a weighted sum of training instances, where the weights are determined by distance in a learned instance-embedding space. Working within the framework of conformal methods, we propose a new measure of nonconformity suggested by our model, and experimentally validate the accompanying theoretical expectations, demonstrating improved transparency, controlled error rates, and robustness to out-of-domain data, without compromising on accuracy or calibration., Comment: 13 pages, 8 figures, 5 tables, added DOI and updated to meet ACM formatting requirements, In Proceedings of FAT* (2019)

The Mochi device is a new pulsed power plasma experiment designed to produce long, collimated, stable, magnetized plasma jets when set up in the LabJet configuration. The LabJet configuration aims to simulate an astrophysical jet in the laboratory by mimicking an accretion disk threaded by a poloidal magnetic field with concentric planar electrodes in front of a solenoidal coil. The unique setup consists of three electrodes, each with azimuthally symmetric gas slits. Two of the electrodes are biased independently with respect to the third electrode to control the radial electric field profile across the poloidal bias magnetic field. This design approximates a shear azimuthal rotation profile in an accretion disk. The azimuthally symmetric gas slits provide a continuously symmetric mass source at the footpoint of the plasma jet, so any azimuthal rotation of the plasma jet is not hindered by a discrete number of gas holes. The initial set of diagnostics consists of current Rogowski coils, voltage probes, magnetic field probe arrays, an interferometer and ion Doppler spectroscopy, supplemented by a fast ion gauge and a retarding grid energy analyzer. The measured parameters of the first plasmas are 1E22 m-3, 0.4 T, 5-25 eV, with velocities of 20-80 km/s. The combination of a controllable electric field profile, a flared poloidal magnetic field, and azimuthally symmetric mass sources in the experiment successfully produces short-lived (10 {\mu}s, >5 Alfv\'en times) collimated magnetic jets with ~10:1 aspect-ratio and long-lived (100 {\mu}s, >40 Alfv\'en times) flow-stabilized, collimated, magnetic jets with ~30:1 aspect-ratio., Comment: Accepted for publication in The Astrophysical Journal Supplement

Most real-world document collections involve various types of metadata, such as author, source, and date, and yet the most commonly-used approaches to modeling text corpora ignore this information. While specialized models have been developed for particular applications, few are widely used in practice, as customization typically requires derivation of a custom inference algorithm. In this paper, we build on recent advances in variational inference methods and propose a general neural framework, based on topic models, to enable flexible incorporation of metadata and allow for rapid exploration of alternative models. Our approach achieves strong performance, with a manageable tradeoff between perplexity, coherence, and sparsity. Finally, we demonstrate the potential of our framework through an exploration of a corpus of articles about US immigration., Comment: 13 pages, 3 figures, 6 tables; updating to version published at ACL 2018

Understanding how ideas relate to each other is a fundamental question in many domains, ranging from intellectual history to public communication. Because ideas are naturally embedded in texts, we propose the first framework to systematically characterize the relations between ideas based on their occurrence in a corpus of documents, independent of how these ideas are represented. Combining two statistics --- cooccurrence within documents and prevalence correlation over time --- our approach reveals a number of different ways in which ideas can cooperate and compete. For instance, two ideas can closely track each other's prevalence over time, and yet rarely cooccur, almost like a "cold war" scenario. We observe that pairwise cooccurrence and prevalence correlation exhibit different distributions. We further demonstrate that our approach is able to uncover intriguing relations between ideas through in-depth case studies on news articles and research papers., Comment: 11 pages, 9 figures, to appear in Proceedings of ACL 2017, code and data available at https://chenhaot.com/pages/idea-relations.html (fixed a typo)

The SUNRISE balloon-borne solar observatory, consisting of a 1~m aperture telescope that provided a stabilized image to a UV filter imager and an imaging vector polarimeter, carried out its second science flight in June 2013. It provided observations of parts of active regions at high spatial resolution, including the first high-resolution images in the Mg~{\sc ii}~k line. The obtained data are of very high quality, with the best UV images reaching the diffraction limit of the telescope at 3000~\AA\ after Multi-Frame Blind Deconvolution reconstruction accounting for phase-diversity information. Here a brief update is given of the instruments and the data reduction techniques, which includes an inversion of the polarimetric data. Mainly those aspects that evolved compared with the first flight are described. A tabular overview of the observations is given. In addition, an example time series of a part of the emerging active region NOAA AR~11768 observed relatively close to disk centre is described and discussed in some detail. The observations cover the pores in the trailing polarity of the active region, as well as the polarity inversion line where flux emergence was ongoing and a small flare-like brightening occurred in the course of the time series. The pores are found to contain magnetic field strengths ranging up to 2500~G and, while large pores are clearly darker and cooler than the quiet Sun in all layers of the photosphere, the temperature and brightness of small pores approach or even exceed those of the quiet Sun in the upper photosphere., Comment: Accepted for publication in The Astrophysical Journal

We evaluate a tracking program in a large urban district where schools with at least one gifted fourth grader create a separate "gifted/high achiever" classroom. Most seats are filled by non-gifted high achievers, ranked by previous-year test scores. We study the program's effects on the high achievers using (1) a rank-based regression discontinuity design, and (2) a between-school/cohort analysis. We find significant effects that are concentrated among black and Hispanic participants. Minorities gain 0.5 standard deviation units in fourth-grade reading and math scores, with persistent gains through sixth grade. We find no evidence of negative or positive spillovers on non-participants. Appendices are included. [This paper was published in "American Economic Review" v106 n10 p2783-2816 2016.]

The accountability focus in higher education historically has been directed at the institutional level. The purpose of this comparative case study, however, was to look at state-level accountability. The research focused on the processes used in Tennessee, Minnesota, and South Dakota. Interviews were conducted with higher education administrators from state governing or coordinating boards, governors' policy advisers, and representatives of the legislative branch to compare viewpoints from those public policy domains. Burke's (2005) Accountability Triangle was applied as an analytical framework to better understand how higher education systems in these three states have balanced state priorities, academic concerns, and market forces. Interview protocol is appended. (Contains 4 tables and 1 figure.)

Based on investigation and analysis of the What Works Clearinghouse (WWC) review process and reports, this panel concluded that: (1) WWC procedures and processes for identifying and extracting information from intervention studies are generally well documented and follow reasonable standards and practices for systematic reviews; and (2) WWC Intervention and Topic Area Reports provide succinct and meaningful summaries of the evidence on the effectiveness of specific education interventions. Support for these conclusions is detailed in the remainder of the report, recommendations for the continued enhancement and improvement of WWC procedures include a comprehensive review of the full range of WWC activities and procedures, with a timeframe to allow a complete consideration of a number of issues not fully evaluated in this report. Two appendixes are included: (1) Panel Charge; and (2) References. (Contains 5 footnotes.)

The first science flight of the balloon-borne \Sunrise telescope took place in June 2009 from ESRANGE (near Kiruna/Sweden) to Somerset Island in northern Canada. We describe the scientific aims and mission concept of the project and give an overview and a description of the various hardware components: the 1-m main telescope with its postfocus science instruments (the UV filter imager SuFI and the imaging vector magnetograph IMaX) and support instruments (image stabilizing and light distribution system ISLiD and correlating wavefront sensor CWS), the optomechanical support structure and the instrument mounting concept, the gondola structure and the power, pointing, and telemetry systems, and the general electronics architecture. We also explain the optimization of the structural and thermal design of the complete payload. The preparations for the science flight are described, including AIV and ground calibration of the instruments. The course of events during the science flight is outlined, up to the recovery activities. Finally, the in-flight performance of the instrumentation is briefly summarized., Comment: 35 pages, 17 figures