Search

It is often asserted that to control for the effects of confounders, one should include the confounding variables of concern in a statistical model as a covariate. Conversely, it is also asserted that control can only be concluded by design, where the results from an analysis can only be interpreted as evidence of an effect because the design controlled for the cause. To suggest otherwise is said to be a fallacy of cum hoc ergo propter hoc. Obviously, these two assertions create a conundrum: How can the effect of confounder be controlled for with analysis instead of by design without committing cum hoc ergo propter hoc? The present manuscript answers this conundrum., Comment: 30 pages, 6 Figures, r code

The problem known as multicolinearity has long been recognized to fundamentally and negatively influence multiple regression. This paper does not intend to either propose a numerical assessment of the degree to which this problem exists within any data set or a solution to the problem itself. Rather, it is our intent to illustrate the potentially serious ramifications multicolinearity has on the traditional development of the multiple linear regression (MLR) model and its associated statistics using established equations, Venn diagrams, and real data., Comment: 18 pages, 3 figures, r code

Microstructural evolution under tribological load is known to change the friction and wear response of the entire tribological system. It is however not yet fully understood, how these changes take place on an elementary, mechanistic level. Revealing the temporal sequence of deformation mechanisms under a tribological load by experiments is scarce. The discrete strain release in the case of deformation twinning allows the identification of their temporal sequence. Therefore, a medium stacking fault energy material, namely CoCrNi, was investigated due to the significant contribution of deformation twinning to the overall deformation at room temperature. The investigated grain showed three activated twin systems. The evolution sequence of these is discussed critically based on several twin intersection mechanisms. The experimental analysis decodes the temporal sequence of the active twinning systems of highest resolved shear stresses for the first time. This approach gives experimental indications for the stress field under a tribological load.

Graph representations of programs are commonly a central element of machine learning for code research. We introduce an open source Python library python_graphs that applies static analysis to construct graph representations of Python programs suitable for training machine learning models. Our library admits the construction of control-flow graphs, data-flow graphs, and composite ``program graphs'' that combine control-flow, data-flow, syntactic, and lexical information about a program. We present the capabilities and limitations of the library, perform a case study applying the library to millions of competitive programming submissions, and showcase the library's utility for machine learning research., Comment: 21 pages, 14 figures

Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are probabilistic models, and may be expressed in the language of graphical models with random variables whose values are complex data types such as strings. Cases with control flow and dynamic structure require techniques from probabilistic programming, which allow implementing disparate model structures and inference strategies in a unified language. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use. We refer to the resulting programs as language model cascades., Comment: Presented as spotlight at the Beyond Bases workshop at ICML 2022 (https://beyond-bayes.github.io)

The execution behavior of a program often depends on external resources, such as program inputs or file contents, and so cannot be run in isolation. Nevertheless, software developers benefit from fast iteration loops where automated tools identify errors as early as possible, even before programs can be compiled and run. This presents an interesting machine learning challenge: can we predict runtime errors in a "static" setting, where program execution is not possible? Here, we introduce a real-world dataset and task for predicting runtime errors, which we show is difficult for generic models like Transformers. We approach this task by developing an interpreter-inspired architecture with an inductive bias towards mimicking program executions, which models exception handling and "learns to execute" descriptions of the contents of external resources. Surprisingly, we show that the model can also predict the location of the error, despite being trained only on labels indicating the presence/absence and kind of error. In total, we present a practical and difficult-yet-approachable challenge problem related to learning program execution and we demonstrate promising new capabilities of interpreter-inspired machine learning models for code., Comment: 20 pages, 7 figures

Large pre-trained language models perform remarkably well on tasks that can be done "in one pass", such as generating realistic text or synthesizing computer programs. However, they struggle with tasks that require unbounded multi-step computation, such as adding integers or executing programs. Surprisingly, we find that these same models are able to perform complex multi-step computations -- even in the few-shot regime -- when asked to perform the operation "step by step", showing the results of intermediate computations. In particular, we train transformers to perform multi-step computations by asking them to emit intermediate computation steps into a "scratchpad". On a series of increasingly complex tasks ranging from long addition to the execution of arbitrary programs, we show that scratchpads dramatically improve the ability of language models to perform multi-step computations.

Graph neural networks (GNNs) have emerged as a powerful tool for learning software engineering tasks including code completion, bug finding, and program repair. They benefit from leveraging program structure like control flow graphs, but they are not well-suited to tasks like program execution that require far more sequential reasoning steps than number of GNN propagation steps. Recurrent neural networks (RNNs), on the other hand, are well-suited to long sequential chains of reasoning, but they do not naturally incorporate program structure and generally perform worse on the above tasks. Our aim is to achieve the best of both worlds, and we do so by introducing a novel GNN architecture, the Instruction Pointer Attention Graph Neural Networks (IPA-GNN), which achieves improved systematic generalization on the task of learning to execute programs using control flow graphs. The model arises by considering RNNs operating on program traces with branch decisions as latent variables. The IPA-GNN can be seen either as a continuous relaxation of the RNN model or as a GNN variant more tailored to execution. To test the models, we propose evaluating systematic generalization on learning to execute using control flow graphs, which tests sequential reasoning and use of program structure. More practically, we evaluate these models on the task of learning to execute partial programs, as might arise if using the model as a heuristic function in program synthesis. Results show that the IPA-GNN outperforms a variety of RNN and GNN baselines on both tasks., Comment: Accepted at NeurIPS 2020

Program synthesis is challenging largely because of the difficulty of search in a large space of programs. Human programmers routinely tackle the task of writing complex programs by writing sub-programs and then analyzing their intermediate results to compose them in appropriate ways. Motivated by this intuition, we present a new synthesis approach that leverages learning to guide a bottom-up search over programs. In particular, we train a model to prioritize compositions of intermediate values during search conditioned on a given set of input-output examples. This is a powerful combination because of several emergent properties. First, in bottom-up search, intermediate programs can be executed, providing semantic information to the neural network. Second, given the concrete values from those executions, we can exploit rich features based on recent work on property signatures. Finally, bottom-up search allows the system substantial flexibility in what order to generate the solution, allowing the synthesizer to build up a program from multiple smaller sub-programs. Overall, our empirical evaluation finds that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches. We demonstrate the effectiveness of our technique on two datasets, one from the SyGuS competition and one of our own creation.

The success and popularity of deep learning is on the rise, partially due to powerful deep learning frameworks such as TensorFlow and PyTorch that make it easier to develop deep learning models. However, these libraries also come with steep learning curves, since programming in these frameworks is quite different from traditional imperative programming with explicit loops and conditionals. In this work, we present a tool called TF-Coder for programming by example in TensorFlow. TF-Coder uses a bottom-up weighted enumerative search, with value-based pruning of equivalent expressions and flexible type- and value-based filtering to ensure that expressions adhere to various requirements imposed by the TensorFlow library. We train models to predict TensorFlow operations from features of the input and output tensors and natural language descriptions of tasks, to prioritize relevant operations during search. TF-Coder solves 63 of 70 real-world tasks within 5 minutes, sometimes finding simpler solutions in less time compared to experienced human programmers., Comment: Published in ACM Transactions on Programming Languages and Systems (TOPLAS) with presentation at PLDI 2022

Sampling is a fundamental technique, and sampling without replacement is often desirable when duplicate samples are not beneficial. Within machine learning, sampling is useful for generating diverse outputs from a trained model. We present an elegant procedure for sampling without replacement from a broad class of randomized programs, including generative neural models that construct outputs sequentially. Our procedure is efficient even for exponentially-large output spaces. Unlike prior work, our approach is incremental, i.e., samples can be drawn one at a time, allowing for increased flexibility. We also present a new estimator for computing expectations from samples drawn without replacement. We show that incremental sampling without replacement is applicable to many domains, e.g., program synthesis and combinatorial optimization.

This paper deals with a study of H-/D-negative ion surface production on diamond in low pressure H2/D2 plasmas. A sample placed in the plasma is negatively biased with respect to plasma potential. Upon positive ion impacts on the sample, some negative ions are formed and detected according to their mass and energy by a mass spectrometer placed in front of the sample. The experimental methods developed to study negative ion surface production and obtain negative ion energy and angle distribution functions are first presented. Different diamond materials ranging from nanocrystalline to single crystal layers, either doped with boron or intrinsic, are then investigated and compared with graphite. The negative ion yields obtained are presented as a function of different experimental parameters such as the exposure time, the sample bias which determines the positive ion impact energy and the sample surface temperature. It is concluded from these experiments that the electronic properties of diamond materials, among them the negative electron affinity, seem to be favourable for negative-ion surface production. However, the negative ion yield decreases with the plasma induced defect density.

Programming languages are emerging as a challenging and interesting domain for machine learning. A core task, which has received significant attention in recent years, is building generative models of source code. However, to our knowledge, previous generative models have always been framed in terms of generating static snapshots of code. In this work, we instead treat source code as a dynamic object and tackle the problem of modeling the edits that software developers make to source code files. This requires extracting intent from previous edits and leveraging it to generate subsequent edits. We develop several neural networks and use synthetic data to test their ability to learn challenging edit patterns that require strong generalization. We then collect and train our models on a large-scale dataset of Google source code, consisting of millions of fine-grained edits from thousands of Python developers. From the modeling perspective, our main conclusion is that a new composition of attentional and pointer network components provides the best overall performance and scalability. From the application perspective, our results provide preliminary evidence of the feasibility of developing tools that learn to predict future edits., Comment: Deanonymized version of ICLR 2019 submission

Due to its potential to improve programmer productivity and software quality, automated program repair has been an active topic of research. Newer techniques harness neural networks to learn directly from examples of buggy programs and their fixes. In this work, we consider a recently identified class of bugs called variable-misuse bugs. The state-of-the-art solution for variable misuse enumerates potential fixes for all possible bug locations in a program, before selecting the best prediction. We show that it is beneficial to train a model that jointly and directly localizes and repairs variable-misuse bugs. We present multi-headed pointer networks for this purpose, with one head each for localization and repair. The experimental results show that the joint model significantly outperforms an enumerative solution that uses a pointer based model for repair alone., Comment: ICLR 2019

A way to assess rare aircraft incidents (e.g., runway excursion) is to identify contributing factors (e.g., late braking, long landing, inappropriate flare, unstable approach) and to build a dependency tree (e.g., long landing may be the result of an unstable approach not followed by a go around) that describes the causality between these factors. Probabilities are then fed into such models in order to evaluate the assessed risk. When estimating such probabilities, many sources can be of interest. Airlines have access to the comprehensive flight data records of their fleet; manufacturers push to collect data for the aircraft they build; air traffic control log radar tracks. Albeit not as complete as other flight data records, Mode S data is very attractive, esp. for academics, as the data is open, may be published without obfuscation and offers reproducible results to the community. Mode S also provides an indiscriminate source of information (not limited to an airline or to an aircraft type) that is of great help for putting in context flights matching unusual patterns. We propose to discuss the advantages and limitations of an analysis based only on Mode S data with a case study around the runway excursion risk assessment.