Search

We consider a Gatenby--Gawlinski-type model of invasive tumors in the presence of an Allee effect. We describe the construction of bistable one-dimensional traveling fronts using singular perturbation techniques in different parameter regimes corresponding to tumor interfaces with, or without, acellular gap. By extending the front as a planar interface, we perform a stability analysis to long wavelength perturbations transverse to the direction of front propagation and derive a simple stability criterion for the front in two spatial dimensions. In particular we find that in general the presence of the acellular gap indicates transversal instability of the associated planar front, which can lead to complex interfacial dynamics such as the development of finger-like protrusions and/or different invasion speeds., Comment: 24 pages, 12 figures

Many experimental studies report that economics students tend to act more selfishly than students of other disciplines, a finding that received widespread public and professional attention. Two main explanations that the existing literature offers for the differences found in the behavior between economists and noneconomists are the selection effect, and the indoctrination effect. We offer an alternative, novel explanation. We argue that these differences can be explained by differences in the interpretation of the context. We test this hypothesis by conducting two social dilemma experiments in the US and Israel with participants from both economics and non-economics majors. In the experiments, participants face a tradeoff between profit maximization, that is the market norm and workers welfare, that is the social norm. We use priming to manipulate the cues that the participants receive before they make their decision. We find that when participants receive cues signaling that the decision has an economic context, both economics and non-economics students tend to maximize profits. When the participants receive cues emphasizing social norms, on the other hand, both economics and non-economics students are less likely to maximize profits. We conclude that some of the differences found between the decisions of economics and non-economics students can be explained by contextual cues., Comment: 57 pages, contains the body of the paper and the appendix

We report the results of surveys we conducted in the US and Israel in 2020, a time when many prices increased following the spread of the pandemic. To assess respondents perceptions of price increases, we focus on goods whose prices have increased during the pandemic, including some essential goods. Consistent with the principle of dual entitlement, we find that respondents perceive price increases as more acceptable if they are due to cost shocks than if they are due to demand shocks. However, we also find large differences across the two populations, as well as across goods.

The finding of small price changes in many retail price datasets is often viewed as a puzzle. We show that a possible explanation for the presence of small price changes is related to sales volume, an observation that has been overlooked in the existing literature. Analyzing a large retail scanner price dataset that contains information on both prices and sales volume, we find that small price changes are more frequent when products sales volume is high. This finding holds across product categories, within product categories, and for individual products. It is also robust to various sensitivity analyses such as measurement errors, the definition of small price changes, the inclusion of measures of price synchronization, the size of producers, the time horizon used to compute the average sales volume, the revenues, the competition, shoppers characteristics, etc.

Triangular meshes are widely used to represent three-dimensional objects. As a result, many recent works have address the need for geometric deep learning on 3D mesh. However, we observe that the complexities in many of these architectures does not translate to practical performance, and simple deep models for geometric graphs are competitive in practice. Motivated by this observation, we minimally extend the update equations of E(n)-Equivariant Graph Neural Networks (EGNNs) (Satorras et al., 2021) to incorporate mesh face information, and further improve it to account for long-range interactions through hierarchy. The resulting architecture, Equivariant Mesh Neural Network (EMNN), outperforms other, more complicated equivariant methods on mesh tasks, with a fast run-time and no expensive pre-processing. Our implementation is available at https://github.com/HySonLab/EquiMesh

We present a natural extension to E(n)-equivariant graph neural networks that uses multiple equivariant vectors per node. We formulate the extension and show that it improves performance across different physical systems benchmark tasks, with minimal differences in runtime or number of parameters. The proposed multichannel EGNN outperforms the standard singlechannel EGNN on N-body charged particle dynamics, molecular property predictions, and predicting the trajectories of solar system bodies. Given the additional benefits and minimal additional cost of multi-channel EGNN, we suggest that this extension may be of practical use to researchers working in machine learning for the physical sciences

We study the price rigidity of regular and sale prices, and how it is affected by pricing formats (pricing strategies). We use data from three large Canadian stores with different pricing formats (Every-Day-Low-Price, Hi-Lo, and Hybrid) that are located within a 1 km radius of each other. Our data contains both the actual transaction prices and actual regular prices as displayed on the store shelves. We combine these data with two generated regular price series (filtered prices and reference prices) and study their rigidity. Regular price rigidity varies with store formats because different format stores treat sale prices differently, and consequently define regular prices differently. Correspondingly, the meanings of price cuts and sale prices vary across store formats. To interpret the findings, we consider the store pricing format distribution across the US., Comment: Economica (Forthcoming), 102 pages, 59 pages paper, 43 pages supplementary online web appendix

We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4., Comment: 100 pages; updated authors list; fixed author names and added citation

Inflation is painful, for firms, customers, employees, and society. But careful study of periods of hyperinflation point to ways that firms can adapt. In particular, companies need to think about how to change prices regularly and cheaply, because constant price changes can ultimately be very, very expensive. And they should consider how to communicate those price changes to customers. Providing clarity and predictability can increase consumer trust and help firms in the long run., Comment: 8 pages

We assess the role of cognitive convenience in the popularity and rigidity of 0 ending prices in convenience settings. Studies show that 0 ending prices are common at convenience stores because of the transaction convenience that 0 ending prices offer. Using a large store level retail CPI data, we find that 0 ending prices are popular and rigid at convenience stores even when they offer little transaction convenience. We corroborate these findings with two large retail scanner price datasets from Dominicks and Nielsen. In the Dominicks data, we find that there are more 0 endings in the prices of the items in the front end candies category than in any other category, even though these prices have no effect on the convenience of the consumers check out transaction. In addition, in both Dominicks and Nielsens datasets, we find that 0 ending prices have a positive effect on demand. Ruling out consumer antagonism and retailers use of heuristics in pricing, we conclude that 0 ending prices are popular and rigid, and that they increase demand at convenience settings, not only for their transaction convenience, but also for the cognitive convenience they offer., Comment: Journal of Economic Behavior and Organization (Forthcoming), 89 pages, 50 pages paper, 39 pages appendix

Recent studies in psychology and neuroscience offer systematic evidence that fictional works exert a surprisingly strong influence on readers and have the power to shape their opinions and worldviews. Building on these findings, we study what we term Potterian economics, the economic ideas, insights, and structure, found in Harry Potter books, to assess how the books might affect economic literacy. A conservative estimate suggests that more than 7.3 percent of the world population has read the Harry Potter books, and millions more have seen their movie adaptations. These extraordinary figures underscore the importance of the messages the books convey. We explore the Potterian economic model and compare it to professional economic models to assess the consistency of the Potterian economic principles with the existing economic models. We find that some of the principles of Potterian economics are consistent with economists models. Many other principles, however, are distorted and contain numerous inaccuracies, contradicting professional economists views and insights. We conclude that Potterian economics can teach us about the formation and dissemination of folk economics, the intuitive notions of naive individuals who see market transactions as a zero-sum game, who care about distribution but fail to understand incentives and efficiency, and who think of prices as allocating wealth but not resources or their efficient use., Comment: 32 pages

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 450 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models. We evaluate the behavior of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters. In addition, a team of human expert raters performed all tasks in order to provide a strong baseline. Findings include: model performance and calibration both improve with scale, but are poor in absolute terms (and when compared with rater performance); performance is remarkably similar across model classes, though with benefits from sparsity; tasks that improve gradually and predictably commonly involve a large knowledge or memorization component, whereas tasks that exhibit "breakthrough" behavior at a critical scale often involve multiple steps or components, or brittle metrics; social bias typically increases with scale in settings with ambiguous context, but this can be improved with prompting., Comment: 27 pages, 17 figures + references and appendices, repo: https://github.com/google/BIG-bench

Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model, potentially improving both the accuracy and the memory-efficiency of deployed models. However, the heavy data imbalance between languages hinders the model from performing uniformly across language pairs. In this paper, we propose a new learning objective for MNMT based on distributionally robust optimization, which minimizes the worst-case expected loss over the set of language pairs. We further show how to practically optimize this objective for large translation corpora using an iterated best response scheme, which is both effective and incurs negligible additional computational cost compared to standard empirical risk minimization. We perform extensive experiments on three sets of languages from two datasets and show that our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings., Comment: Long paper accepted by EMNLP2021 main conference

We develop algorithms for private stochastic convex optimization that adapt to the hardness of the specific function we wish to optimize. While previous work provide worst-case bounds for arbitrary convex functions, it is often the case that the function at hand belongs to a smaller class that enjoys faster rates. Concretely, we show that for functions exhibiting $\kappa$-growth around the optimum, i.e., $f(x) \ge f(x^*) + \lambda \kappa^{-1} \|x-x^*\|_2^\kappa$ for $\kappa > 1$, our algorithms improve upon the standard ${\sqrt{d}}/{n\varepsilon}$ privacy rate to the faster $({\sqrt{d}}/{n\varepsilon})^{\tfrac{\kappa}{\kappa - 1}}$. Crucially, they achieve these rates without knowledge of the growth constant $\kappa$ of the function. Our algorithms build upon the inverse sensitivity mechanism, which adapts to instance difficulty (Asi & Duchi, 2020), and recent localization techniques in private optimization (Feldman et al., 2020). We complement our algorithms with matching lower bounds for these function classes and demonstrate that our adaptive algorithm is \emph{simultaneously} (minimax) optimal over all $\kappa \ge 1+c$ whenever $c = \Theta(1)$., Comment: 28 pages

We propose and analyze algorithms to solve a range of learning tasks under user-level differential privacy constraints. Rather than guaranteeing only the privacy of individual samples, user-level DP protects a user's entire contribution ($m \ge 1$ samples), providing more stringent but more realistic protection against information leaks. We show that for high-dimensional mean estimation, empirical risk minimization with smooth losses, stochastic convex optimization, and learning hypothesis classes with finite metric entropy, the privacy cost decreases as $O(1/\sqrt{m})$ as users provide more samples. In contrast, when increasing the number of users $n$, the privacy cost decreases at a faster $O(1/n)$ rate. We complement these results with lower bounds showing the minimax optimality of our algorithms for mean estimation and stochastic convex optimization. Our algorithms rely on novel techniques for private mean estimation in arbitrary dimension with error scaling as the concentration radius $\tau$ of the distribution rather than the entire range., Comment: NeurIPS 2021. 43 pages, 0 figure

We propose and analyze algorithms for distributionally robust optimization of convex losses with conditional value at risk (CVaR) and $\chi^2$ divergence uncertainty sets. We prove that our algorithms require a number of gradient evaluations independent of training set size and number of parameters, making them suitable for large-scale applications. For $\chi^2$ uncertainty sets these are the first such guarantees in the literature, and for CVaR our guarantees scale linearly in the uncertainty level rather than quadratically as in previous work. We also provide lower bounds proving the worst-case optimality of our algorithms for CVaR and a penalized version of the $\chi^2$ problem. Our primary technical contributions are novel bounds on the bias of batch robust risk estimation and the variance of a multilevel Monte Carlo gradient estimator due to [Blanchet & Glynn, 2015]. Experiments on MNIST and ImageNet confirm the theoretical scaling of our algorithms, which are 9--36 times more efficient than full-batch methods., Comment: 63 pages, NeurIPS 2020

We study the impact of the constraint set and gradient geometry on the convergence of online and stochastic methods for convex optimization, providing a characterization of the geometries for which stochastic gradient and adaptive gradient methods are (minimax) optimal. In particular, we show that when the constraint set is quadratically convex, diagonally pre-conditioned stochastic gradient methods are minimax optimal. We further provide a converse that shows that when the constraints are not quadratically convex---for example, any $\ell_p$-ball for $p < 2$---the methods are far from optimal. Based on this, we can provide concrete recommendations for when one should use adaptive, mirror or stochastic gradient methods., Comment: 23 pages. To appear at NeurIPS 2019

Homomorphic encryption enables arbitrary computation over data while it remains encrypted. This privacy-preserving feature is attractive for machine learning, but requires significant computational time due to the large overhead of the encryption scheme. We present Faster CryptoNets, a method for efficient encrypted inference using neural networks. We develop a pruning and quantization approach that leverages sparse representations in the underlying cryptosystem to accelerate inference. We derive an optimal approximation for popular activation functions that achieves maximally-sparse encodings and minimizes approximation error. We also show how privacy-safe training techniques can be used to reduce the overhead of encrypted inference for real-world datasets by leveraging transfer learning and differential privacy. Our experiments show that our method maintains competitive accuracy and achieves a significant speedup over previous methods. This work increases the viability of deep learning systems that use homomorphic encryption to protect user privacy.

We present a general-purpose method to train Markov chain Monte Carlo kernels, parameterized by deep neural networks, that converge and mix quickly to their target distribution. Our method generalizes Hamiltonian Monte Carlo and is trained to maximize expected squared jumped distance, a proxy for mixing speed. We demonstrate large empirical gains on a collection of simple but challenging distributions, for instance achieving a 106x improvement in effective sample size in one case, and mixing when standard HMC makes no measurable progress in a second. Finally, we show quantitative and qualitative gains on a real-world task: latent-variable generative modeling. We release an open source TensorFlow implementation of the algorithm., Comment: ICLR 2018

Policy optimization methods have shown great promise in solving complex reinforcement and imitation learning tasks. While model-free methods are broadly applicable, they often require many samples to optimize complex policies. Model-based methods greatly improve sample-efficiency but at the cost of poor generalization, requiring a carefully handcrafted model of the system dynamics for each task. Recently, hybrid methods have been successful in trading off applicability for improved sample-complexity. However, these have been limited to continuous action spaces. In this work, we present a new hybrid method based on an approximation of the dynamics as an expectation over the next state under the current policy. This relaxation allows us to derive a novel hybrid policy gradient estimator, combining score function and pathwise derivative estimators, that is applicable to discrete action spaces. We show significant gains in sample complexity, ranging between $1.7$ and $25\times$, when learning parameterized policies on Cart Pole, Acrobot, Mountain Car and Hand Mass. Our method is applicable to both discrete and continuous action spaces, when competing pathwise methods are limited to the latter., Comment: In AAAI 2018 proceedings

Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amortized cost. Our idea hinges on using Gumbel random variable perturbations and a pre-computed Maximum Inner Product Search data structure to access the most-likely elements in sublinear amortized time. Our method yields provable runtime and accuracy guarantees. Further, we present empirical experiments on ImageNet and Word Embeddings showing significant speedups for sampling, inference, and learning in log-linear models., Comment: In UAI proceedings

Data noising is an effective technique for regularizing neural network models. While noising is widely adopted in application domains such as vision and speech, commonly used noising primitives have not been developed for discrete sequence-level settings such as language modeling. In this paper, we derive a connection between input noising in neural network language models and smoothing in $n$-gram models. Using this connection, we draw upon ideas from smoothing to develop effective noising schemes. We demonstrate performance gains when applying the proposed schemes to language modeling and machine translation. Finally, we provide empirical analysis validating the relationship between noising and smoothing., Comment: ICLR 2017

Mammography is the most widely used method to screen breast cancer. Because of its mostly manual nature, variability in mass appearance, and low signal-to-noise ratio, a significant number of breast masses are missed or misdiagnosed. In this work, we present how Convolutional Neural Networks can be used to directly classify pre-segmented breast masses in mammograms as benign or malignant, using a combination of transfer learning, careful pre-processing and data augmentation to overcome limited training data. We achieve state-of-the-art results on the DDSM dataset, surpassing human performance, and show interpretability of our model., Comment: NIPS 2016 ML4HC Workshop