Search

We consider the problem of predicting several response variables using the same set of explanatory variables. This setting naturally induces a group structure over the coefficient matrix, in which every explanatory variable corresponds to a set of related coefficients. Most of the existing methods that utilize this group formation assume that the similarities between related coefficients arise solely through a joint sparsity structure. In this paper, we propose a procedure for constructing an estimator of a multivariate regression coefficient matrix that directly models and captures the within-group similarities, by employing a multivariate linear mixed model formulation, with a joint estimation of covariance matrices for coefficients and errors via penalized likelihood. Our approach, which we term Multivariate random Regression with Covariance Estimation (MrRCE) encourages structured similarity in parameters, in which coefficients for the same variable in related tasks sharing the same sign and similar magnitude. We illustrate the benefits of our approach in synthetic and real examples, and show that the proposed method outperforms natural competitors and alternative estimators under several model settings.

Typically, real-world stochastic processes are not easy to analyze. In this work we study the representation of any stochastic process as a memoryless innovation process triggering a dynamic system. We show that such a representation is always feasible for innovation processes taking values over a continuous set. However, the problem becomes more challenging when the alphabet size of the innovation is finite. In this case, we introduce both lossless and lossy frameworks, and provide closed-form solutions and practical algorithmic methods. In addition, we discuss the properties and uniqueness of our suggested approach. Finally, we show that the innovation representation problem has many applications. We focus our attention to Entropic Causal Inference, which has recently demonstrated promising performance, compared to alternative methods., Comment: arXiv admin note: text overlap with arXiv:1611.04035 by other authors

In the biomedical domain, the lack of sharable datasets often limit the possibility of developing natural language processing systems, especially dialogue applications and natural language understanding models. To overcome this issue, we explore data generation using templates and terminologies and data augmentation approaches. Namely, we report our experiments using paraphrasing and word representations learned on a large EHR corpus with Fasttext and ELMo, to learn a NLU model without any available dataset. We evaluate on a NLU task of natural language queries in EHRs divided in slot-filling and intent classification sub-tasks. On the slot-filling task, we obtain a F-score of 0.76 with the ELMo representation; and on the classification task, a mean F-score of 0.71. Our results show that this method could be used to develop a baseline system., Comment: Machine Learning for Health (ML4H) Workshop at NeurIPS 2018 arXiv:1811.07216

Ensemble methods are among the state-of-the-art predictive modeling approaches. Applied to modern big data, these methods often require a large number of sub-learners, where the complexity of each learner typically grows with the size of the dataset. This phenomenon results in an increasing demand for storage space, which may be very costly. This problem mostly manifests in a subscriber based environment, where a user-specific ensemble needs to be stored on a personal device with strict storage limitations (such as a cellular device). In this work we introduce a novel method for lossless compression of tree-based ensemble methods, focusing on random forests. Our suggested method is based on probabilistic modeling of the ensemble's trees, followed by model clustering via Bregman divergence. This allows us to find a minimal set of models that provides an accurate description of the trees, and at the same time is small enough to store and maintain. Our compression scheme demonstrates high compression rates on a variety of modern datasets. Importantly, our scheme enables predictions from the compressed format and a perfect reconstruction of the original ensemble. In addition, we introduce a theoretically sound lossy compression scheme, which allows us to control the trade-off between the distortion and the coding rate.

Independent Component Analysis (ICA) is a statistical tool that decomposes an observed random vector into components that are as statistically independent as possible. ICA over finite fields is a special case of ICA, in which both the observations and the decomposed components take values over a finite alphabet. This problem is also known as minimal redundancy representation or factorial coding. In this work we focus on linear methods for ICA over finite fields. We introduce a basic lower bound which provides a fundamental limit to the ability of any linear solution to solve this problem. Based on this bound, we present a greedy algorithm that outperforms all currently known methods. Importantly, we show that the overhead of our suggested algorithm (compared with the lower bound) typically decreases, as the scale of the problem grows. In addition, we provide a sub-optimal variant of our suggested method that significantly reduces the computational complexity at a relatively small cost in performance. Finally, we discuss the universal abilities of linear transformations in decomposing random vectors, compared with existing non-linear solutions.

Semidefinite relaxations are widely used to compute upper bounds on the objective of optimization problems involving noncommutative polynomials. Such optimization problems are prevalent in quantum information. We present an algorithm able to discover automatically and exploit the symmetries present in the problem formulation. We also provide an open source software library written in Scala ( https://denisrosset.github.io/symdpoly ) that computes symmetry-adapted semidefinite relaxations with interfaces to a variety of open-source and commercial semidefinite solvers. We discuss the advantages of symmetrization, namely reductions in memory use, computation time, and increase in the solution precision., Comment: 21 pages, comments most welcome. For associated software, see https://denisrosset.github.io/symdpoly

We present a technique for reducing the computational requirements by several orders of magnitude in the evaluation of semidefinite relaxations for bounding the set of quantum correlations arising from finite-dimensional Hilbert spaces. The technique, which we make publicly available through a user-friendly software package, relies on the exploitation of symmetries present in the optimisation problem to reduce the number of variables and the block sizes in semidefinite relaxations. It is widely applicable in problems encountered in quantum information theory and enables computations that were previously too demanding. We demonstrate its advantages and general applicability in several physical problems. In particular, we use it to robustly certify the non-projectiveness of high-dimensional measurements in a black-box scenario based on self-tests of $d$-dimensional symmetric informationally complete POVMs., Comment: A. T. and D. R. contributed equally for this project

We present 353 GHz full-sky maps of the polarization fraction $p$, angle $\psi$, and dispersion of angles $S$ of Galactic dust thermal emission produced from the 2018 release of Planck data. We confirm that the mean and maximum of $p$ decrease with increasing $N_H$. The uncertainty on the maximum polarization fraction, $p_\mathrm{max}=22.0$% at 80 arcmin resolution, is dominated by the uncertainty on the zero level in total intensity. The observed inverse behaviour between $p$ and $S$ is interpreted with models of the polarized sky that include effects from only the topology of the turbulent Galactic magnetic field. Thus, the statistical properties of $p$, $\psi$, and $S$ mostly reflect the structure of the magnetic field. Nevertheless, we search for potential signatures of varying grain alignment and dust properties. First, we analyse the product map $S \times p$, looking for residual trends. While $p$ decreases by a factor of 3--4 between $N_H=10^{20}$ cm$^{-2}$ and $N_H=2\times 10^{22}$ cm$^{-2}$, $S \times p$ decreases by only about 25%, a systematic trend observed in both the diffuse ISM and molecular clouds. Second, we find no systematic trend of $S \times p$ with the dust temperature, even though in the diffuse ISM lines of sight with high $p$ and low $S$ tend to have colder dust. We also compare Planck data with starlight polarization in the visible at high latitudes. The agreement in polarization angles is remarkable. Two polarization emission-to-extinction ratios that characterize dust optical properties depend only weakly on $N_H$ and converge towards the values previously determined for translucent lines of sight. We determine an upper limit for the polarization fraction in extinction of 13%, compatible with the $p_\mathrm{max}$ observed in emission. These results provide strong constraints for models of Galactic dust in diffuse gas., Comment: Accepted for publication in Astronomy & Astrophysics

The European Space Agency's Planck satellite, which was dedicated to studying the early Universe and its subsequent evolution, was launched on 14 May 2009. It scanned the microwave and submillimetre sky continuously between 12 August 2009 and 23 October 2013, producing deep, high-resolution, all-sky maps in nine frequency bands from 30 to 857GHz. This paper presents the cosmological legacy of Planck, which currently provides our strongest constraints on the parameters of the standard cosmological model and some of the tightest limits available on deviations from that model. The 6-parameter LCDM model continues to provide an excellent fit to the cosmic microwave background data at high and low redshift, describing the cosmological information in over a billion map pixels with just six parameters. With 18 peaks in the temperature and polarization angular power spectra constrained well, Planck measures five of the six parameters to better than 1% (simultaneously), with the best-determined parameter (theta_*) now known to 0.03%. We describe the multi-component sky as seen by Planck, the success of the LCDM model, and the connection to lower-redshift probes of structure formation. We also give a comprehensive summary of the major changes introduced in this 2018 release. The Planck data, alone and in combination with other probes, provide stringent constraints on our models of the early Universe and the large-scale structure within which all astrophysical objects form and evolve. We discuss some lessons learned from the Planck mission, and highlight areas ripe for further experimental advances., Comment: 61 pages, 40 figures, matches version accepted by A&A

We present measurements of the cosmic microwave background (CMB) lensing potential using the final $\textit{Planck}$ 2018 temperature and polarization data. We increase the significance of the detection of lensing in the polarization maps from $5\,\sigma$ to $9\,\sigma$. Combined with temperature, lensing is detected at $40\,\sigma$. We present an extensive set of tests of the robustness of the lensing-potential power spectrum, and construct a minimum-variance estimator likelihood over lensing multipoles $8 \le L \le 400$. We find good consistency between lensing constraints and the results from the $\textit{Planck}$ CMB power spectra within the $\rm{\Lambda CDM}$ model. Combined with baryon density and other weak priors, the lensing analysis alone constrains $\sigma_8 \Omega_{\rm m}^{0.25}=0.589\pm 0.020$ ($1\,\sigma$ errors). Also combining with baryon acoustic oscillation (BAO) data, we find tight individual parameter constraints, $\sigma_8=0.811\pm0.019$, $H_0=67.9_{-1.3}^{+1.2}\,\text{km}\,\text{s}^{-1}\,\rm{Mpc}^{-1}$, and $\Omega_{\rm m}=0.303^{+0.016}_{-0.018}$. Combining with $\textit{Planck}$ CMB power spectrum data, we measure $\sigma_8$ to better than $1\,\%$ precision, finding $\sigma_8=0.811\pm 0.006$. We find consistency with the lensing results from the Dark Energy Survey, and give combined lensing-only parameter constraints that are tighter than joint results using galaxy clustering. Using $\textit{Planck}$ cosmic infrared background (CIB) maps we make a combined estimate of the lensing potential over $60\,\%$ of the sky with considerably more small-scale signal. We demonstrate delensing of the $\textit{Planck}$ power spectra, detecting a maximum removal of $40\,\%$ of the lensing-induced power in all spectra. The improvement in the sharpening of the acoustic peaks by including both CIB and the quadratic lensing reconstruction is detected at high significance (abridged)., Comment: Abstract abridged for arxiv submission. Lensing data products available at https://wiki.cosmos.esa.int/planck-legacy-archive/index.php/Lensing. Matches version accepted by A&A, with minor updates from v1

We report on the implications for cosmic inflation of the 2018 Release of the Planck CMB anisotropy measurements. The results are fully consistent with the two previous Planck cosmological releases, but have smaller uncertainties thanks to improvements in the characterization of polarization at low and high multipoles. Planck temperature, polarization, and lensing data determine the spectral index of scalar perturbations to be $n_\mathrm{s}=0.9649\pm 0.0042$ at 68% CL and show no evidence for a scale dependence of $n_\mathrm{s}.$ Spatial flatness is confirmed at a precision of 0.4% at 95% CL with the combination with BAO data. The Planck 95% CL upper limit on the tensor-to-scalar ratio, $r_{0.002}<0.10$, is further tightened by combining with the BICEP2/Keck Array BK15 data to obtain $r_{0.002}<0.056$. In the framework of single-field inflationary models with Einstein gravity, these results imply that: (a) slow-roll models with a concave potential, $V" (\phi) < 0,$ are increasingly favoured by the data; and (b) two different methods for reconstructing the inflaton potential find no evidence for dynamics beyond slow roll. Non-parametric reconstructions of the primordial power spectrum consistently confirm a pure power law. A complementary analysis also finds no evidence for theoretically motivated parameterized features in the Planck power spectrum, a result further strengthened for certain oscillatory models by a new combined analysis that includes Planck bispectrum data. The new Planck polarization data provide a stringent test of the adiabaticity of the initial conditions. The polarization data also provide improved constraints on inflationary models that predict a small statistically anisotropic quadrupolar modulation of the primordial fluctuations. However, the polarization data do not confirm physical models for a scale-dependent dipolar modulation., Comment: References added and minor improvements. BICEP2/Keck Array BK15 is used in the place of BICEP2/Keck Array BK14

We present cosmological parameter results from the final full-mission Planck measurements of the CMB anisotropies. We find good consistency with the standard spatially-flat 6-parameter $\Lambda$CDM cosmology having a power-law spectrum of adiabatic scalar perturbations (denoted "base $\Lambda$CDM" in this paper), from polarization, temperature, and lensing, separately and in combination. A combined analysis gives dark matter density $\Omega_c h^2 = 0.120\pm 0.001$, baryon density $\Omega_b h^2 = 0.0224\pm 0.0001$, scalar spectral index $n_s = 0.965\pm 0.004$, and optical depth $\tau = 0.054\pm 0.007$ (in this abstract we quote $68\,\%$ confidence regions on measured parameters and $95\,\%$ on upper limits). The angular acoustic scale is measured to $0.03\,\%$ precision, with $100\theta_*=1.0411\pm 0.0003$. These results are only weakly dependent on the cosmological model and remain stable, with somewhat increased errors, in many commonly considered extensions. Assuming the base-$\Lambda$CDM cosmology, the inferred late-Universe parameters are: Hubble constant $H_0 = (67.4\pm 0.5)$km/s/Mpc; matter density parameter $\Omega_m = 0.315\pm 0.007$; and matter fluctuation amplitude $\sigma_8 = 0.811\pm 0.006$. We find no compelling evidence for extensions to the base-$\Lambda$CDM model. Combining with BAO we constrain the effective extra relativistic degrees of freedom to be $N_{\rm eff} = 2.99\pm 0.17$, and the neutrino mass is tightly constrained to $\sum m_\nu< 0.12$eV. The CMB spectra continue to prefer higher lensing amplitudes than predicted in base -$\Lambda$CDM at over $2\,\sigma$, which pulls some parameters that affect the lensing amplitude away from the base-$\Lambda$CDM model; however, this is not supported by the lensing reconstruction or (in models that also change the background geometry) BAO data. (Abridged), Comment: 73 pages; Updated with published reionization result corrigendum on p59. Parameter tables and chains available at https://wiki.cosmos.esa.int/planck-legacy-archive/index.php/Cosmological_Parameters

In this paper we consider the inverse problem of determining a rigid inclusion inside a thin plate by applying a couple field at the boundary and by measuring the induced transversal displacement and its normal derivative at the boundary of the plate. The plate is made by non-homogeneous, linearly elastic and isotropic material. Under suitable a priori regularity assumptions on the boundary of the inclusion, we prove a constructive stability estimate of log type. Key mathematical tool is a recently proved optimal three spheres inequality at the boundary for solutions to the Kirchhoff-Love plate's equation., Comment: 19 pages, 0 figures. arXiv admin note: text overlap with arXiv:1111.0604

Multiple testing problems are a staple of modern statistical analysis. The fundamental objective of multiple testing procedures is to reject as many false null hypotheses as possible (that is, maximize some notion of power), subject to controlling an overall measure of false discovery, like family-wise error rate (FWER) or false discovery rate (FDR). In this paper we formulate multiple testing of simple hypotheses as an infinite-dimensional optimization problem, seeking the most powerful rejection policy which guarantees strong control of the selected measure. In that sense, our approach is a generalization of the optimal Neyman-Pearson test for a single hypothesis. We show that for exchangeable hypotheses, for both FWER and FDR and relevant notions of power, these problems can be formulated as infinite linear programs and can in principle be solved for any number of hypotheses. We also characterize maximin rules for complex alternatives, and demonstrate that such rules can be found in practice, leading to improved practical procedures compared to existing alternatives. We derive explicit optimal tests for FWER or FDR control for three independent normal means. We find that the power gain over natural competitors is substantial in all settings examined. Finally, we apply our optimal maximin rule to subgroup analyses in systematic reviews from the Cochrane library, leading to an increase in the number of findings while guaranteeing strong FWER control against the one sided alternative.

In web search, typically a candidate generation step selects a small set of documents---from collections containing as many as billions of web pages---that are subsequently ranked and pruned before being presented to the user. In Bing, the candidate generation involves scanning the index using statically designed match plans that prescribe sequences of different match criteria and stopping conditions. In this work, we pose match planning as a reinforcement learning task and observe up to 20% reduction in index blocks accessed, with small or no degradation in the quality of the candidate sets., Comment: ACM SIGIR 2018 short paper (pre-print)

The problem of handling adaptivity in data analysis, intentional or not, permeates a variety of fields, including test-set overfitting in ML challenges and the accumulation of invalid scientific discoveries. We propose a mechanism for answering an arbitrarily long sequence of potentially adaptive statistical queries, by charging a price for each query and using the proceeds to collect additional samples. Crucially, we guarantee statistical validity without any assumptions on how the queries are generated. We also ensure with high probability that the cost for $M$ non-adaptive queries is $O(\log M)$, while the cost to a potentially adaptive user who makes $M$ queries that do not depend on any others is $O(\sqrt{M})$., Comment: 22 pages, accepted to NeurIPS 2018

We prove a three sphere inequality with optimal exponent at the boundary for solutions to the Kirchhoff-Love plate's equation satisfying homogeneous Dirichlet conditions. This result implies the Strong Unique Continuation Property at the Boundary (SUCPB). Our approach is based on the method of Carleman estimates, and involves the construction of an ad hoc conformal mapping preserving the structure of the operator and the employment of a suitable reflection of the solution with respect to the flatten boundary which ensures the needed regularity of the extended solution. To the authors' knowledge, this is the first (nontrivial) SUCPB result for fourth-order equations with bi-Laplacian principal part., Comment: 36 pages, to appear on Archive for Rational Mechanics and Analysis

Non-local correlations that obey the no-signalling principle contain intrinsic randomness. In particular, for a specific Bell experiment, one can derive relations between the amount of randomness produced, as quantified by the min-entropy of the output data, and its associated violation of a Bell inequality. In practice, due to finite sampling, certifying randomness requires the development of statistical tools to lower-bound the min-entropy of the data as a function of the estimated Bell violation. The quality of such bounds relies on the choice of certificate, i.e., the Bell inequality whose violation is estimated. In this work, we propose a method for choosing efficiently such a certificate. It requires sacrificing a part of the output data in order to estimate the underlying correlations. Regularising this estimate then allows one to find a Bell inequality that is well suited for certifying practical randomness from these specific correlations. We then study the effects of various parameters on the obtained min-entropy bound and explain how to tune them in a favourable way. Lastly, we carry out several numerical simulations of a Bell experiment to show the efficiency of our method: we nearly always obtain higher min-entropy rates than when we use a pre-established Bell inequality, namely the Clauser-Horne-Shimony-Holt inequality., Comment: 12 pages + 6 figures. Comments welcome

In quantum state tomography, the estimated frequencies do not correspond directly to a physical quantum state, due to statistical fluctuations. Thus, one resorts to point estimators that return the state that matches observations the best, and a variety of estimators have been proposed -- linear inversion, least squares, maximum likelihood (ML) -- each making different trade-offs. In this short note, we investigate an alternative approach inspired by device-independent quantum information protocols. We embed a tomographic complete set of measurement operators within the framework of a Bell scenario, and first regularize the estimated frequencies using a device-independent maximum likelihood algorithm. We then run the standard maximum likelihood algorithm to estimate the underlying quantum state. In this preliminary investigation, we do not observe significant differences in the reconstructed state when using the new algorithm., Comment: 6 pages, 2 figures, analysis substantially improved

Common model selection criteria, such as $AIC$ and its variants, are based on in-sample prediction error estimators. However, in many applications involving predicting at interpolation and extrapolation points, in-sample error cannot be used for estimating the prediction error. In this paper new prediction error estimators, $tAI$ and $Loss(w_{t})$ are introduced. These estimators generalize previous error estimators, however are also applicable for assessing prediction error in cases involving interpolation and extrapolation. Based on the prediction error estimators, two model selection criteria with the same spirit as $AIC$ are suggested. The advantages of our suggested methods are demonstrated in simulation and real data analysis of studies involving interpolation and extrapolation in a Linear Mixed Model framework., Comment: Corrected typos and editing change. The content remain the same

The goal of self-testing is to characterize an a priori unknown quantum system based solely on measurement statistics, i.e. using an uncharacterized measurement device. Here we develop self-testing methods for quantum prepare-and-measure experiments, thus not necessarily relying on entanglement and/or violation of a Bell inequality. We present noise-robust techniques for self-testing sets of quantum states and measurements, assuming an upper bound on the Hilbert space dimension. We discuss in detail the case of a $2 \rightarrow 1$ random access code with qubits, for which we provide analytically optimal self-tests. The simplicity and noise robustness of our methods should make them directly applicable to experiments.

We examine Deep Canonically Correlated LSTMs as a way to learn nonlinear transformations of variable length sequences and embed them into a correlated, fixed dimensional space. We use LSTMs to transform multi-view time-series data non-linearly while learning temporal relationships within the data. We then perform correlation analysis on the outputs of these neural networks to find a correlated subspace through which we get our final representation via projection. This work follows from previous work done on Deep Canonical Correlation (DCCA), in which deep feed-forward neural networks were used to learn nonlinear transformations of data while maximizing correlation., Comment: 8 pages, 3 figures, accepted as the undergraduate honors thesis for Neil Mallinar by The Johns Hopkins University

The study of polarized dust emission has become entwined with the analysis of the cosmic microwave background (CMB) polarization. We use new Planck maps to characterize Galactic dust emission as a foreground to the CMB polarization. We present Planck EE, BB, and TE power spectra of dust polarization at 353 GHz for six nested sky regions covering from 24 to 71 % of the sky. We present power-law fits to the angular power spectra, yielding evidence for statistically significant variations of the exponents over sky regions and a difference between the values for the EE and BB spectra. The TE correlation and E/B power asymmetry extend to low multipoles that were not included in earlier Planck polarization papers. We also report evidence for a positive TB dust signal. Combining data from Planck and WMAP, we determine the amplitudes and spectral energy distributions (SEDs) of polarized foregrounds, including the correlation between dust and synchrotron polarized emission, for the six sky regions as a function of multipole. This quantifies the challenge of the component separation procedure required for detecting the reionization and recombination peaks of primordial CMB B modes. The SED of polarized dust emission is fit well by a single-temperature modified blackbody emission law from 353 GHz to below 70 GHz. For a dust temperature of 19.6 K, the mean spectral index for dust polarization is $\beta_{\rm d}^{P} = 1.53\pm0.02 $. By fitting multi-frequency cross-spectra, we examine the correlation of the dust polarization maps across frequency. We find no evidence for decorrelation. If the Planck limit for the largest sky region applies to the smaller sky regions observed by sub-orbital experiments, then decorrelation might not be a problem for CMB experiments aiming at a primordial B-mode detection limit on the tensor-to-scalar ratio $r\simeq0.01$ at the recombination peak., Comment: Final version to appear in A&A

Modern data sets in various domains often include units that were sampled non-randomly from the population and have a latent correlation structure. Here we investigate a common form of this setting, where every unit is associated with a latent variable, all latent variables are correlated, and the probability of sampling a unit depends on its response. Such settings often arise in case-control studies, where the sampled units are correlated due to spatial proximity, family relations, or other sources of relatedness. Maximum likelihood estimation in such settings is challenging from both a computational and statistical perspective, necessitating approximations that take the sampling scheme into account. We propose a family of approximate likelihood approaches which combine composite likelihood and expectation propagation. We demonstrate the efficacy of our solutions via extensive simulations. We utilize them to investigate the genetic architecture of several complex disorders collected in case-control genetic association studies, where hundreds of thousands of genetic variants are measured for every individual, and the underlying disease liabilities of individuals are correlated due to genetic similarity. Our work is the first to provide a tractable likelihood-based solution for case-control data with complex dependency structures.