Search

The need for regression models to predict circular values arises in many scientific fields. In this work we explore a family of expressive and interpretable distributions over circle-valued random functions related to Gaussian processes targeting two Euclidean dimensions conditioned on the unit circle. The resulting probability model has connections with continuous spin models in statistical physics. Moreover, its density is very simple and has maximum-entropy, unlike previous Gaussian process-based approaches, which use wrapping or radial marginalization. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling. We argue that transductive learning in these models favors a Bayesian approach to the parameters. We present experiments applying this model to the prediction of (i) wind directions and (ii) the percentage of the running gait cycle as a function of joint angles., Comment: Contribution to the Structured Probabilistic Inference & Generative Modeling workshop of ICML 2024

We show that Hamiltonian Monte Carlo, applied to the von Mises distribution with Laplace distribution for the momentum, has exactly solvable equations of motion. With an appropriate travel time, the Markov chain has negative autocorrelation at odd lags and yields a relative effective sample size bigger than one., Comment: 5 pages, 3 figures

Probability density models based on deep networks have achieved remarkable success in modeling complex high-dimensional datasets. However, unlike kernel density estimators, modern neural models do not yield marginals or conditionals in closed form, as these quantities require the evaluation of seldom tractable integrals. In this work, we present the Marginalizable Density Model Approximator (MDMA), a novel deep network architecture which provides closed form expressions for the probabilities, marginals and conditionals of any subset of the variables. The MDMA learns deep scalar representations for each individual variable and combines them via learned hierarchical tensor decompositions into a tractable yet expressive CDF, from which marginals and conditional densities are easily obtained. We illustrate the advantage of exact marginalizability in several tasks that are out of reach of previous deep network-based density estimation models, such as estimating mutual information between arbitrary subsets of variables, inferring causality by testing for conditional independence, and inference with missing data without the need for data imputation, outperforming state-of-the-art models on these tasks. The model also allows for parallelized sampling with only a logarithmic dependence of the time complexity on the number of variables.

The integration and transfer of information from multiple sources to multiple targets is a core motive of neural systems. The emerging field of partial information decomposition (PID) provides a novel information-theoretic lens into these mechanisms by identifying synergistic, redundant, and unique contributions to the mutual information between one and several variables. While many works have studied aspects of PID for Gaussian and discrete distributions, the case of general continuous distributions is still uncharted territory. In this work we present a method for estimating the unique information in continuous distributions, for the case of one versus two variables. Our method solves the associated optimization problem over the space of distributions with fixed bivariate marginals by combining copula decompositions and techniques developed to optimize variational autoencoders. We obtain excellent agreement with known analytic results for Gaussians, and illustrate the power of our new approach in several brain-inspired neural models. Our method is capable of recovering the effective connectivity of a chaotic network of rate neurons, and uncovers a complex trade-off between redundancy, synergy and unique information in recurrent networks trained to solve a generalized XOR task.

Learning community structures in graphs has broad applications across scientific domains. While graph neural networks (GNNs) have been successful in encoding graph structures, existing GNN-based methods for community detection are limited by requiring knowledge of the number of communities in advance, in addition to lacking a proper probabilistic formulation to handle uncertainty. We propose a simple framework for amortized community detection, which addresses both of these issues by combining the expressive power of GNNs with recent methods for amortized clustering. Our models consist of a graph representation backbone that extracts structural information and an amortized clustering network that naturally handles variable numbers of clusters. Both components combine into well-defined models of the posterior distribution of graph communities and are jointly optimized given labeled graphs. At inference time, the models yield parallel samples from the posterior of community labels, quantifying uncertainty in a principled way. We evaluate several models from our framework on synthetic and real datasets, and demonstrate improved performance compared to previous methods. As a separate contribution, we extend recent amortized probabilistic clustering architectures by adding attention modules, which yield further improvements on community detection tasks., Comment: Accepted by the Structured Probabilistic Inference & Generative Modeling workshop of ICML 2024, Vienna, Austria

Probabilistic clustering models (or equivalently, mixture models) are basic building blocks in countless statistical models and involve latent random variables over discrete spaces. For these models, posterior inference methods can be inaccurate and/or very slow. In this work we introduce deep network architectures trained with labeled samples from any generative model of clustered datasets. At test time, the networks generate approximate posterior samples of cluster labels for any new dataset of arbitrary size. We develop two complementary approaches to this task, requiring either O(N) or O(K) network forward passes per dataset, where N is the dataset size and K the number of clusters. Unlike previous approaches, our methods sample the labels of all the data points from a well-defined posterior, and can learn nonparametric Bayesian posteriors since they do not limit the number of mixture components. As a scientific application, we present a novel approach to neural spike sorting for high-density multielectrode arrays.

We develop methods for efficient amortized approximate Bayesian inference over posterior distributions of probabilistic clustering models, such as Dirichlet process mixture models. The approach is based on mapping distributed, symmetry-invariant representations of cluster arrangements into conditional probabilities. The method parallelizes easily, yields iid samples from the approximate posterior of cluster assignments with the same computational cost of a single Gibbs sampler sweep, and can easily be applied to both conjugate and non-conjugate models, as training only requires samples from the generative model., Comment: Presented at BNP@NeurIPS 2018 Workshop

The Bouncy Particle Sampler is a novel rejection-free non-reversible sampler for differentiable probability distributions over continuous variables. We generalize the algorithm to piecewise differentiable distributions and apply it to generic binary distributions using a piecewise differentiable augmentation. We illustrate the new algorithm in a binary Markov Random Field example, and compare it to binary Hamiltonian Monte Carlo. Our results suggest that binary BPS samplers are better for easy to mix distributions., Comment: 4 pages

We introduce a novel stochastic version of the non-reversible, rejection-free Bouncy Particle Sampler (BPS), a Markov process whose sample trajectories are piecewise linear. The algorithm is based on simulating first arrival times in a doubly stochastic Poisson process using the thinning method, and allows efficient sampling of Bayesian posteriors in big datasets. We prove that in the BPS no bias is introduced by noisy evaluations of the log-likelihood gradient. On the other hand, we argue that efficiency considerations favor a small, controllable bias in the construction of the thinning proposals, in exchange for faster mixing. We introduce a simple regression-based proposal intensity for the thinning method that controls this trade-off. We illustrate the algorithm in several examples in which it outperforms both unbiased, but slowly mixing stochastic versions of BPS, as well as biased stochastic gradient-based samplers., Comment: ICML Camera ready version

Partition functions of probability distributions are important quantities for model evaluation and comparisons. We present a new method to compute partition functions of complex and multimodal distributions. Such distributions are often sampled using simulated tempering, which augments the target space with an auxiliary inverse temperature variable. Our method exploits the multinomial probability law of the inverse temperatures, and provides estimates of the partition function in terms of a simple quotient of Rao-Blackwellized marginal inverse temperature probability estimates, which are updated while sampling. We show that the method has interesting connections with several alternative popular methods, and offers some significant advantages. In particular, we empirically find that the new method provides more accurate estimates than Annealed Importance Sampling when calculating partition functions of large Restricted Boltzmann Machines (RBM); moreover, the method is sufficiently accurate to track training and validation log-likelihoods during learning of RBMs, at minimal computational cost., Comment: 15 pages, 8 figures; Appearing at International Conference on Machine Learning 2016

Model-free reinforcement learning algorithms, such as Q-learning, perform poorly in the early stages of learning in noisy environments, because much effort is spent unlearning biased estimates of the state-action value function. The bias results from selecting, among several noisy estimates, the apparent optimum, which may actually be suboptimal. We propose G-learning, a new off-policy learning algorithm that regularizes the value estimates by penalizing deterministic policies in the beginning of the learning process. We show that this method reduces the bias of the value-function estimation, leading to faster convergence to the optimal value and the optimal policy. Moreover, G-learning enables the natural incorporation of prior domain knowledge, when available. The stochastic nature of G-learning also makes it avoid some exploration costs, a property usually attributed only to on-policy algorithms. We illustrate these ideas in several examples, where G-learning results in significant improvements of the convergence rate and the cost of the learning process.

We present efficient Bayesian methods for extracting neuronal spiking information from calcium imaging data. The goal of our methods is to sample from the posterior distribution of spike trains and model parameters (baseline concentration, spike amplitude etc) given noisy calcium imaging data. We present discrete time algorithms where we sample the existence of a spike at each time bin using Gibbs methods, as well as continuous time algorithms where we sample over the number of spikes and their locations at an arbitrary resolution using Metropolis-Hastings methods for point processes. We provide Rao-Blackwellized extensions that (i) marginalize over several model parameters and (ii) provide smooth estimates of the marginal spike posterior distribution in continuous time. Our methods serve as complements to standard point estimates and allow for quantification of uncertainty in estimating the underlying spike train and model parameters.

We present a new approach to sample from generic binary distributions, based on an exact Hamiltonian Monte Carlo algorithm applied to a piecewise continuous augmentation of the binary distribution of interest. An extension of this idea to distributions over mixtures of binary and possibly-truncated Gaussian or exponential variables allows us to sample from posteriors of linear and probit regression models with spike-and-slab priors and truncated parameters. We illustrate the advantages of these algorithms in several examples in which they outperform the Metropolis or Gibbs samplers., Comment: 11 pages, 4 figures. Proceedings of the 27th Annual Conference Neural Information Processing Systems (NIPS), 2013

We present a Hamiltonian Monte Carlo algorithm to sample from multivariate Gaussian distributions in which the target space is constrained by linear and quadratic inequalities or products thereof. The Hamiltonian equations of motion can be integrated exactly and there are no parameters to tune. The algorithm mixes faster and is more efficient than Gibbs sampling. The runtime depends on the number and shape of the constraints but the algorithm is highly parallelizable. In many cases, we can exploit special structure in the covariance matrices of the untruncated Gaussian to further speed up the runtime. A simple extension of the algorithm permits sampling from distributions whose log-density is piecewise quadratic, as in the "Bayesian Lasso" model., Comment: 25 pages, 7 figures

We consider "gauge invariant" operators in Sym^N T^4, the symmetric product orbifold of N copies of the 2d supersymmetric sigma model with T^4 target. We discuss a spin chain representation for single-cycle operators and study their two point functions at large N. We perform systematic calculations at the orbifold point ("tree level"), where non-trivial mixing is already present, and some sample calculations to first order in the blow-up mode of the orbifold ("one loop")., Comment: 52 pages, 10 figures

We develop a diagrammatic language for symmetric product orbifolds of two-dimensional conformal field theories. Correlation functions of twist operators are written as sums of diagrams: each diagram corresponds to a branched covering map from a surface where the fields are single-valued to the base sphere where twist operators are inserted. This diagrammatic language facilitates the study of the large N limit and makes more transparent the analogy between symmetric product orbifolds and free non-abelian gauge theories. We give a general algorithm to calculate the leading large N contribution to four-point correlators of twist fields., Comment: 43 pages, 19 figures, v2: references added

We study correlation functions of single-cycle chiral operators in the symmetric product orbifold of N supersymmetric four-tori. Correlators of twist operators are evaluated on covering surfaces, generally of different genera, where fields are single-valued. We compute some simple four-point functions and study how the sum over inequivalent branched covering maps splits under OPEs. We then discuss extremal n-point correlators, i.e. correlators of n-1 chiral and one anti-chiral operators. They obey simple recursion relations involving numbers obtained from counting branched covering maps with particular properties. In most cases we are able to solve explicitly the recursion relations. Remarkably, extremal correlators turn out to be equal to Hurwitz numbers., Comment: 36 pages, 3 figures, v2: minor improvements

We study the entanglement entropy in confining theories with gravity duals using the holographic prescription of Ryu and Takayanagi. The entanglement entropy between a region and its complement is proportional to the minimal area of a bulk hypersurface ending on their border. We consider a disk in 2+1 dimensions and a ball in 3+1 dimensions and find in both cases two types of bulk hypersurfaces with different topology, similar to the case of the slab geometry considered by Klebanov, Kutasov and Murugan. Depending on the value of the radius, one or the other type of hypersurfaces dominates the calculation of entanglement entropy. In 2+1 dimensions a useful measure of topological order of the ground state is the topological entanglement entropy, which is defined to be the constant term in the entanglement entropy of a disk in the limit of large radius. We compute this quantity and find that it vanishes for confining gauge theory, in accord with our expectations. In 3+1 dimensions the analogous quantity is shown to be generically nonzero and cutoff-dependent., Comment: harvmac, 26 pages, 14 figures, references added

We study the spectral flowed sectors of the H3 WZW model in the context of the holographic duality between type IIB string theory in AdS(3)x S^3 x T^4 with NSNS flux and the symmetric product orbifold of T^4. We construct explicitly the physical vertex operators in the flowed sectors that belong to short representations of the superalgebra, thus completing the bulk-to-boundary dictionary for 1/2 BPS states. We perform a partial calculation of the string three-point functions of these operators. A complete calculation would require the three-point couplings of non-extremal flowed operators in the H3 WZW model, which are at present unavailable. In the unflowed sector, perfect agreement has recently been found between the bulk and boundary three-point functions of 1/2 BPS operators. Assuming that this agreement persists in the flowed sectors, we determine certain unknown three-point couplings in the H3 WZW model in terms of three-point couplings of affine descendants in the SU(2) WZW model., Comment: 50 pages, 2 figures

We extend to chiral N=4 operators the holographic agreement recently found between correlators of the symmetric orbifold of M^4 at large N and type IIB strings propagating in AdS(3) x S^3 x M^4, where M^4=T^4 or K3. We also present expressions for some bulk correlators not yet computed in the boundary., Comment: 3 pages. v2:minor changes

We carry out an exact worldsheet computation of tree level three-point correlators of chiral operators in string theory on AdS(3) x S^3 x T^4 with NS-NS flux. We present a simple representation for the string chiral operators in the coordinate basis of the dual boundary CFT. Striking cancelations occur between the three-point functions of the H3+ and the SU(2) WZW models which result in a simple factorized form for the final correlators. We show, by fixing a single free parameter in the H3+ WZW model, that the fusion rules and the structure constants of the N=2 chiral ring in the bulk are in precise agreement with earlier computations in the boundary CFT of the symmetric product of T^4 at the orbifold point in the large N limit., Comment: 47 pages, JHEP style. v2:references added, minor changes. v3:references added

We study the duality between the two dimensional black hole and the sine-Liouville conformal field theories via exact operator quantization of a classical scattering problem. The ideas are first illustrated in Liouville theory, which is dual to itself under the interchange of the Liouville parameter b by 1/b. In both cases, a classical scattering problem does not determine uniquely the quantum reflection coefficient. The latter is only fixed by assuming that the dual scattering problem has the same reflection coefficient. We also discuss the relation of this approach to the method that exploits the parafermionic symmetry of the model to compute the reflection coefficient., Comment: 19 pages, JHEP style. v2: Minor changes in the proposed field of sine-Liouville type, new section discussing the relation with parafermionic symmetry, references added

The duality between the Sine-Liouville conformal field theory and the two dimensional black hole is revisited by considering the two possible Sine-Liouville dressings together. We show that this choice is consistent with the structure of correlation functions, and that the OPE of the two dressings yields the black hole deformation operator. As an application of this approach, we investigate the role of higher winding perturbations in the context of c=1 strings, where we argue that they are related to higher-spin discrete states that generalize the 2d black hole operator., Comment: 23 pages, JHEP style. v2: small modifications to better clarify some of the arguments

We show that the crossing symmetry of the four-point function in the Liouville conformal field theory on the sphere contains more information than what was hitherto considered. Under certain assumptions, it provides the special structure constants that were previously computed perturbatively and allows to solve the theory without using the Liouville interaction., Comment: 9 pages

We analyze in detail the D-branes in the near-horizon limit of NS5-branes on a circle, the holographic dual of little string theory in a double scaling limit. We emphasize their geometry in the background of the NS5-branes and show the relation with D-branes in coset models. The exact one-point function giving the coupling of the closed string states with the D-branes and the spectrum of open strings is computed. Using these results, we analyze several aspects of Hanany-Witten setups, using exact CFT analysis. In particular we identify the open string spectrum on the D-branes stretched between NS5-branes which confirms the low-energy analysis in brane constructions, and that allows to go to higher energy scales. As an application we show the emergence of the beta-function of the N=2 gauge theory on D4-branes stretching between NS5-branes from the boundary states describing the D4-branes. We also speculate on the possibility of getting a matrix model description of little string theory from the effective theory on the D1-branes. By considering D3-branes orthogonal to the NS5-branes we find a CFT incarnation of the Hanany-Witten effect of anomalous creation of D-branes. Finally we give an brief description of some non-BPS D-branes., Comment: JHEP class; 61 pages, 12 figures. v2: small corrections, references added; v3: some corrections in the non-BPS branes section, published version

We study D-branes in the mirror pair N=2 Liouville / supersymmetric SL(2,R)/U(1) coset superconformal field theories. After revisiting the duality between the two models, we build D0, D1 and D2 branes, on the basis of the boundary state construction for the Euclidean AdS(3) conformal field theory. We also construct D0-branes in an orbifold that rotates the angular direction of the cigar. We show how the poles of correlators associated to localized states and bulk interactions naturally decouple in the one-point functions of localized and extended branes. We stress the role played in the analysis of D-brane spectra by primaries in SL(2,R)/U(1) which are descendents of the parent theory., Comment: JHEP class, 46 pages, 2 figures. v2: typos corrected, references addded

We compute the partition function of the supersymmetric two-dimensional Euclidean black hole geometry described by the SL(2,R)/U(1) superconformal field theory. We decompose the result in terms of characters of the N=2 superconformal symmetry. We point out puzzling sectors of states besides finding expected discrete and continuous contributions to the partition function. By adding an N=2 minimal model factor of the correct central charge and projecting on integral N=2 charges we compute the partition function of the background dual to little string theory in a double scaling limit. We show the precise correspondence between this theory and the background for NS5-branes on a circle, due to an exact description of the background as a null gauging of SL(2,R) x SU(2). Finally, we discuss the interplay between GSO projection and target space geometry., Comment: JHEP class, 35 pages, no figures; v2: minor changes, typos corrected, published version

We define extended SL(2,R)/U(1) characters which include a sum over winding sectors. By embedding these characters into similarly extended characters of N=2 algebras, we show that they have nice modular transformation properties. We calculate the modular matrices of this simple but non-trivial non-rational conformal field theory explicitly . As a result, we show that discrete SL(2,R) representations mix with continuous SL(2,R) representations under modular transformations in the coset conformal field theory. We comment upon the significance of our results for a general theory of non-rational conformal field theories., Comment: JHEP style, 25 pages, 2 figures, v2: minor corrections, reference added, version to appear in JHEP

We study some aspects of type 0 strings propagating in the two dimensional black hole geometry, corresponding to the exact SL(2)/U(1) SCFT background., Comment: 33 pages, harvmac. v2: minor clarification added

We study the BRST quantization of bosonic and NSR strings propagating in AdS(3) x N backgrounds. The no-ghost theorem is proved using the Frenkel-Garland-Zuckerman method. Regular and spectrally-flowed representations of affine SL(2,R) appear on an equal footing. Possible generalizations to related curved backgrounds are discussed., Comment: JHEP style, 23 pages; v2:minor changes and references added; v3: typos corrected, version to appear in JHEP; v4: one reference added

We study superstring theories on AdS(3) x N backgrounds yielding N=2,3,4 extended superconformal symmetries in the dual boundary CFT. In each case the necessary constraints on the internal worldsheet theory N are found., Comment: JHEP style, 22 pages; v2: new references and a couple of sentences on N=1 are added

The unitarity of the NS supersymmetric coset SL(2,R)/U(1) is studied for the discrete representations. The results are applied to the proof of the no-ghost theorem for fermionic strings in AdS(3) x N in the NS sector. A no-ghost theorem is proved for states in flowed discrete representations., Comment: LaTeX in JHEP style, 16 pages, typos corrected