Search

This paper considers data-driven chance-constrained stochastic optimization problems in a Bayesian framework. Bayesian posteriors afford a principled mechanism to incorporate data and prior knowledge into stochastic optimization problems. However, the computation of Bayesian posteriors is typically an intractable problem and has spawned a large literature on approximate Bayesian computation. Here, in the context of chance-constrained optimization, we focus on the question of statistical consistency (in an appropriate sense) of the optimal value, computed using an approximate posterior distribution. To this end, we rigorously prove a frequentist consistency result demonstrating the convergence of the optimal value to the optimal value of a fixed, parameterized constrained optimization problem. We augment this by also establishing a probabilistic rate of convergence of the optimal value. We also prove the convex feasibility of the approximate Bayesian stochastic optimization problem. Finally, we demonstrate the utility of our approach on an optimal staffing problem for an M/M/c queueing model. [ABSTRACT FROM AUTHOR]

In episodic memory tasks, associations are formed between items presented close together in time. The temporal context model (TCM) hypothesizes that this contiguity effect is a consequence of shared temporal context rather than temporal proximity per se. Using double-function lists of paired associates (e.g., A-B, B-C) presented in a random order, the authors examined associations between items that were not presented close together in time but that were presented in similar temporal contexts. After learning, across-pair associations fell off with distance in the list, as if subjects had integrated the pairs into a coherent memory structure. Within-pair associations (e.g., A-B) were strongly asymmetric favoring forward transitions; across-pair associations (e.g., A-C) showed no evidence of asymmetry. While this pattern of results presented a stern challenge for a heteroassociative mediated chaining model, TCM provided an excellent fit to the data. These findings suggest that contiguity effects in episodic memory do not reflect direct associations between items but rather a process of binding, encoding, and retrieval of a gradually changing representation of temporal context. (Contains 6 figures, 2 tables and 7 footnotes.)

While differential privacy (DP) offers strong theoretical privacy guarantees, implementations of DP mechanisms may be vulnerable to side-channel attacks, such as timing attacks. When sampling methods such as MCMC or rejection sampling are used to implement a privacy mechanism, the runtime can leak private information. We characterize the additional privacy cost due to the runtime of a rejection sampler in terms of both (ϵ, δ)-DP as well as f-DP. We also show that unless the acceptance probability is constant across databases, the runtime of a rejection sampler does not satisfy ϵ-DP for any ϵ. We show that there is a similar breakdown in privacy with adaptive rejection samplers. We propose three modifications to the rejection sampling algorithm, with varying assumptions, to protect against timing attacks by making the runtime independent of the data. The modification with the weakest assumptions is an approximate sampler, introducing a small increase in the privacy cost, whereas the other modifications give perfect samplers. We also use our techniques to develop an adaptive rejection sampler for log-Hölder densities, which also has data-independent runtime. We give several examples of DP mechanisms that fit the assumptions of our methods and can thus be implemented using our samplers. [ABSTRACT FROM AUTHOR]

In episodic memory tasks, associations are formed between items presented close together in time. The temporal context model (TCM) hypothesizes that this contiguity effect is a consequence of shared temporal context rather than temporal proximity per se. Using double-function lists of paired associates (e.g., A-B, B-C) presented in a random order, the authors examined associations between items that were not presented close together in time but that were presented in similar temporal contexts. After learning, across-pair associations fell off with distance in the list, as if subjects had integrated the pairs into a coherent memory structure. Within-pair associations (e.g., A-B) were strongly asymmetric favoring forward transitions; across-pair associations (e.g., A-C) showed no evidence of asymmetry. While this pattern of results presented a stern challenge for a heteroassociative mediated chaining model, TCM provided an excellent fit to the data. These findings suggest that contiguity effects in episodic memory do not reflect direct associations between items but rather a process of binding, encoding, and retrieval of a gradually changing representation of temporal context. Keywords: episodic memory, semantic memory, contiguity effect, mathematical models of memory

Stochastic differential equations (SDEs) or diffusions are continuous-valued continuous-time stochastic processes widely used in the applied and mathematical sciences. Simulating paths from these processes is usually an intractable problem, and typically involves time-discretization approximations. We propose an exact Markov chain Monte Carlo sampling algorithm that involves no such time-discretization error. Our sampler is applicable to the problem of prior simulation from an SDE, posterior simulation conditioned on noisy observations, as well as parameter inference given noisy observations. Our work recasts an existing rejection sampling algorithm for a class of diffusions as a latent variable model, and then derives an auxiliary variable Gibbs sampling algorithm that targets the associated joint distribution. At a high level, the resulting algorithm involves two steps: simulating a random grid of times from an inhomogeneous Poisson process, and updating the SDE trajectory conditioned on this grid. Our work allows the vast literature of Monte Carlo sampling algorithms from the Gaussian process literature to be brought to bear to applications involving diffusions. We study our method on synthetic and real datasets, where we demonstrate superior performance over competing methods. for this article are available online. [ABSTRACT FROM AUTHOR]

Markov jump processes (MJPs) are continuous-time stochastic processes widely used in a variety of applied disciplines. Inference typically proceeds via Markov chain Monte Carlo (MCMC), the state-of-the-art being a uniformization-based auxiliary variable Gibbs sampler. This was designed for situations where the process parameters are known, and Bayesian inference over unknown parameters is typically carried out by incorporating it into a larger Gibbs sampler. This strategy of sampling parameters given path, and path given parameters can result in poor Markov chain mixing. In this work, we propose a simple and efficient algorithm to address this problem. Our scheme brings Metropolis–Hastings (MH) approaches for discrete-time hidden Markov models to the continuous-time setting, resulting in a complete and clean recipe for parameter and path inference in MJPs. In our experiments, we demonstrate superior performance over Gibbs sampling, a more naïve MH algorithm, as well as another popular approach, particle MCMC. We also show our sampler inherits geometric mixing from an "ideal" sampler that is computationally much more expensive. for this article are available online. [ABSTRACT FROM AUTHOR]

We study the asymptotic consistency properties of α-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the α-Renyi divergence from the true posterior. Unique to our work is that we consider settings with α > 1, resulting in approximations that upperbound the log-likelihood, and consequently have wider spread than traditional variational approaches that minimize the Kullback-Liebler (KL) divergence from the posterior. Our primary result identifies sufficient conditions under which consistency holds, centering around the existence of a 'good' sequence of distributions in the approximating family that possesses, among other properties, the right rate of convergence to a limit distribution. We further characterize the good sequence by demonstrating that a sequence of distributions that converges too quickly cannot be a good sequence. We also extend our analysis to the setting where a equals one, corresponding to the minimizer of the reverse KL divergence, and to models with local latent variables. We also illustrate the existence of good sequence with a number of examples. Our results complement a growing body of work focused on the frequentist properties of variational Bayesian methods. [ABSTRACT FROM AUTHOR]

This paper establishes the asymptotic consistency of the loss-calibrated variational Bayes (LCVB) method. LCVB is a method for approximately computing Bayesian posterior approximations in a ‘‘loss aware’’ manner. This methodology is also highly relevant in general data-driven decision-making contexts. Here, we establish the asymptotic consistency of both the loss- calibrated approximate posterior and the resulting decision rules. We also establish the asymptotic consistency of decision rules obtained from a‘‘naive’’ two-stage procedure that first computes a standard variational Bayes approximation and then uses this in the decision-making procedure. [ABSTRACT FROM AUTHOR]

The Stiefel manifold Vp,d is the space of all d × p orthonormal matrices, with the d-1 hypersphere and the space of all orthogonal matrices constituting special cases. In modeling data lying on the Stiefel manifold, parametric distributions such as the matrix Langevin distribution are often used; however, model misspecification is a concern and it is desirable to have nonparametric alternatives. Current nonparametric methods are mainly Fréchet-mean based. We take a fully generative nonparametric approach, which relies on mixing parametric kernels such as the matrix Langevin. The proposed kernel mixtures can approximate a large class of distributions on the Stiefel manifold, and we develop theory showing posterior consistency. While there exists work developing general posterior consistency results, extending these results to this particular manifold requires substantial new theory. Posterior inference is illustrated on a dataset of near-Earth objects. [ABSTRACT FROM AUTHOR]

Background: Periodontitis is bacteria-related chronic inflammatory condition characterized by pocket formation, loss of clinical attachment, gingival recession, mobility, and eventual loss of teeth. The purpose of this study was to clinically evaluate the need for elimination of the pocket epithelium during mucoperiosteal flap surgery aimed at reattachment or re-adaptation. Materials and Methods: A split mouth design was done to compare modified Widman flap (MWF) with removal of the pocket epithelium and crevicular mucoperiosteal flap (CMF) without removing the pocket epithelium. The following measurements were taken after 1 month of completion of nonsurgical phase gingival index (Loe and Silness), plaque index (Silness and Loe), mobility, furcation involvement, level of attachment, pocket depth, gingival recession, gingival contour index, and dentinal hypersensitivity (ice stick test). In addition to these measurements, which were taken immediately prior to the surgery (baseline), 1- and 3-month and 6 months postsurgical measurements were also recorded. Results: The results of this study showed a greater reduction of mean probing depth in the test group (MWF). The control group (CMF) showed greater mean gingival recession compared to the test group throughout the study period. The test group showed more gain in the clinical attachment levels compared with the control group. The difference between the two groups was statistically significant (P < 0.001). Conclusions: The results of this study demonstrate that MWF surgery was more effective in reducing mean probing depth, showed greater gain in clinical attachment, and demonstrated less gingival recession. [ABSTRACT FROM AUTHOR]

Datasets displaying temporal dependencies abound in science and engineering applications, with Markov models representing a simplified and popular view of the temporal dependence structure. In this paper, we consider Bayesian settings that place prior distributions over the parameters of the transition kernel of a Markov model, and seek to characterize the resulting, typically intractable, posterior distributions. We present a Probably Approximately Correct (PAC)-Bayesian analysis of variational Bayes (VB) approximations to tempered Bayesian posterior distributions, bounding the model risk of the VB approximations. Tempered posteriors are known to be robust to model misspecification, and their variational approximations do not suffer the usual problems of over confident approximations. Our results tie the risk bounds to the mixing and ergodic properties of the Markov data generating model. We illustrate the PAC-Bayes bounds through a number of example Markov models, and also consider the situation where the Markov model is misspecified. [ABSTRACT FROM AUTHOR]

Semantic memory refers to our knowledge of facts and relationships between concepts. A successful semantic memory depends on inferring relationships between items that are not explicitly taught. Recent mathematical modeling of episodic memory argues that episodic recall relies on retrieval of a gradually-changing representation of temporal context. We show that retrieved context enables the development of a global memory space that reflects relationships between all items that have been previously learned. When newly-learned information is integrated into this structure, it is placed in some relationship to all other items, even if that relationship has not been explicitly learned. We demonstrate this effect for global semantic structures shaped topologically as a ring, and as a two-dimensional sheet. We also examined the utility of this learning algorithm for learning a more realistic semantic space by training it on a large pool of synonym pairs. Retrieved context enabled the model to "infer" relationships between synonym pairs that had not yet been presented.