Search

We systematically study various network Expectation-Maximization (EM) algorithms for the Gaussian mixture model within the framework of decentralized federated learning. Our theoretical investigation reveals that directly extending the classical decentralized supervised learning method to the EM algorithm exhibits poor estimation accuracy with heterogeneous data across clients and struggles to converge numerically when Gaussian components are poorly-separated. To address these issues, we propose two novel solutions. First, to handle heterogeneous data, we introduce a momentum network EM (MNEM) algorithm, which uses a momentum parameter to combine information from both the current and historical estimators. Second, to tackle the challenge of poorly-separated Gaussian components, we develop a semi-supervised MNEM (semi-MNEM) algorithm, which leverages partially labeled data. Rigorous theoretical analysis demonstrates that MNEM can achieve statistical efficiency comparable to that of the whole sample estimator when the mixture components satisfy certain separation conditions, even in heterogeneous scenarios. Moreover, the semi-MNEM estimator enhances the convergence speed of the MNEM algorithm, effectively addressing the numerical convergence challenges in poorly-separated scenarios. Extensive simulation and real data analyses are conducted to justify our theoretical findings.

Spatial autoregressive (SAR) models are important tools for studying network effects. However, with an increasing emphasis on data privacy, data providers often implement privacy protection measures that make classical SAR models inapplicable. In this study, we introduce a privacy-protected SAR model with noise-added response and covariates to meet privacy-protection requirements. However, in this scenario, the traditional quasi-maximum likelihood estimator becomes infeasible because the likelihood function cannot be directly formulated. To address this issue, we first consider an explicit expression for the likelihood function with only noise-added responses. Then, we develop techniques to correct the biases for derivatives introduced by noise. Correspondingly, a Newton-Raphson-type algorithm is proposed to obtain the estimator, leading to a corrected likelihood estimator. To further enhance computational efficiency, we introduce a corrected least squares estimator based on the idea of bias correction. These two estimation methods ensure both data security and the attainment of statistically valid estimators. Theoretical analysis of both estimators is carefully conducted, statistical inference methods and model extensions are discussed. The finite sample performances of different methods are demonstrated through extensive simulations and the analysis of a real dataset.

This paper presents a selective review of statistical computation methods for massive data analysis. A huge amount of statistical methods for massive data computation have been rapidly developed in the past decades. In this work, we focus on three categories of statistical computation methods: (1) distributed computing, (2) subsampling methods, and (3) minibatch gradient techniques. The first class of literature is about distributed computing and focuses on the situation, where the dataset size is too huge to be comfortably handled by one single computer. In this case, a distributed computation system with multiple computers has to be utilized. The second class of literature is about subsampling methods and concerns about the situation, where the sample size of dataset is small enough to be placed on one single computer but too large to be easily processed by its memory as a whole. The last class of literature studies those minibatch gradient related optimization techniques, which have been extensively used for optimizing various deep learning models.

Distributed computing is critically important for modern statistical analysis. Herein, we develop a distributed quasi-Newton (DQN) framework with excellent statistical, computation, and communication efficiency. In the DQN method, no Hessian matrix inversion or communication is needed. This considerably reduces the computation and communication complexity of the proposed method. Notably, related existing methods only analyze numerical convergence and require a diverging number of iterations to converge. However, we investigate the statistical properties of the DQN method and theoretically demonstrate that the resulting estimator is statistically efficient over a small number of iterations under mild conditions. Extensive numerical analyses demonstrate the finite sample performance., Comment: 56 pages, 3 figures

Modern statistical analysis often encounters datasets with large sizes. For these datasets, conventional estimation methods can hardly be used immediately because practitioners often suffer from limited computational resources. In most cases, they do not have powerful computational resources (e.g., Hadoop or Spark). How to practically analyze large datasets with limited computational resources then becomes a problem of great importance. To solve this problem, we propose here a novel subsampling-based method with jackknifing. The key idea is to treat the whole sample data as if they were the population. Then, multiple subsamples with greatly reduced sizes are obtained by the method of simple random sampling with replacement. It is remarkable that we do not recommend sampling methods without replacement because this would incur a significant cost for data processing on the hard drive. Such cost does not exist if the data are processed in memory. Because subsampled data have relatively small sizes, they can be comfortably read into computer memory as a whole and then processed easily. Based on subsampled datasets, jackknife-debiased estimators can be obtained for the target parameter. The resulting estimators are statistically consistent, with an extremely small bias. Finally, the jackknife-debiased estimators from different subsamples are averaged together to form the final estimator. We theoretically show that the final estimator is consistent and asymptotically normal. Its asymptotic statistical efficiency can be as good as that of the whole sample estimator under very mild conditions. The proposed method is simple enough to be easily implemented on most practical computer systems and thus should have very wide applicability.

We study a fully decentralized federated learning algorithm, which is a novel gradient descent algorithm executed on a communication-based network. For convenience, we refer to it as a network gradient descent (NGD) method. In the NGD method, only statistics (e.g., parameter estimates) need to be communicated, minimizing the risk of privacy. Meanwhile, different clients communicate with each other directly according to a carefully designed network structure without a central master. This greatly enhances the reliability of the entire algorithm. Those nice properties inspire us to carefully study the NGD method both theoretically and numerically. Theoretically, we start with a classical linear regression model. We find that both the learning rate and the network structure play significant roles in determining the NGD estimator's statistical efficiency. The resulting NGD estimator can be statistically as efficient as the global estimator, if the learning rate is sufficiently small and the network structure is well balanced, even if the data are distributed heterogeneously. Those interesting findings are then extended to general models and loss functions. Extensive numerical studies are presented to corroborate our theoretical findings. Classical deep learning models are also presented for illustration purpose.

Improving the clock stability is of fundamental importance for the development of quantum-enhanced metrology. One of the main limitations arises from the randomly-fluctuating local oscillator (LO) frequency, which introduces "phase slips" for long interrogation times and hence failure of the frequency-feedback loop. Here we propose a strategy to improve the stability of atomic clocks by interrogating two out-of-phase state sharing the same LO. While standard Ramsey interrogation can only determine phases unambiguously in the interval $[-\pi/2,\pi/2]$, the joint interrogation allows for an extension to $[-\pi,\pi]$, resulting in a relaxed restriction of the Ramsey time and improvement of absolute clock stability. Theoretical predictions are supported by ab-initio numerical simulation for white and correlated LO noise. While our basic protocol uses uncorrelated atoms, we have further extended it to include spin-squeezing and further improving the scaling of clock stability with the number of atoms. Our protocol can be readily tested in current state-of-the-art experiments., Comment: 15 pages, 10 figures

We investigate the non-equilibrium dynamics across the miscible-immiscible phase separation in a binary mixture of Bose-Einstein condensates. The excitation spectra reveal that the Landau critical velocity vanishes at the critical point, where the superfluidity spontaneously breaks down. We analytically extract the dynamical critical exponent $z=2$ from the Landau critical velocity. Moreover, by simulating the real-time dynamics across the critical point, we find the average domain number and the average bifurcation delay show universal scaling laws with respect to the quench time. We then numerically extract the static correlation length critical exponent $v=1/2$ and the dynamical critical exponent $z=2$ according to Kibble-Zurek mechanism. The scaling exponents $(v=1/2, z=2)$ in the phase separation driven by quenching the atom-atom interaction are different from the ones $(v=1/2, z=1)$ in the phase separation driven by quenching the Rabi coupling strength [PRL \textbf{102}, 070401 (2009); PRL \textbf{107}, 230402 (2011)]. Our study explores the connections between the spontaneous superfluidity breakdown and the spontaneous defect formation in the phase separation dynamics., Comment: 13 pages, 5 figures

We investigate the universal spatiotemporal dynamics in spin-orbit coupled Bose-Einstein condensates which are driven from the zero-momentum phase to the plane-wave phase. The excitation spectrum reveals that, at the critical point, the Landau critical velocity vanishes and the correlation length diverges. Therefore, according to the Kibble-Zurek mechanism, spatial domains will spontaneously appear in such a quench through the critical point. By simulating the real-time dynamics, we numerically extract the static correlation length critical exponent v and the dynamic critical exponent z from the scalings of the temporal bifurcation delay and the spatial domain number. The numerical scalings consist well with the analytical ones obtained by analyzing the excitation spectrum.

According to the famous Kibble-Zurek mechanism (KZM), the universality of spontaneous defect generation in continuous phase transitions (CPTs) can be understood by the critical slowing down. In most CPTs of atomic Bose-Einstein condensates (BECs), the universality of spontaneous defect generations has been explained by the divergent relaxation time associated with the nontrivial gapless Bogoliubov excitations. However, for atomic BECs in synthetic gauge fields, their spontaneous superfluidity breakdown is resulted from the divergent correlation length associated with the zero Landau critical velocity. Here, by considering an atomic BEC ladder subjected to a synthetic magnetic field, we reveal that the spontaneous superfluidity breakdown obeys the KZM. The Kibble-Zurek scalings are derived from the Landau critical velocity which determines the correlation length. In further, the critical exponents are numerically extracted from the critical spatial-temporal dynamics of the bifurcation delay and the spontaneous vortex generation. Our study provides a general way to explore and understand the spontaneous superfluidity breakdown in CPTs from a single-well dispersion to a double-well one, such as, BECs in synthetic gauge fields, spin-orbit coupled BECs, and BECs in shaken optical lattices., Comment: accepted for publication in Phys. Rev. A

Universal dynamics of spontaneous symmetry breaking is central to understanding the universal behavior of spontaneous defect formation in various system from the early universe, condensed-matter systems to ultracold atomic systems. We explore the universal real-time dynamics in an array of coupled binary atomic Bose-Einstein condensates in optical lattices, which undergo a spontaneous symmetry breaking from the symmetric Rabi oscillation to the broken-symmetry self-trapping. In addition to Goldstone modes, there exist gapped Higgs mode whose excitation gap vanishes at the critical point. In the slow passage through the critical point, we analytically find that the symmetry-breaking dynamics obeys the Kibble-Zurek mechanism. From the scalings of bifurcation delay and domain formation, we numerically extract two Kibble-Zurek exponents $b_{1}=\nu/(1+\nu z)$ and $b_{2}=1/(1+\nu z)$, which give the static correlation-length critical exponent $\nu$ and the dynamic critical exponent $z$. Our approach provides an efficient way to simultaneous determination of the critical exponents $\nu$ and $z$ for a continuous phase transition., Comment: 6 pages, 4 figures, accepted for publication in EPL (Europhysics Letters)

Quantum metrology is the science that aims to achieve precision measurements by making use of quantum principles. Attribute to the well-developed techniques of manipulating and detecting cold atoms, cold atomic systems provide an excellent platform for implementing precision quantum metrology. In this chapter, we review the general procedures of quantum metrology and some experimental progresses in quantum metrology with cold atoms. Firstly, we give the general framework of quantum metrology and the calculation of quantum Fisher information, which is the core of quantum parameter estimation. Then, we introduce the quantum interferometry with single and multiparticle states. In particular, for some typical multiparticle states, we analyze their ultimate precision limits and show how quantum entanglement could enhance the measurement precision beyond the standard quantum limit. Further, we review some experimental progresses in quantum metrology with cold atomic systems., Comment: 53 pages, 9 figures, revised version