Search

The coming big data era brings data of unprecedented size and launches an innovation of learning algorithms in statistical and machine-learning communities. The classical kernel-based regularized least-squares (RLS) algorithm is excluded in the innovation, due to its computational and storage bottlenecks. This article presents a scalable algorithm based on subsampling, called learning with selected features (LSF), to reduce the computational burden of RLS. Almost the optimal learning rate together with a sufficient condition on selecting kernels and centers to guarantee the optimality is derived. Our theoretical assertions are verified by numerical experiments, including toy simulations, UCI standard data experiments, and a real-world massive data application. The studies in this article show that LSF can reduce the computational burden of RLS without sacrificing its generalization ability very much.

Eigenspace estimation is a fundamental tool in data analytics, which has found applications in PCA, dimension reduction, and clustering, among others. The modern machine learning community usually involves data that come from and belong to different organizations. The low communication power and possible data privacy breaches make the eigenspace estimation challenging. To address these issues, we propose a class of algorithms called FedPower within the federated learning (FL) framework. FedPower leverages the well-known power method by alternating multiple local power iterations and a global aggregation step, thus improving communication efficiency. In the aggregation, we propose to weight each local eigenvector matrix with Orthogonal Procrustes Transformation (OPT) for better alignment. We add Gaussian noise in each iteration to ensure strong privacy protection by adopting the notion of differential privacy (DP). We provide convergence bounds for FedPower composed of different interpretable terms corresponding to the effects of Gaussian noise, parallelization, and random sampling of local machines. Additionally, we conduct experiments to demonstrate the effectiveness of our proposed algorithms. [ABSTRACT FROM AUTHOR]

Measuring the impact of extreme winds is important in high-speed railway bridge construction to avoid the risk of engineering accidents. This study presents an early-warning framework for high-speed railway (HSR) bridge construction under extreme wind conditions, using a long-span continuous beam bridge constructed in a typhoon-prone area as a case study. Specifically, on-site wind environment measurements during a typhoon are utilized, and parameters such as fluctuating wind turbulence intensity, gust factor, turbulence integral scale, and power spectral density are employed to characterize the wind environment. Multi-step prediction of gust wind speed during the bridge construction is performed using the hybrid Back Propagation-Genetic Algorithm (BP-GA). Hierarchical warnings and control actions are proposed and implemented based on the prediction results. The analysis results of wind parameters revealed a discrepancy between the measured and specified typhoon turbulence intensities, with the windward turbulence intensity being lower than specified. The longitudinal average gust factor exceeded the transverse value. The prediction curve based on the BP-GA algorithm closely resembles the actual curve, meeting accuracy requirements. Notably, the one-step prediction provided the most accurate results. Based on the predicted wind speed trend for the upcoming half-hour, appropriate hierarchical warnings and control measures were conducted. [ABSTRACT FROM AUTHOR]

Efficient determination of the rock mass properties is vitally important for calculating and evaluating tunnel stability in tunnel engineering. The back analysis method has been widely used as an indirect method for determining rock mass parameters based on field measurements. However, most back analysis methods are generally time-consuming for numerical simulation and are merely based on the measured displacement, which leads to the identification of rock mass parameters that cannot fully reflect the characteristics of the surrounding rock. To improve the accuracy of the estimation of rock mass parameters, this paper presents a back analysis method based on multi-output support vector regression (MSVR) and differential evolution (DE) algorithms. Firstly, the global sensitivity analysis of rock mass parameters is analyzed using the elementary effects method. Numerical simulation is then carried out to prepare training samples. DE algorithm is used to determine the optimum hyperparameters of MSVR. Based on the monitoring data, the rock mass properties of the selected sensitive parameters are estimated by the constructed MSVR model. A high-speed railway tunnel is utilized to demonstrate the effectiveness of the MSVR with the DE algorithm (DE-MSVR). The results show that the DE-MSVR with mixed monitoring data of vault settlement, convergence, and rock mass stress has higher forecasting performance than these models with a single type of monitoring data. It is feasible to use the monitoring data at the early stages combined with the numerical simulation for parameter back analysis. Moreover, the comparison results show that the presented method exhibits higher prediction accuracy than the existing back analysis models. [ABSTRACT FROM AUTHOR]

The previous results describing the generalization ability of Empirical Risk Minimization (ERM) algorithm are usually based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by establishing the first exponential bound on the rate of uniform convergence of the ERM algorithm with V-geometrically ergodic Markov chain samples, as the application of the bound on the rate of uniform convergence, we also obtain the generalization bounds of the ERM algorithm with V-geometrically ergodic Markov chain samples and prove that the ERM algorithm with V-geometrically ergodic Markov chain samples is consistent. The main results obtained in this paper extend the previously known results of i.i.d. observations to the case of V-geometrically ergodic Markov chain samples. [ABSTRACT FROM AUTHOR]

A main issue in machine learning research is to analyze the generalization performance of a learning machine. Most classical results on the generalization performance of regularization algorithms are derived merely with the complexity of hypothesis space or the stability property of a learning algorithm. However, in practical applications, the performance of a learning algorithm is not actually affected only by an unitary factor just like the complexity of hypothesis space, stability of the algorithm and data quality. Therefore, in this paper, we develop a framework of evaluating the generalization performance of regularization algorithms combinatively in terms of hypothesis space complexity, algorithmic stability and data quality. We establish new bounds on the learning rate of regularization algorithms based on the measure of uniform stability and empirical covering number for general type of loss functions. As applications of the generic results, we evaluate the learning rates of support vector machines and regularization networks, and propose a new strategy for regularization parameter setting. [ABSTRACT FROM AUTHOR]

The existing studies involving matrix or tensor completion problems are commonly under the nuclear norm penalization framework due to the computational efficiency of the resulting convex optimization problem. Folded-concave penalization methods have demonstrated surprising developments in sparse learning problems due to their nice practical and theoretical properties. To share the same light of folded-concave penalization methods, we propose a new tensor completion model via folded-concave penalty for estimating missing values in tensor data. Two typical folded-concave penalties, the minmax concave plus (MCP) penalty and the smoothly clipped absolute deviation (SCAD) penalty, are employed in the new model. To solve the resulting nonconvex optimization problem, we develop a local linear approximation augmented Lagrange multiplier (LLA-ALM) algorithm which combines a two-step LLA strategy to search a local optimum of the proposed model efficiently. Finally, we provide numerical experiments with phase transitions, synthetic data sets, real image and video data sets to exhibit the superiority of the proposed model over the nuclear norm penalization method in terms of the accuracy and robustness. [ABSTRACT FROM AUTHOR]