Search

Kernel methods are often used for nonlinear regression and classification in statistics and machine learning because they are computationally cheap and accurate. The wavelet kernel functions based on wavelet analysis can efficiently approximate any nonlinear functions. In this article, we construct a novel wavelet kernel function in terms of random wavelet bases and define a linear vector space that captures nonlinear structures in reproducing kernel Hilbert spaces (RKHS). Based on the wavelet transform, the data are mapped into a low-dimensional randomized feature space and convert kernel function into operations of a linear machine. We then propose a new Bayesian approximate kernel model with the random wavelet expansion and use the Gibbs sampler to compute the model's parameters. Finally, some simulation studies and two real datasets analyses are carried out to demonstrate that the proposed method displays good stability, prediction performance compared to some other existing methods. [ABSTRACT FROM AUTHOR]

In this paper, a new approach is presented to fit a fuzzy regression model with the fuzzy coefficients when the explanatory variables and the response variable are as fuzzy numbers. In this approach, a nonlinear function is introduced to predict the response variables based on the kernel function. To estimate the parameters of regression model, the objective function is calculated using the quantile loss function on fuzzy numbers. To evaluate the goodness of fit of the optimal quantile fuzzy regression models, two indices are introduced using the similarity measures. Based on the presented results, the proposed fuzzy regression models have the perfect performances on the original data and also in the presences of different types of outliers. [ABSTRACT FROM AUTHOR]

In this paper, Gaussian Process Regression (GPR)‐based models which use the Bayesian approach to regression analysis problem such as load forecasting (LF) are proposed. The GPR is a non‐parametric kernel‐based learning method having the ability to provide correct predictions with uncertainty in measurements. The proposed model provides an hourly and monthly load forecast for an Australian city and four Indian cities in the Maharashtra state. Twelve GPR models are trained with historical datasets including hourly load and environmental data. To evaluate the trained model, the actual and predicted load demand curve is plotted and mean average percentage error (MAPE) is calculated corresponding to different kernel functions of the GPR model. To the best of the author's knowledge, the prediction of load demand using GPR for Indian cities of Maharashtra state has been made for the first time. The calculated MAPE in LF is 0.15% for Australia and 0.002%, 0.209%, 0.077%, and 0.140% for Indian cities viz. Nasik, Bhusawal, Kolhapur, and Aurangabad, respectively. The test results illustrate that minimum MAPE in load prediction is obtained using the proposed model that is GPR with 'Exponential' kernel functions. Furthermore, the comparative analysis with the existing approaches confirms the dominance of the proposed model. [ABSTRACT FROM AUTHOR]

Remaining useful life (RUL) of lithium-ion batteries is an important indicator for battery health management, and accurate prediction can promote reliable battery system design, as well as safety and effectiveness of practical use. Therefore, we extract the health factor during charging and a multi-kernel support vector regression (MKSVR) RUL prediction model to achieve high accuracy estimation of the RUL of lithium-ion batteries. Firstly, based on the current, voltage, and temperature data during charging, seven characteristic parameters that can reflect the battery capacity decay are extracted, and then, three health factors (HF) that are highly correlated with the capacity decay are screened out using the Pearson coefficients. Secondly, the Gray wolf cuckoo search optimization (GWOCS) model is used to realize the intelligent optimization search of the kernel function parameter combinations of the multi-kernel support vector regression, and then the RUL prediction model of the multi-kernel support vector regression is established. Finally, the validation analysis is performed based on the NASA battery aging data set. The results show that the improved multi-kernel support vector regression has higher prediction accuracy compared with the single-kernel support vector regression, and its RUL prediction errors are all less than 5 cycles, and the maximum root mean square error is all less than 0.028. [ABSTRACT FROM AUTHOR]

In this paper, a new machine learning (ML) technique is proposed that uses the fine-tuned version of support vector regression for stock forecasting of time series data. Grid search technique is applied over training dataset to select the best kernel function and to optimize its parameters. The optimized parameters are validated through validation dataset. Thus, the tuning of this parameters to their optimized value not only increases model's overall accuracy but also requires less time and memory. Further, this also minimizes the model from being data overfitted. The proposed method is used to analysis different performance parameters of stock market like up-to-daily and up-to-monthly return, cumulative monthly return, its volatility nature and the risk associated with it. Eight different large-sized datasets are chosen from different domain, and stock is predicted for each case by using the proposed method. A comparison is carried out among the proposed method and some similar methods of same interest in terms of computed root mean square error and the mean absolute percentage error. The comparison reveals the proposed method to be more accurate in predicting the stocks for the chosen datasets. Further, the proposed method requires much less time than its counterpart methods. [ABSTRACT FROM AUTHOR]

In this paper, a new hybrid method is developed that incorporates the rationale of truncated power basis (TPB) and artificial bee colony-back propagation neural networks (ABC-BPNN) into an adaptive fuzzy semi-parametric regression model yielding more accurate results and better generalization ability. The proposed adaptive fuzzy semi-parametric regression model comprises sub-model formulation and approximates the observed fuzzy outputs from the outside employing neural networks computation, such that the proposed adaptive fuzzy regression model better explains the inherent dependence and vagueness that exist in a given dataset. Using the cross-validation criterion and absolute deviation-based distance measures for LR-type fuzzy numbers, a target function optimization problem of constructing the adaptive fuzzy semi-parametric regression is performed by solving the smooth function, bandwidth of kernel function, and regression coefficients. This strategy significantly increases the goodness of fit for the proposed algorithm and offers a dependence framework among magnitude and uncertainty for the fuzzy regression model. We also use three formula measures to evaluate the fit quality of the regression results within each membership function as well as in the center or spread tendency property, respectively. The proposed algorithm is numerically evaluated on three experimental examples including a simulation study and two practical cases to prove its practicality and efficiency. Comparative analyses of our proposed method are provided to support its cogency, and the results show that our proposed model is more effective and stable than some other existing fuzzy regressions. [ABSTRACT FROM AUTHOR]

Most works on Support Vector Regression (SVR) focus on kernel or loss functions, with the corresponding support vectors obtained using a fixed-radius ε -tube, affording good predictive performance on datasets. However, the fixed radius limitation prevents the adaptive selection of support vectors according to the data distribution characteristics, compromising the performance of the SVR-based methods. Therefore, this study proposes an "Alterable ε i -Support Vector Regression" (A ε i -SVR) model by applying a novel ε , named "Alterable ε i ," to the SVR model. Based on the data point sparsity at each location, the model solves the different ε i at the corresponding position, and thus zoom-in or zoom-out the ε -tube by changing its radius. Such a variable ε -tube strategy diminishes noise and outliers in the dataset, enhancing the prediction performance of the A ε i -SVR model. Therefore, we suggest a novel non-deterministic algorithm to iteratively solve the complex problem of optimizing ε i associated with every location. Extensive experimental results demonstrate that our approach can improve the accuracy and stability on simulated and real data compared with the baseline methods. [ABSTRACT FROM AUTHOR]

Varying coefficient models have been studied for many years, and local kernel estimation is an important method to estimate the coefficient functions. Since same bandwidth parameter is used for each coefficient function, local kernel estimation is not effective when the degree of smoothness is different for each coefficient function. In this paper, a new global estimation method based on kernel function and backfitting is proposed. In this method, different bandwidth parameters can be used for each coefficient function to improve the estimation accuracy, and asymptotic properties of the estimators are proved. To reduce the computational complexity, an adaptive global estimation method, which is based on an estimation of the bandwidth ratio and has only one bandwidth parameter to be selected, is proposed. Simulation results show that the proposed methods work well, and two real data examples further demonstrate the potential of the proposed methods. [ABSTRACT FROM AUTHOR]

The interval-valued functional data belong to the category of symbolic data in which the values of each observation consist of two functions, a lower limit function and an upper limit function. This study introduces some functional non-parametric approaches to fit a functional regression model on interval-valued functional data. First, using kernel functions, two non-parametric regression models are presented to predict a scalar interval-valued response variable based on interval-valued functional predictors. For this purpose, a local linear regression and a kernel regression estimator of the Nadaraya–Watson type are proposed. These approaches are based on two independent non-parametric models, which may not hold in practice. Therefore, some non-parametric methods are introduced, including bivariate functional Nadaraya–Watson, bivariate functional locally linear smoothing and another method based on bootstrap, which maintain selecting points randomly in the intervals of the proposed data. The applicability and advantages of the proposed models are investigated through simulation experiments and a real data example. [ABSTRACT FROM AUTHOR]

Based on the approach of left and right kernel smoothing with unilateral kernel function, we, in this paper, define estimators of change point and jump size in nonparametric regression model with response missing at random. It is shown that the change point estimator is n-consistent and converges to the smallest maximizer of one-dimensional bilateral compound Poisson process, the jump size estimator is asymptotically normal. A simulation study is conducted to investigate the finite sample behavior for the proposed methods. [ABSTRACT FROM AUTHOR]

We consider a new nonparametric test for serial correlation of unknown form in the estimated residuals of a panel regression model, where individual and time effects can be fixed or random, and the panel data can be balanced or unbalanced. Our test is robust against potential weak error cross-sectional dependence and error serial dependence in higher-order moments. This is in contrast to existing tests for serial correlation in panel data models, which assume error components to be cross-sectionally and serially independent. Our test has an asymptotic N(0, 1) distribution under the null hypothesis and is consistent against serial correlation of unknown form. No common alternative is assumed and hence our test allows for substantial inhomogeneity in serial correlation across individuals. A simulation study highlights the merits of the proposed test relative to a variety of existing tests in the literature. We apply the new test to the empirical study of Wolfers on the relationship between unilateral divorce laws and divorce rates and find strong evidence against serial uncorrelatedness even controlling for the fixed effect. [ABSTRACT FROM AUTHOR]

In this paper, we propose an interval local linear method (ILLM) to fit the regression model with interval-valued explanatory and response variables. The proposed method has no restriction on the form of the regression function. Moreover, it reduces the boundary effect of interval kernel method. Some experimental studies including two simulations and four real datasets are examined to evaluate the proposed method. Experimental results show that our method has higher predictive accuracy than some existed methods, including the center and range method, the interval least absolute method, the interval kernel method, multi-output support vector regression and the interval multilayer perceptron. [ABSTRACT FROM AUTHOR]

The accurate assessment of lightning-induced over-voltages is essential for proper insulation coordination studies. In this paper, a novel machine learning based Gaussian process regression model is developed for the prediction of lightning-induced over-voltages considering wide range of ground resistivity and permittivity. The lightning-induced over-voltages have been computed using two-dimensional finite-difference time-domain technique associated with Agrawal field to line coupling model. To account for the input space variability, seven predictor variables namely: resistivity, permittivity, return stroke velocity, distance from flashpoint, return stroke peak current, front-time, and height of an overhead line are considered to determine the response variable, that is, lightning-induced over-voltages. Accordingly, the training of the Gaussian process regression model is carried out using lightning-induced over-voltages obtained from two-dimensional finite-difference time-domain technique for both first and the subsequent strokes. The estimation accuracy of model is appraised using root mean squared error indicating that the exponential kernel function has the least error than other kernel types for describing the covariance. The predictive performance is evaluated by corroborating the predictions from the model against the random test cases generated from two-dimensional finite-difference time-domain technique. The predicted results showed a good agreement with those obtained from finite-difference time-domain technique. [ABSTRACT FROM AUTHOR]

A bent line expectile regression model can describe the effect of a covariate on the response variable with two different straight lines, which intersect at an unknown change-point. Due to the existence of the change-point, the objective function of the model is not differentiable with respect to the change-point, so it cannot be solved by the method of the traditional linear expectile regression model. For this model, a new estimation method is proposed by a smoothing technique, that is, using Gaussian kernel function to approximate the indicator function in the objective function. It can not only estimate the regression coefficients and change-point location simultaneously, but also have better estimation effect, which compensates for the insufficiency of the previous estimation methods. Under the given regularity conditions, the theoretical proofs of the consistency and asymptotic normality of the proposed estimators are derived. There are two parts of numerical simulations in this paper. Simulation 1 discusses various error distributions at different expectile levels under different conditions, the results show that the mean values of the biases of the estimation method in this paper, and other indicators, are very small, which indicates the robust property of the new method. Simulation 2 considers the symmetric and asymmetric bent lien expectile regression models, the results show that the estimated values of the estimation method in this paper are similar to the true values, which indicates the estimation effect and large sample performance of the proposed method are excellent. In the application research, the method in this paper is applied to the Arctic annual average temperature data and the Nile annual average flow data. The research shows that the standard errors of the estimation method in this paper are very similar to 0, indicating that the parameter estimation accuracy of the new method is very high, and the location of the change-point can be accurately estimated, which further confirms that the new method is effective and feasible. [ABSTRACT FROM AUTHOR]

Support Vector Regression (SVR) formulates is an optimization problem to learn a regression function that maps from input predictor variables to output observed response values. The SVR is useful because it balances model complexity and prediction error, and it has good performance for handling high-dimensional data. In this paper, we use the SVR model to improve the principal component analysis and the factor analysis methods. Simulation experiments are performed to assessment the new method. Some useful applications to real data sets are presented for comparing the competitive SVR models. It is noted that with increasing sample size, the 𝜀-SVR type under the principal component analysis is the best model. However, under the small sample sizes the 𝑣 -SVR type under the factor analysis provided adequate results. [ABSTRACT FROM AUTHOR]

In recent years, fused deposition molding (FDM) has attracted much attention as one of the most common and promising 3D printing technologies. Forming accuracy is one of the most concerned quality characteristics in the FDM process and is influenced by many factors. Based on the fact that the temperature gradient affects the molding accuracy, this paper presents a method for optimizing the accuracy of fused deposition molded parts based on least square support vector regression (LS-SVR), which considers a functional input: the printing speed varies continuously in the printing process, thus reducing the temperature gradients. Some parameters that can affect the temperature and cooling of the part such as nozzle temperature, hotbed temperature, and filling rate are also included in the study. Integrating the characteristics of a functional input and the principle of experimental design, we propose to model the printing speed curve using a Bézier curve and use the curve control points together with the scalar inputs as the variables to be optimized. Then, the sample set is obtained experimentally using stratified Latin hypercube sampling for experimental point selection. The regression modeling of the sample data is performed using LS-SVR with an improved kernel function, where the kernel function is improved by the Fréchet distance. Finally, the entire model is optimized by means of the genetic algorithm. The results show that the dimensional accuracy of the parts is significantly optimized by the proposed method. A comparison with existing methods demonstrates the efficiency and practicality of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we propose Bayesian skew‐normal regression models where the location, scale and shape parameters follow (linear or nonlinear) regression structures, and the variable of interest follows the Azzalini skew‐normal distribution. A Bayesian method is developed to fit the proposed models, using working variables to build the kernel transition functions. To illustrate the performance of the proposed Bayesian method and application of the model to analyze statistical data, we present results of simulated studies and of the application to studies of forced displacement in Colombia. [ABSTRACT FROM AUTHOR]

Estimation of error-invariable models is a specific problem in different fields such as medicine, economics, industry, and biostatistics. The main different between classical regression and error-in-variable models is that explanatory variables involve random error terms. Therefore, classical estimation methods that do not include the necessary adjustments for the contaminated explanatory variables give biased results. Regarding the error-in variables, there are important studied in the literature such as [1], [2], [3], [4], [5] and [6]. In this paper, nonparametric regression with measurement error is considered and estimated by kernel smoothing estimator which is studied detailed by [6]. This paper differs from their study with the idea of using two different kernel functions to compared them on quality of estimations. These functions are suitable for different error behaviors (see [7]). The goal of the paper is encouraged by a Monte Carlo simulation study and results are presented. [ABSTRACT FROM AUTHOR]

• The paper provides a robust regression method for interval-valued variables. • The objective function of the method considers the full interval information. • The computation of the sum of squares errors uses exponential-type kernel functions. • Outliers have a small weight for both center and radius parameter estimates. • Applications on synthetic and real data sets corroborate the proposed method. Interval-valued data are needed to manage either the uncertainty related to measurements, or the variability inherent to the description of complex objects representing group of individuals. A number of regression methods suitable to interval variables describing variability of complex objects are already available. However, less attention has been given to methods that, simultaneously, take into account the full interval information and are resistant to interval outlier observations, even with the frequent presence of atypical observations on interval-valued data sets. This paper proposes a new robust linear regression method for interval variables, where the presence of outliers either in the center or in the radius penalize both the center and the radius regression models. Moreover, the interval observations with outliers on both center and radius are more penalized than those observations with outliers only in the center (or in the radius). Besides, this paper provides a suitable iterative algorithm to estimate the parameters of the proposed method. The algorithm estimates the parameters of the center (or of the radius) model taking into account both information of the center and the radius. The convergence and time complexity of the iterative algorithm are also presented. Finally, the performance of the new method is compared with some previous robust regression approaches and evaluated on synthetic and real interval-valued data sets. [ABSTRACT FROM AUTHOR]

Genes do not function independently; rather, they interact with each other to fulfill their joint tasks. Identification of gene–gene interactions has been critically important in elucidating the molecular mechanisms responsible for the variation of a phenotype. Regression models are commonly used to model the interaction between two genes with a linear product term. The interaction effect of two genes can be linear or nonlinear, depending on the true nature of the data. When nonlinear interactions exist, the linear interaction model may not be able to detect such interactions; hence, it suffers from substantial power loss. While the true interaction mechanism (linear or nonlinear) is generally unknown in practice, it is critical to develop statistical methods that can be flexible to capture the underlying interaction mechanism without assuming a specific model assumption. In this study, we develop a mixed kernel function which combines both linear and Gaussian kernels with different weights to capture the linear or nonlinear interaction of two genes. Instead of optimizing the weight function, we propose a grid search strategy and use a Cauchy transformation of the P- values obtained under different weights to aggregate the P- values. We further extend the two-gene interaction model to a high-dimensional setup using a de-biased LASSO algorithm. Extensive simulation studies are conducted to verify the performance of the proposed method. Application to two case studies further demonstrates the utility of the model. Our method provides a flexible and computationally efficient tool for disentangling complex gene–gene interactions associated with complex traits. [ABSTRACT FROM AUTHOR]

Geographically weighted Poisson regression (GWPR) models are the class of spatial count regression models that capture the localization effect on various influencing factors on the dependent variable. The main challenge with the GWPR models is to set appropriate kernel function to give weights for each neighboring point during the model calibration. In this article, we consider GWPR models for many different kernel functions, including box-car, bi-square, tri-cube, exponential, and Gaussian function. Likelihood function, parameter estimation, and model selection criteria have been shown in details. We applied the model formulation to the road traffic accident (RTA) data in Oman as the country is one of the largest RTA-prone countries in the Gulf region. Akaike information criterion, corrected Akaike information criterion, and geographically weighted deviance have been used to assess the model fitting. The model with the exponential kernel weighted function provides the best fit for the data and captures the spatial heterogeneity and factors better with the exponential kernel weighting function. [ABSTRACT FROM AUTHOR]

Support vector regression (SVR) is able to consider the nonlinear relationship between explanatory variables X and a target variable y to build a regression model with high predictive accuracy. Additionally, y values predicted with SVR models for new samples can exceed the actual y values in training data. However, because the Gaussian kernel, which is a kernel function generally used in SVR, is based on the Euclidean distance between samples, it is unable to consider the importance of X when building the regression model. Therefore, in this study, the focus was on the importance of X that can be calculated by random forests (RF), and a novel SVR method, called variable importance‐considering support vector regression (VI‐SVR), was proposed based on this importance. Because X is weighted based on importance, the greater the importance of X, the greater its contribution to the predicted value. Analysis using the spectral, quantitative structure–property relationship (QSPR), and quantitative structure–activity relationship (QSAR) datasets confirmed that the predictive accuracy of VI‐SVR was better than that of SVR. VI‐SVR Python code is available at https://github.com/hkaneko1985/dcekit. We proposed a novel SVR method, called variable importance‐considering support vector regression (VI‐SVR), based on this importance calculated with random forests. Because X is weighted based on importance, the greater the importance of X, the greater its contribution to the predicted value. Analysis using the spectral, QSPR, and QSAR datasets confirmed that the predictive accuracy of VI‐SVR was better than that of SVR. [ABSTRACT FROM AUTHOR]

Background and Aim: Fetal biparietal diameter (BPD) is a feasible parameter to predict canine parturition date due to its inverted correlation with days before parturition (DBP). Although such a relationship is generally described using a simple linear regression (SLR) model, the imprecision of this model in predicting the parturition date in small- to medium-sized dogs is a common problem among veterinarian practitioners. Support vector regression (SVR) is a useful machine learning model for prediction. This study aimed to compare the accuracy of SVR with that of SLR in predicting DBP. Materials and Methods: After measuring 101 BPDs in 35 small- to medium-sized pregnant bitches, we fitted the data to the routine SLR model and the SVR model using three different kernel functions, radial basis function SVR, linear SVR, and polynomial SVR. The predicted DBP acquired from each model was further utilized for calculating the coefficient of determination (R2), mean absolute error, and mean squared error scores for determining the prediction accuracy. Results: All SVR models were more accurate than the SLR model at predicting DBP. The linear and polynomial SVRs were identified as the two most accurate models (p<0.01). Conclusion: With available machine learning software, linear and polynomial SVRs can be applied to predicting DBP in small- to medium-sized pregnant bitches. [ABSTRACT FROM AUTHOR]

Tweedie regression models (TRMs) are flexible tools to deal with non‐negative right‐skewed data and can handle semi‐continuous data, that is, continuous data with probability mass at zero. The geometric sums of Tweedie random variables lead to the geometric Tweedie distributions. Their corresponding regression models (GTRMs) provide not only additional flexibility to deal with continuous, semi‐continuous, heavily right‐skewed data but also a possibility of under‐variation than TRMs. Estimation and inference based on the likelihood approach for TRMs and GTRMs are challenging owing to the presence of an infinity sum and an intractable integral in the probability function along with non‐trivial restrictions on the Tweedie power parameter space. Thus, methods based on quasi‐likelihood have been proposed and successfully applied for estimation and inference in these classes of regression models. In this paper, our central focus is upon characterizing as well as comparing TRMs and GTRMs taking into consideration their variation and zero‐mass indices. Besides, we attempt to illustrate their application through some data analyses. Furthermore, we discuss the challenges for the computational implementation of such probability distributions and corresponding regression models referring to some available implementations in R. This article is categorized under:Statistical Models > Fitting ModelsStatistical Models > Generalized Linear ModelsStatistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms [ABSTRACT FROM AUTHOR]

Local spatiotemporal nonstationarity occurs in various natural and socioeconomic processes. Many studies have attempted to introduce time as a new dimension into a geographically weighted regression (GWR) model, but the actual results are sometimes not satisfying or even worse than the original GWR model. The core issue here is a mechanism for weighting the effects of both temporal variation and spatial variation. In many geographical and temporal weighted regression (GTWR) models, the concept of time distance has been inappropriately treated as a time interval. Consequently, the combined effect of temporal and spatial variation is often inaccurate in the resulting spatiotemporal kernel function. This limitation restricts the configuration and performance of spatiotemporal weights in many existing GTWR models. To address this issue, we propose a new spatiotemporal weighted regression (STWR) model and the calibration method for it. A highlight of STWR is a new temporal kernel function, wherein the method for temporal weighting is based on the degree of impact from each observed point to a regression point. The degree of impact, in turn, is based on the rate of value variation of the nearby observed point during the time interval. The updated spatiotemporal kernel function is based on a weighted combination of the temporal kernel with a commonly used spatial kernel (Gaussian or bi-square) by specifying a linear function of spatial bandwidth versus time. Three simulated datasets of spatiotemporal processes were used to test the performance of GWR, GTWR, and STWR. Results show that STWR significantly improves the quality of fit and accuracy. Similar results were obtained by using real-world data for precipitation hydrogen isotopes (δ2 H) in the northeastern United States. The leave-one-out cross-validation (LOOCV) test demonstrates that, compared with GWR, the total prediction error of STWR is reduced by using recent observed points. Prediction surfaces of models in this case study show that STWR is more localized than GWR. Our research validates the ability of STWR to take full advantage of all the value variation of past observed points. We hope STWR can bring fresh ideas and new capabilities for analyzing and interpreting local spatiotemporal nonstationarity in many disciplines. [ABSTRACT FROM AUTHOR]

The training effect is not only affected by many environmental disturbance factors, but also related to various factors such as the athlete itself. In this paper, the author analyze the regression prediction model of competitive sports based on SVM and artificial intelligence. Traditional statistical modeling simply compares existing data between players and compares them between data. Moreover, it is unable to formulate corresponding tactical strategies according to the situation of the opponent, and targeted training to strengthen the level of individual sports skills.By com-paring the effects of several kernel functions on the SVM modeling side, it is found that the RBF kernel function can make the SVM's prediction performance the best when dealing with the speed prediction problem. The experimental results show that this parameter optimization method can significantly improve the performance of the SVM regression machine. The prediction model based on support vector machine can effectively improve the prediction direction. Using artificial intelligence and image capture technology in sports can effectively improve the statistical efficiency and prediction effect of competition. [ABSTRACT FROM AUTHOR]

Semisupervised classification aims to construct a classifier by making use of both labeled data and unlabeled data. This paper proposes a semisupervised classifier based on a piecewise linear regression model implemented by using a gated linear network. The semisupervised classifier is constructed in two steps. In the first step, instead of estimating the break points of a piecewise linear model directly, a label‐guided autoencoder‐based semisupervised gating mechanism is designed to generate binary gate control signals to realize the partitioning. In the second step, the piecewise linear model is first transformed into linear regression form, and the linear parameters are then optimized globally by a Laplacian regularized least squares (LapRLS) algorithm using a kernel function comprising the gate control signals obtained in the first step. Moreover, the composed kernel function is used as a better similarity function for the graph construction. As a result, we capture data manifold from both labeled and unlabeled data, and the data manifold is ingeniously incorporated into both the kernel and the graph Laplacian in LapRLS. Numerical experiments on various real‐world datasets exhibit the effectiveness of the proposed method. © 2020 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc. [ABSTRACT FROM AUTHOR]

In this paper, a new approach is presented to fit a robust fuzzy regression model based on some fuzzy quantities. In this approach, we first introduce a new distance between two fuzzy numbers using the kernel function, and then, based on the least squares method, the parameters of fuzzy regression model is estimated. The proposed approach has a suitable performance to present the robust fuzzy model in the presence of different types of outliers. Using some simulated data sets and some real data sets, the application of the proposed approach in modeling some characteristics with outliers, is studied. [ABSTRACT FROM AUTHOR]

This study proposes a novel and efficient dual regression model for possibilistic regression analysis that incorporates the principles of support vector machine (SVM) theory. The dual regression model, which comprises an upper model and a lower model, approximates the observed fuzzy phenomena from the outside and inside directions, respectively, such that the inclusion relationship between those two models holds. The proposed dual regression model better explains the inherent vagueness that exists in a given dataset. It provides the outer and inner bounds for the estimated vagueness region, and allows an estimation of the degree of confidence in the predicted fuzzy output. Using the principles for a twin support vector machine (TSVM), the upper- and lower models are estimated by solving two smaller SVM-type quadratic programming problems (QPPs), instead of a single larger QPP. This strategy significantly increases the learning speed for the proposed algorithm. The structural risk minimization principle of SVM makes the proposed method to yield better generalization ability. The kernel function method offers a model-independent framework for the proposed dual regression model. This paper focuses on the class of radial kernels, which enables the proposed method to conquer the problem of increasing spreads. The radial kernel also gives the proposed method a unified framework that allows both crisp and fuzzy inputs. The experimental results verify the effectiveness and efficiency of the proposed method. In comparison with previous SVM-based dual regression model, the proposed approach significantly improves the sparsity, prediction speed, and training speed. [ABSTRACT FROM AUTHOR]

Local spatiotemporal non-stationarity occurs in various natural and socioeconomic processes. Many studies have attempted to introduce time as a new dimension into the geographically weighted regression model (GWR), but the actual results are sometimes not satisfied or even worse than the original GWR model. The core issue here is a mechanism for weighting effects of both temporal variation and spatial variation. In many geographical and temporal weighted regression models (GTWR), the concept of time distance has been inappropriately treated as time interval. Consequently, the combined effect of temporal and spatial variation is often inaccurate in the resulting spatiotemporal kernel function. This limitation restricts the configuration and performance of spatiotemporal weights in many existing GTWR models. To address this issue, we propose a new spatiotemporal weighted regression (STWR) model and the calibration method for it. A highlight of STWR is a new temporal kernel function, in which the method for temporal weighting is based on the degree of impact from each observed point to a regression point. The degree of impact, in turn, is based on the rate of value variation of the nearby observed point during the time interval. The updated spatiotemporal kernel function is based on a weighted combination of the temporal kernel with a commonly used spatial kernel (Gaussian or bi-square) by specifying a linear function of spatial bandwidth versus time. Three simulated datasets of spatiotemporal processes were used to test the performance of GWR, GTWR and STWR. Results show that STWR significantly improves the quality of fit and accuracy. Similar results were obtained by using real-world data for the precipitation hydrogen isotopes (δ2H) in Northeastern United States. The Leave-one-out cross-validation (LOOCV) test demonstrates that, comparing with GWR, the total prediction error of STWR is reduced by using recent observed points. Prediction surfaces of models in this case study show that STWR is more localized than GWR. Our research validates the ability of STWR to take full advantage of all the value variation of past observed points. We hope STWR can bring fresh ideas and new capabilities for analyzing and interpreting local spatiotemporal non-stationarity in many disciplines. [ABSTRACT FROM AUTHOR]

Regularized least-squares (kernel-ridge / Gaussian process) regression is a fundamental algorithm of statistics and machine learning. Because generic algorithms for the exact solution have cubic complexity in the number of datapoints, large datasets require to resort to approximations. In this work, the computation of the least-squares prediction is itself treated as a probabilistic inference problem. We propose a structured Gaussian regression model on the kernel function that uses projections of the kernel matrix to obtain a low-rank approximation of the kernel and the matrix. A central result is an enhanced way to use the method of conjugate gradients for the specific setting of least-squares regression as encountered in machine learning. Our method improves the approximation of the kernel ridge regressor / Gaussian process posterior mean over vanilla conjugate gradients and, allows computation of the posterior variance and the log marginal likelihood (evidence) without further overhead. [ABSTRACT FROM AUTHOR]

Datasets are often contaminated by various noises in many practical applications. The classical extreme learning machine (ELM) shows poor prediction accuracy and large fluctuation of prediction results in dealing with such datasets. To overcome this drawback, an exponential Laplace loss function was proposed, which can weaken the influences of noises. The proposed loss function is based on the Gauss kernel function, and is differentiable, non-convex, bounded and able to approach the Laplace function. Then the proposed loss function was introduced into the ELM to build a robust ELM model for regression estimation. The iterative re-weighted algorithm was employed to solve the resultant optimization problem. In each iteration, the training samples with noises were given smaller weights, which can effectively improve the prediction accuracy. Experiments on real-world datasets show that, the proposed model has better learning performance and robustness. [ABSTRACT FROM AUTHOR]

Hierarchical study design often occurs in many areas such as epidemiology, psychology, sociology, public health, engineering, and agriculture. This imposes correlation in data structure that needs to be account for in modelling process. In this study, a three-level mixed-effects least squares support vector regression (MLS-SVR) model is proposed to extend the standard least squares support vector regression (LS-SVR) model for handling cluster correlated data. The MLS-SVR model incorporates multiple random effects which allow handling unequal number of observations for each case at non-fixed time points (a very unbalanced situation) and correlation between subjects simultaneously. The methodology consists of a regression modelling step that is performed straightforwardly by solving a linear system. The proposed model is illustrated through numerical studies on simulated data sets and a real data example on human Brucellosis frequency. The generalization performance of the proposed MLS-SVR is evaluated by comparing to ordinary LS-SVR and some other parametric models. [ABSTRACT FROM AUTHOR]

In order to effectively improve the accuracy of runoff forecasting, this paper proposed an effective hybrid optimization strategy which was based on the combination of particle swarm optimization and simulated annealing algorithm, and it also optimized the type of kernel function and the kernel parameter setting of support vector regression to establish an effective hybrid optimization support vector regression runoff forecasting model. The proposed method provided an effective way for the choice of kernel functions and parameter optimization. By analyzing the examples of Guangxi Liujiang' s River runoff and with pure support vector regression model comparison, the results of the study show that the model is stable in prediction and it has high generalization performance and accuracy of prediction, and it can provide an effective prediction method for runoff forecast. [ABSTRACT FROM AUTHOR]

Abstract Torrefaction of biomass can be explained as a mild type of pyrolysis at temperatures usually ranging between 200 and 300 °C in lack of oxygen. Torrefaction of biomass enhances properties as the moisture content and calorific value. The objective of this study was to acquire a predictive model of the higher heating value (HHV) in a biomass torrefaction process. This study introduces a new hybrid algorithm, relied on support vector machines (SVMs) combined with the simulated annealing (SA) optimization technique, for predicting the calorific value (HHV) of biomass from operation input parameters determined experimentally during the torrefaction process. Additionally, a multivariate adaptive regression splines (MARS) approach and random forest (RF) technique were fitted to the experimental data for comparison purposes. The results of the present study are two-fold. In the first place, the significance of each physical–chemical variables on the higher heating value (HHV) is presented through the model. Secondly, several models for forecasting the calorific value of torrefied biomass are obtained. Indeed, when this hybrid SVM–SA-based model with RBF kernel function was applied to the experimental dataset with the optimal hyperparameters, a coefficient of determination equal to 0.90 was achieved for the higher heating value estimation of torrefied biomass. Moreover, the results accomplished with the MARS–SA-based approach and RF–SA-based technique are worse than those achieved with the RBF–SVM–SA-based model. The agreement between experimental data and the model demonstrated the good performance of the latter. Highlights • A RBF–SVM–SA-based predictive model is built for the torrefied biomass' HHV. • The characterization of the biomass' HHV is important in its quality assessment. • The relative importance of input variables in torrefaction process is determined. • The results of this new hybrid model are compared with MARS–SA and RF–SA models. [ABSTRACT FROM AUTHOR]

Poverty is a very serious problem and often faced by the countries in the world, especially developing countries. The percentage of poor people in Indonesia reached 11.47 percent in 2013. The seven provinces with the highest poverty in Indonesia are Papua, West Papua, East Nusa Tenggara, Maluku, Gorontalo, Bengkulu and Aceh. This problem is modeled using mixed nonparametric regression of Kernel and Fourier Series. The response variable of this model is percentage of poor people (y), the predictor variables that follow Kernel regression curve are Mean of Years Schooling or MYS (v1) and Literacy Rate or LR (v2), whereas the predictor variable that follow the Fourier Series regression curve are Unemployment Rate or UR (t1). This modeling produces R2 = 62.78%. [ABSTRACT FROM AUTHOR]

• A novel floating platform motion forecasting model is proposed. • A novel chaotic efficient bat algorithm is proposed to receive optimized parameter. • The ensemble empirical mode decomposition algorithm is proposed to decompose floating platform motion data. • The proposed hybrid kernel based support vector regression model receives higher forecasting accuracy. This paper presents a model for forecasting the motion of a floating platform with satisfactory forecasting accuracy. First, owing to the complex nonlinear characteristics of a time series of floating platform motion data, a support vector regression model with a hybrid kernel function is used to simulate the motion of a floating platform. Second, the proposed chaotic efficient bat algorithm, based on the chaotic, niche search, and evolution mechanisms, is used to optimize the parameters of the hybrid kernel-based support vector regression model. Third, the ensemble empirical mode decomposition algorithm is utilized to decompose the original floating platform motion time series into a series of intrinsic mode functions and residuals. The ultimate forecasting results are obtained by summing the outputs of these functions. Subsequently, motion data for a real floating platform are used to evaluate the reliability and effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

Consider the linear process , where is a sequence of identically distributed, negatively associated random variables with , and is a sequence of real numbers with. Under some mild conditions, we first establish a general result on complete convergence for weighted sums of such linear process, and then describe its statistical properties and interpretations in both semiparametric and nonparametric regression models. We also prove the complete consistency for the parameter estimators. In addition, we have conducted comprehensive simulation studies to demonstrate the validity of obtained theoretical results. [ABSTRACT FROM AUTHOR]

The medium-term power load forecasting is often disturbed by a variety of external factors (such as temperature, holidays and wind power) and uncertainties. Also, the factors affecting the medium-term power load forecasting are complex and changeable, and it is difficult to predict accurately. In the big data environment, how to obtain valuable information quickly in a variety of large number of influence factors has become the key to the power load forecasting problems. A method of density forecasting based on LASSO quantile regression was proposed in this paper. First, the important influence factors were selected from the various external factors affecting the power load forecasting, and the LASSO quantile regression model was established. Then, by using the triangular kernel function, LASSO quantile regression was combined with the method of kernel density estimation for the medium-term power load probability density forecasting. Taking the historical load and external influence factors (including temperature, holidays and wind power) of a sub-provincial city in eastern China as an example, the probability density prediction of medium-term power load was carried out. The average absolute error obtained was respectively 3.53% and 3.69% in the median and the mode, which was better than the results without considering the external factors and without variable selection. In order to further verify the superiority of the method, the method was compared with the nonlinear quantile regression (NLQR) and the quantile regression neural network based on triangle kernel (QRNNT) probability density forecasting methods. The results illustrate that this method can better solve the high-dimensional data problem in power load forecasting, and obtain more accurate results of power load forecasting. [ABSTRACT FROM AUTHOR]

The objective of the present study is to estimate total organic carbon (TOC) content over the entire thickness of Cambay Shale, in the boreholes of Jambusar–Broach block of Cambay Basin, India. To achieve this objective, support vector regression (SVR), a supervised data mining technique, has been utilized using five basic wireline logs as input variables. Suitable SVR model has been developed by selecting epsilon-SVR algorithm and varying three different kernel functions and parameters like gamma and cost on a sample dataset. The best result is obtained when the radial-basis kernel function with gamma = 1 and cost = 1, are used. Finally, the performance of developed SVR model is compared with the ΔlogR method. The TOC computed by SVR method is found to be more precise than the ΔlogR method, as it has better agreement with the core-TOC. Thus, in the present study area, the SVR method is found to be a powerful tool for estimating TOC of Cambay Shale in a continuous and rapid manner. [ABSTRACT FROM AUTHOR]

Under a nonparametric robust regression model, we consider the problem of estimating the score function ψ x for a fixed x in a functional space and with unknown scale parameter. The principal aim of this work is to establish the asymptotic normality of this estimator for a stationary ergodic process without any use of traditional mixing conditions. [ABSTRACT FROM AUTHOR]

Conventional traffic crash analyzing methods focus on identifying the relationship between traffic crash outcomes and impact risk factors and explaining the effects of risk factors, which ignore the changes of roadway systems and can lead to inaccurate results in traffic crash predictions. To address this issue, an innovative two-step method is proposed and a support vector regression (SVR) model is formulated into state-space model (SSM) framework for traffic crash prediction. The SSM was developed in the first step to identify the dynamic evolution process of the roadway systems that are caused by the changes of traffic flow and predict the changes of impact factors in roadway systems. Using the predicted impact factors, the SVR model was incorporated in the second step to perform the traffic crash prediction. A five-year dataset that obtained from 1152 roadway segments in Tennessee was employed to validate the model effectiveness. The proposed models result in an average prediction MAPE of 7.59%, a MAE of 0.11, and a RMSD of 0.32. For the performance comparison, a SVR model and a multivariate negative binomial (MVNB) model were developed to do the same task. The results show that the proposed model has superior performances in terms of prediction accuracy compared to the SVR and MVNB models. Compared to the SVR and MVNB models, the benefit of incorporating a state-space model to identify the changes of roadway systems is significant evident in the proposed models for all crash types, and the prediction accuracy that measured by MAPE can be improved by 4.360% and 6.445% on average, respectively. Apart from accuracy improvement, the proposed models are more robust and the predictions can retain a smoother pattern. Furthermore, the results show that the proposed model has a more precise and synchronized response behavior to the high variations of the observed data, especially for the phenomenon of extra zeros. [ABSTRACT FROM AUTHOR]

Highlights • A fuzzy semi-parametric quintile regression model is introduced. • A kernel-based signed-distance measure is proposed. • The method is illustrated via a simulation study and a real-world application. • The proposed method is compared with other existing ones. Abstract As an extension to statistical techniques, fuzzy regression analysis estimates the relationship between fuzzy independent variables and non-fuzzy and/or fuzzy dependent variables. However, facing skewed or non-normal data may lead to vague results if the conventional least square models are used. However, there are many situations in applications of intelligence systems where regression components are fuzzy quantities instead of exact values. The present work is aimed to propose a fuzzy semi-parametric quantile regression model with fuzzy predictors, a fuzzy smooth function, exact coefficients and fuzzy responses. The proposed method is based on a new class of novel kernel-based signed-distance measures in space of fuzzy numbers. Main features of the proposed signed-distance including its robustness were studied. It was shown that the proposed signed-distance measure is more reasonable and effective than a common signed-distance which extensively has been used in fuzzy set theory. This is the reason why we decided to apply the proposed signed-distance to extend the classical quantile regression model. Then, a hybrid algorithm was suggested to evaluate optimal bandwidth, non-fuzzy regression coefficients and fuzzy smooth function at a specific quantile level. The proposed method was then compared with some common fuzzy regression models based on two data sets including a simulation study and an applied example. The numerical results clearly indicated that the proposed fuzzy semi-parametric quantile linear regression model performs better than the other available methods in terms of some common goodness-of-fit criteria in cases where the data set includes skewness and/or some outlier data point. The main advantages of the proposed method are as follows: 1-it is not so sensitive to the outlier data points and/or skewed data, 2- it possesses better fitting effect compared to other existing fuzzy regression models. Thus, the proposed quantile regression model could be successfully applied in many practical studies of fuzzy regression model in real-world applications. [ABSTRACT FROM AUTHOR]

In this paper, we consider the estimation of both the parameters and the nonparametric link function in partially linear single‐index models for longitudinal data that may be unbalanced. In particular, a new three‐stage approach is proposed to estimate the nonparametric link function using marginal kernel regression and the parametric components with generalized estimating equations. The resulting estimators properly account for the within‐subject correlation. We show that the parameter estimators are asymptotically semiparametrically efficient. We also show that the asymptotic variance of the link function estimator is minimized when the working error covariance matrices are correctly specified. The new estimators are more efficient than estimators in the existing literature. These asymptotic results are obtained without assuming normality. The finite‐sample performance of the proposed method is demonstrated by simulation studies. In addition, two real‐data examples are analyzed to illustrate the methodology. [ABSTRACT FROM AUTHOR]

Abstract In this paper, a novel sparse least squares support vector regression algorithm, referred to as LSSVR-SBF, is introduced which uses a new low rank kernel based on simplex basis function, which has a set of nonlinear parameters. It is shown that the proposed model can be represented as a sparse linear regression model based on simplex basis functions. We propose a fast algorithm for least squares support vector regression solution at the cost of O (N) by avoiding direct kernel matrix inversion. An iterative estimation algorithm has been proposed to optimize the nonlinear parameters associated with the simplex basis functions with the aim of minimizing model mean square errors using the gradient descent algorithm. The proposed fast least square solution and the gradient descent algorithm are alternatively applied. Finally it is shown that the model has a dual representation as a piecewise linear model with respect to the system input. Numerical experiments are carried out to demonstrate the effectiveness of the proposed approaches. [ABSTRACT FROM AUTHOR]

Symmetric kernel smoothing is commonly used in estimating the nonparametric component in the partial linear regression models. In this article, we propose a new estimation method for the partial linear regression models using the inverse Gaussian kernel when the explanatory variable of the nonparametric component is non-negatively supported. As an asymmetric kernel function, the inverse Gaussian kernel is also supported on the non-negative half line. The asymptotic properties, including the asymptotic normality, uniform almost sure convergence, and the iterated logarithm laws, of the proposed estimators are thoroughly discussed for both homoscedastic and heteroscedastic cases. The simulation study is conducted to evaluate the finite sample performance of the proposed estimators. [ABSTRACT FROM AUTHOR]

The prediction accuracy of the traditional kernel with data-driven regression methods strongly depends on the appropriate selection of the kernel function and aims at solving the nonlinearity of the input space. In this paper, a self-learning kernel regression model is proposed. A special kernel space from the measured data is learned and designed, so that combined with dimension reductions on the input variables, the regression behavior between the projected input variables and the output variable is found. The model is posed as a semidefinite programming problem with the objective function to find the maximum variance between the learned manifolds. The kernel is data dependent and can be generated online whenever a new data point is available. The effectiveness of the proposed algorithm is demonstrated through the case studies on a simple nonlinear system and a real semiconductor process. Note to Practitioners—Virtual sensing of quality/key variables is critical for the control and optimization of industrial processes. Traditional data-driven kernel regression methods cannot achieve guaranteed high prediction accuracy. This paper aims to develop a high-accuracy virtual sensor based on the self-learning kernel regression (SLKR) model, which simultaneously constructs the relationship between the input and the output data as well as represents the data in a low-dimensional manifold. For practical implementation, the following are necessary to: 1) select a historical data set with abundant information of the process; 2) preprocess industrial data well and eliminate irregular entries; 3) establish a self-learning regression kernel based on local manifold structure and regression relationship; 4) approximate the explicit data-dependent kernel function and determine suitable parameter; and 5) extract regression coefficients from the learned kernel matrix. [ABSTRACT FROM AUTHOR]