Search

To avoid the expensive quadratic programming in the L2-loss support vector regression (SVR) model, smooth approximation and iteratively reweighted least square (IRLS) techniques were introduced in literature, resulting in smoothed SVR (SSVR) and IRLS-SVR. However, for nonlinear models, SSVR and IRLS-SVR both involve operations with matrices and vectors of the same size as the training set. Thus, as the training set becomes large, nonlinear SSVR and IRLS-SVR both need long training time and large memory. To further alleviate the training cost, this paper projects the original data into a low dimensional space via random Fourier feature. The inner product of the random Fourier features of two data points is approximately the same as the kernel function evaluated at these two data points. Hence, it is possible to use a linear model in the new low dimensional space to approximate the original nonlinear model, and consequently the time/memory efficient linear training algorithms could be applied. This paper applies the idea of random Fourier features to nonlinear SSVR and IRLS-SVR, and our testing results on real-world datasets show that, the introduction of random Fourier features makes SSVR and IRLS-SVR achieve similar prediction accuracy as the original nonlinear version with substantially higher time efficiency. [ABSTRACT FROM AUTHOR]

To overcome the computational burden of quadratic programming in kernel expectile regression (KER), iteratively reweighted least square (IRLS) technique was introduced in literature, resulting in IRLS-KER. However, for nonlinear models, IRLS-KER involves operations with matrices and vectors of the same size as the training set. Thus, as the training set becomes large, nonlinear IRLS-KER needs a long training time and large memory. To further alleviate the training cost, this paper projects the original data into a low-dimensional space via random Fourier feature. The inner product of the random Fourier features of two data points is approximately the same as the kernel function evaluated at these two data points. Hence, it is possible to use a linear model in the new low-dimensional space to approximate the original nonlinear model, and consequently, the time/memory efficient linear training algorithms could be applied. This paper applies the idea of random Fourier features to IRLS-KER, and our testing results on simulated and real-world datasets show that, the introduction of random Fourier features makes IRLS-KER achieve similar prediction accuracy as the original nonlinear version with substantially higher time efficiency. [ABSTRACT FROM AUTHOR]

It is important to detect and extract the major cortical sulci from brain images, but manually annotating these sulci is a time-consuming task and requires the labeler to follow complex protocols. This paper proposes a learning-based algorithm for automated extraction of the major cortical sulci from magnetic resonance imaging (MRI) volumes and cortical surfaces. Unlike alternative methods for detecting the major cortical sulci, which use a small number of predefined rules based on properties of the cortical surface such as the mean curvature, our approach learns a discriminative model using the probabilistic boosting tree algorithm (PBT). PBT is a supervised learning approach which selects and combines hundreds of features at different scales, such as curvatures, gradients and shape index. Our method can be applied to either MRI volumes or cortical surfaces. It first outputs a probability map which indicates how likely each voxel lies on a major sulcal curve. Next, it applies dynamic programming to extract the best curve based on the probability map and a shape prior. The algorithm has almost no parameters to tune for extracting different major sulci. It is very fast (it runs in under 1 min per sulcus including the time to compute the discriminative models) due to efficient implementation of the features (e.g., using the integral volume to rapidly compute the responses of 3-D Haar filters). Because the algorithm can be applied to MRI volumes directly, there is no need to perform preprocessing such as tissue segmentation or mapping to a canonical space. The learning aspect of our approach makes the system very flexible and general. For illustration, we use volumes of the right hemisphere with several major cortical sulci manually labeled. The algorithm is tested on two groups of data, including some brains from patients with Williams Syndrome, and the results are very encouraging. Index Terms--Cortical sulci, discriminative models, dynamic programming, learning, magnetic resonance (MR) images, probability boosting tree.

In a support vector regression (SVR) model, using the squared ϵ-insensitive loss function makes the optimization problem strictly convex and yields a more concise solution. However, the formulation of L 2 -SVR leads to a quadratic programming which is expensive to solve. This paper reformulates the optimization problem of L 2 -SVR by absorbing the constraints in the objective function, which can be solved efficiently by a majorization-minimization approach, in which an upper bound for the objective function is derived in each iteration which is easier to be minimized. The proposed approach is easy to implement, without requiring any additional computing package other than basic linear algebra operations. Numerical studies on real-world datasets show that, compared to the alternatives, the proposed approach can achieve similar prediction accuracy with substantially higher time efficiency in training. [ABSTRACT FROM AUTHOR]

In support vector regression (SVR) model, using the squared ϵ -insensitive loss function makes the objective function of the optimization problem strictly convex and yields a more concise solution. However, the formulation leads to a quadratic programing which is expensive to solve. This paper reformulates the optimization problem by absorbing the constraints in the objective function, and the new formulation shares similarity with weighted least square regression problem. Based on this formulation, we propose an iteratively reweighted least square approach to train the L2-loss SVR, for both linear and nonlinear models. The proposed approach is easy to implement, without requiring any additional computing package other than basic linear algebra operations. Numerical studies on real-world datasets show that, compared to the alternatives, the proposed approach can achieve similar prediction accuracy with substantially higher time efficiency. [ABSTRACT FROM AUTHOR]

This paper gives an improvement to Bennett's inequality for tail probability of sum of independent random variables, without imposing any additional condition. The improved version has a closed form expression. Using a refined arithmetic-geometric mean inequality, we further improve the obtained inequality. Numerical comparisons show that the proposed inequalities often improve the upper bound significantly in the far tail area, and these improvements get more prominent for larger sample size. [ABSTRACT FROM AUTHOR]

High-spatial-resolution aerial photographs can provide detailed distribution of sea ice features. However, very few studies have ever considered shadows on the photographs for sea ice detection. In this letter, sea ice shadows, retrieved from 163 selected aerial photographs acquired on July 26, 2010, in a marginal-ice-zone area near Barrow, Alaska, utilizing an object-based classification scheme, are used to estimate the sea ice ridge attributes through local solar illumination geometry. The photograph-averaged ridge frequency (354.6–8908.7 km−2), length (1.22–10.33 m), and height (0.15–1.29 m) are obtained from the 163 photographs using batch processing. This letter provides an important batch processing method for ridge detection and ridge attribute retrieval from the high-resolution imagery of sea ice. [ABSTRACT FROM PUBLISHER]

Abstract: This paper proposes two gradient based methods to fit a Probit regression model by maximizing the sample log-likelihood function. Using the property of the Hessian of the objective function, the first method performs weighted least square regression in each iteration of the Newton–Raphson framework, resulting in ProbitBoost, a boosting-like algorithm. Motivated by the gradient boosting algorithm , the second proposed approach maximizes the sample log-likelihood function by updating the fitted function a small step in the gradient direction, performing gradient ascent in functional space, resulting in Gradient ProbitBoost. We also generalize the algorithms to multi-class problems by two strategies, one of which is to use the gradient ascent to maximize the multi-class sample log-likelihood function for fitting all the classifiers simultaneously, and the second approach uses the one-versus-all scheme to reduce the multi-class problem to a series of binary classification problems. The proposed algorithms are tested on typical classification problems including face detection, cancer classification, and handwritten digit recognition. The results show that compared to the alternative methods, the proposed algorithms perform similar or better in terms of testing error rates. [Copyright &y& Elsevier]

The decision tree method has grown fast in the past two decades and its performance in classification is promising. The tree-based ensemble algorithms have been used to improve the performance of an individual tree. In this study, we compared four basic ensemble methods, that is, bagging tree, random forest, AdaBoost tree and AdaBoost random tree in terms of the tree size, ensemble size, band selection (BS), random feature selection, classification accuracy and efficiency in ecological zone classification in Clark County, Nevada, through multi-temporal multi-source remote-sensing data. Furthermore, two BS schemes based on feature importance of the bagging tree and AdaBoost tree were also considered and compared. We conclude that random forest or AdaBoost random tree can achieve accuracies at least as high as bagging tree or AdaBoost tree with higher efficiency; and although bagging tree and random forest can be more efficient, AdaBoost tree and AdaBoost random tree can provide a significantly higher accuracy. All ensemble methods provided significantly higher accuracies than the single decision tree. Finally, our results showed that the classification accuracy could increase dramatically by combining multi-temporal and multi-source data set. [ABSTRACT FROM AUTHOR]

Efficient and reliable spectrum sensing plays a critical role in cognitive radio networks. This paper presents a cooperative sequential detection scheme to reduce the average sensing time that is required to reach a detection decision. In the scheme, each cognitive radio computes the log-likelihood ratio for its every measurement, and the base station sequentially accumulates these log-likelihood statistics and determines whether to stop making measurement. The paper studies how to implement the scheme in a robust manner when the assumed signal models have unknown parameters, such as signal strength and noise variance. These ideas are illustrated through two examples in spectrum sensing. One assumes both the signal and noise are Gaussian distributed, while the other assumes the target signal is deterministic. [ABSTRACT FROM AUTHOR]

Recently Saul and Lee proposed a mixture model for discriminative classification of non-negative data via non-negative matrix factorization for feature extraction. In order to improve the generalization, this paper considers a sparse version of the model. The basic idea is to minimize the sum of the weights of un-normalized mixture models for posterior distributions according to regularization method. Experiments on CBCL face database and USPS digit data set assess the validity of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper presents a learning based method for automatic extraction of the major cortical sulci from MRI volumes or extracted surfaces. Instead of using a few pre-defined rules such as the mean curvature properties, to detect the major sulci, the algorithm learns a discriminative model by selecting and combining features from a large pool of candidates. We used the Probabilistic Boosting Tree algorithm to learn the model, which implicitly discovers and combines rules based on manually annotated sulci traced by neuroanatomists. The algorithm almost has no parameters to tune and is fast because of the adoption of integral volume and 3D Haar filters. For a given approximately registered MRI volume, the algorithm computes the probability of how likely it is that each voxel lies on a major sulcus curve. Dynamic programming is then applied to extract the curve based on the probability map and a shape prior. Because the algorithm can be applied to MRI volumes directly, there is no need to perform preprocessing such as tissue segmentation or mapping to a canonical space. The learning aspect makes the approach flexible and it also works on extracted cortical surfaces.