Search

Bayesian shrinkage methods have been widely employed to perform variable selection with high-dimensional data. However, the presence of missing data hinders the implementation of these methods. Since complete case analyses can lead to biased estimates, applicable and efficient methods of variable selection with imputation are needed to obtain valid results. In order to address this issue, we propose an algorithm that employs the horseshoe shrinkage prior for shrinkage and multiple imputation for missing data in high-dimensional settings with a practical suggestion on model selection decision strategy. Simulation studies and real data analyses are presented and compared with those of other possible approaches. The simulation results suggest that the proposed algorithm can be considered as a general strategy for model selection of incomplete continuous data. [ABSTRACT FROM AUTHOR]

Clustering is an essential technique that groups similar data points to uncover the underlying structure and features of the data. Although traditional clustering methods such as k-means are widely utilized, they have limitations in identifying nonlinear clusters. Thus, alternative techniques, such as kernel k-means and spectral clustering, have been developed to address this issue. However, another challenge arises when irrelevant variables are present in the data; this can be mitigated by employing variable selection methods such as the filter, wrapper, and embedded approaches. In this study, with a particular focus on kernel k-means clustering, we propose an embedded variable selection method using a tensor product space along with a general analysis of variance kernel for nonlinear clustering. Comprehensive experiments involving simulations and real data analysis demonstrated that the proposed method achieves competitive performance compared to existing approaches. Thus, the proposed method may serve as a reliable tool for accurate cluster identification and variable selection to gain insights into complex datasets. [ABSTRACT FROM AUTHOR]

Functional magnetic resonance imaging (fMRI) plays a crucial role in neuroimaging, enabling the exploration of brain activity through complex-valued signals. These signals, composed of magnitude and phase, offer a rich source of information for understanding brain functions. Traditional fMRI analyses have largely focused on magnitude information, often overlooking the potential insights offered by phase data. In this paper, we propose a novel fully Bayesian model designed for analyzing single-subject complex-valued fMRI (cv-fMRI) data. Our model, which we refer to as the CV-M&P model, is distinctive in its comprehensive utilization of both magnitude and phase information in fMRI signals, allowing for independent prediction of different types of activation maps. We incorporate Gaussian Markov random fields (GMRFs) to capture spatial correlations within the data, and employ image partitioning and parallel computation to enhance computational efficiency. Our model is rigorously tested through simulation studies, and then applied to a real dataset from a unilateral finger-tapping experiment. The results demonstrate the model's effectiveness in accurately identifying brain regions activated in response to specific tasks, distinguishing between magnitude and phase activation. • Most fMRI studies use real-valued, magnitude only data, ignoring information in the complex part. • Identifying changes in magnitude and phase allows for detection of meaningful neuronal activity that would be missed by studying magnitude alone. • We propose a fully Bayesian model for identifying magnitude and phase changes in task-based BOLD fMRI. • The model accounts for spatial association via sparse Gaussian Markov random fields and is computationally scalable via parcellation. • We demonstrate that the model is able to not only identify task-related changes in BOLD signal, but the type of change (magnitude, phase, or both). [ABSTRACT FROM AUTHOR]

Variable selection involves selecting truly important predictors from p-dimensional multivariate functional predictors in functional predictive models. In this paper, a variable selection method is designed for scalar-on-function predictions entangled with nonlinear joint associations among scalar response and multiple functional predictors. First, a nonparametric functional nonlinear conditional correlation coefficient, namely, the FunNCC coefficient, is proposed to measure complex dependencies, including the nonmonotonic marginal dependence, along with the conditional associations of redundancy, complement, and interaction. Then, a model-free feature ordering and selection method is designed, where the FunNCC is utilized to rank relevance, enabling the selection of a subset of predictors with the strongest joint dependence. Since this method allows for quantitatively evaluating the contributions of predictors in explaining responses, it achieves moderate model interpretability. Finally, extensive simulation studies and two real-data cases involving air pollution regression and hand gesture recognition are conducted to evaluate the finite sample performance of the proposed method, and the results show that the proposed FunNCC and variable selection methods outperform state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

This paper considers the sparse estimation problem of regression coefficients in the linear model. Note that the global–local shrinkage priors do not allow the regression coefficients to be truly estimated as zero, we propose three threshold rules and compare their contraction properties, and also tandem those rules with the popular horseshoe prior and the horseshoe+ prior that are normally regarded as global–local shrinkage priors. The hierarchical prior expressions for the horseshoe prior and the horseshoe+ prior are obtained, and the full conditional posterior distributions for all parameters for algorithm implementation are also given. Simulation studies indicate that the horseshoe/horseshoe+ prior with the threshold rules are both superior to the spike-slab models. Finally, a real data analysis demonstrates the effectiveness of variable selection of the proposed method. [ABSTRACT FROM AUTHOR]

This paper presents a novel forecasting methodology based on neural networks and second-generation wavelets, and an example of application of such methodology for forecasting the oil temperature of an actual power transformer. The proposed method consists on selecting the variables with the highest correlation concerning the magnitude to be predicted (in situations featuring multiple variables), selecting the most appropriate second-generation wavelet from a set of candidate lifting wavelet transforms to smooth each variable's data and using the values smoothed by the selected second-generation wavelet to train the considered neural network. The authors applied the proposed methodology to predict the oil temperature of an existing transformer for four days and for one month using field data collected during four years of equipment operation. The obtained results indicate that selecting the most appropriate second-generation wavelet considering different selection techniques and using them to smooth the input variables significantly reduce the forecasting errors. The application of the proposed approach to forecast the oil temperature from the field-collected data using a long short-term memory neural network considering different batch sizes, neuron numbers and second-generation wavelet selection techniques decreased the average forecasting errors presented by using just the considered long short-term memory network by up to eleven times. The innovations of this proposal about literature are the use of correlation-based variable selection techniques given improving neural network forecasting accuracy, which is unprecedented in transformer insulating oil temperature forecasting applications, and the employment of SGWs to smooth the network's inputs to increase the accuracy of the model significantly. For the application case study presented in this research, the minor average forecasting errors in percentage to predict the four-day and one-month future temperature were, respectively, 0.012 and 0.326%. Also, for one month of forecasting, the average error in degrees Celsius was less than 0.2. [ABSTRACT FROM AUTHOR]

Although the fractional polynomials (FPs) can act as a concise and accurate formula for examining smooth relationships between response and predictors, modelling conditional mean functions observes the partial view of a distribution of response variable, as distributions of many response variables such as blood pressure (BP) measures are typically skew. Conditional quantile functions with FPs provide a comprehensive relationship between the response variable and its predictors, such as median and extremely high-BP measures that may be often required in practical data analysis generally. To the best of our knowledge, this is new in the literature. Therefore, in this article, we develop and employ Bayesian variable selection with quantile-dependent prior for the FP model to propose a Bayesian variable selection with parametric non-linear quantile regression model. The objective is to examine a non-linear relationship between BP measures and their risk factors across median and upper quantile levels using data extracted from the 2007 to 2008 National Health and Nutrition Examination Survey (NHANES). The variable selection in the model analysis identified that the non-linear terms of continuous variables (body mass index, age), and categorical variables (ethnicity, gender, and marital status) were selected as important predictors in the model across all quantile levels. [ABSTRACT FROM AUTHOR]

AbstractThis paper presents a Bayesian nonparametric model for clustering datasets with continuous, ordinal, and nominal variables. The ordinal and nominal variables are treated using the latent variables framework based on the multivariate probit and the multinomial probit models. Combining the continuous variables with the latent continuous variables allows us to jointly model a set of mixed-type variables via the Dirichlet process Gaussian mixture model. The use of hierarchical shrinkage prior on the component means leads to improved clustering performances and provides an intuitive way to identify relevant clustering variables. The numerical results on simulated and real data illustrate the applicability of the proposed model. [ABSTRACT FROM AUTHOR]

AbstractIn various data science applications, the relationship between predictor variables and the response is dynamic in the sense that the corresponding model parameters are varying coefficients. Estimation and variable selection for such dynamic models are challenging with a large number of parameters and complex optimization. In this work, we propose a regularized dynamic logistic regression for efficient variable selection and model estimation. The proposed method considers a combination of fused and group regularization to estimate varying effects of important predictors on responses in the presence of irrelevant predictors. Specifically, we select the important variable with dynamic impact on responses through the selection of the entire group of piece-wise constant functions for parameters, which can characterize dynamic impacts of predictor variables. Moreover, we develop an efficient algorithm based on the alternating direction method of multipliers for parameter estimation. The performance of the proposed method is evaluated by both simulations and real case studies. [ABSTRACT FROM AUTHOR]

Foldover designs often have attractive properties. Among these is that the effects can be divided into two orthogonal subspaces, one for odd effects and one for even effects. In this paper, we introduce a new method for analyzing foldover designs called the decoupling method that exploits this trait. Utilizing mirror image pair runs, two new responses are created, where each of them is only affected by effects in one of the orthogonal subspaces. Thereby the analysis of odd and even effects can be performed in two independent steps, enabling use of standard statistical procedures and formal testing of the presence of higher‐order interactions. The method is demonstrated on real data from a foldover of a 12‐run Plackett‐Burman (PB) design, and further evaluated through a simulation study, in which the decoupling method is compared to existing analysis methods. To get a thorough understanding of the properties, both a PB design and an OMARS design are used, and different design sizes and heredity scenarios considered. The method is especially suited for screening, as it yields high power for detecting the active effects. [ABSTRACT FROM AUTHOR]

In statistical process control of high-dimensional data streams, in addition to online monitoring of abnormal changes, fault diagnosis of responsible components has become increasingly important. Existing diagnostic procedures have been designed for some typical models with distribution assumptions. Moreover, there is a lack of systematic approaches to provide a theoretical guarantee of significance in estimating shifted components. In this article, we introduce a new procedure to control the False Discovery Rate (FDR) of fault diagnosis. The proposed method formulates the fault diagnosis as a variable selection problem and utilizes the symmetrized data aggregation technique via sample splitting, data screening, and information pooling to control the FDR. Under some mild conditions, we show that the proposed method can achieve FDR control asymptotically. Extensive numerical studies and two real-data examples demonstrate satisfactory FDR control and remarkable diagnostic power in comparison to existing methods. [ABSTRACT FROM AUTHOR]

Alzheimer's disease is a progressive neurodegenerative disorder, the early detection of which is crucial for timely intervention and enrollment in clinical trials. However, the preclinical diagnosis of Alzheimer's encounters difficulties with gold-standard methods. The current definitive diagnosis of Alzheimer's still relies on expensive instrumentation and post-mortem histological examinations. Here, we explore label-free Raman spectroscopy with machine learning as an alternative to preclinical Alzheimer's diagnosis. A special feature of this study is the inclusion of patient samples from different cohorts, sampled and measured in different years. To develop reliable classification models, partial least squares discriminant analysis in combination with variable selection methods identified discriminative molecules, including nucleic acids, amino acids, proteins, and carbohydrates such as taurine/hypotaurine and guanine, when applied to Raman spectra taken from dried samples of cerebrospinal fluid. The robustness of the model is remarkable, as the discriminative molecules could be identified in different cohorts and years. A unified model notably classifies preclinical Alzheimer's, which is particularly surprising because of Raman spectroscopy's high sensitivity regarding different measurement conditions. The presented results demonstrate the capability of Raman spectroscopy to detect preclinical Alzheimer's disease for the first time and offer invaluable opportunities for future clinical applications and diagnostic methods. [ABSTRACT FROM AUTHOR]

The penalized Lasso Cox proportional hazards model has been widely used to identify prognosis biomarkers in high-dimension settings. However, this method tends to select many false positives, affecting its interpretability. In order to improve the reproducibility, we develop a knockoff procedure that consists on wrapping the Lasso Cox model with the model-X knockoff, resulting in a powerful tool for variable selection that allows for the control of the false discovery rate in the presence of finite sample guarantees. In this paper, we propose a novel approach to sample valid knockoffs for ordinal and continuous variables whose distributions can be skewed or heavy-tailed, which employs a Latent Mixed Gaussian Copula model to account for the dependence structure between the variables, leading to what we call the Latent Gaussian Copula Knockoff (LGCK) procedure. We then combine the LGCK method with the Lasso coefficient difference (LCD) statistic as the importance metric. To our knowledge, our proposal is the first knockoff framework for jointly considering ordinal and continuous data in a non-Gaussian setting and a survival context. We illustrate the proposed methodology's effectiveness by applying it to a real lung cancer gene expression dataset. [ABSTRACT FROM AUTHOR]

Finite mixture of linear regression (FMLR) models are among the most exemplary statistical tools to deal with various heterogeneous data. In this paper, we introduce a new procedure to simultaneously determine the number of components and perform variable selection for the different regressions for FMLR models via an exponential power error distribution, which includes normal distributions and Laplace distributions as special cases. Under some regularity conditions, the consistency of order selection and the consistency of variable selection are established, and the asymptotic normality for the estimators of non-zero parameters is investigated. In addition, an efficient modified expectation-maximization (EM) algorithm and a majorization-maximization (MM) algorithm are proposed to implement the proposed optimization problem. Furthermore, we use the numerical simulations to demonstrate the finite sample performance of the proposed methodology. Finally, we apply the proposed approach to analyze a baseball salary data set. Results indicate that our proposed method obtains a smaller BIC value than the existing method. [ABSTRACT FROM AUTHOR]

This article presents a novel method for learning time‐varying dynamic Bayesian networks. The proposed method breaks down the dynamic Bayesian network learning problem into a sequence of regression inference problems and tackles each problem using the Markov neighborhood regression technique. Notably, the method demonstrates scalability concerning data dimensionality, accommodates time‐varying network structure, and naturally handles multi‐subject data. The proposed method exhibits consistency and offers superior performance compared to existing methods in terms of estimation accuracy and computational efficiency, as supported by extensive numerical experiments. To showcase its effectiveness, we apply the proposed method to an fMRI study investigating the effective connectivity among various regions of interest (ROIs) during an emotion‐processing task. Our findings reveal the pivotal role of the subcortical‐cerebellum in emotion processing. [ABSTRACT FROM AUTHOR]

This paper introduces a novel Principal-Weighted Penalized (PWP) regression model, designed for dimensionality reduction in large datasets without sacrificing essential information. This new model retains the favorable features of the principal component analysis (PCA) technique and penalized regression models. It weighs the variables in a large data set based on their contributions to principal components identified by PCA, enhancing its capacity to uncover crucial hidden variables. The PWP model also efficiently performs variable selection and estimates regression coefficients through regularization. An application of the proposed model on high-dimensional economic data is studied. The results of comparative studies in simulations and a real example in economic modeling demonstrate its superior fitting and predictive abilities. The resulting model excels in accuracy and interpretability, outperforming existing methods. [ABSTRACT FROM AUTHOR]

AbstractSparse regression and classification estimators that respect group structures have application to an assortment of statistical and machine learning problems, from multitask learning to sparse additive modeling to hierarchical selection. This work introduces structured sparse estimators that combine group subset selection with shrinkage. To accommodate sophisticated structures, our estimators allow for arbitrary overlap between groups. We develop an optimization framework for fitting the nonconvex regularization surface and present finite-sample error bounds for estimation of the regression function. As an application requiring structure, we study sparse semiparametric additive modeling, a procedure that allows the effect of each predictor to be zero, linear, or nonlinear. For this task, the new estimators improve across several metrics on synthetic data compared to alternatives. Finally, we demonstrate their efficacy in modeling supermarket foot traffic and economic recessions using many predictors. These demonstrations suggest sparse semiparametric additive models, fit using the new estimators, are an excellent compromise between fully linear and fully nonparametric alternatives. All of our algorithms are made available in the scalable implementation grpsel. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

We propose a new simultaneous variable selection and estimation procedure with the Gaussian seamless‐L0$$ {L}_0 $$ (GSELO) penalty for Cox proportional hazard model and additive hazards model. The GSELO procedure shows good potential to improve the existing variable selection methods by taking strength from both best subset selection (BSS) and regularization. In addition, we develop an iterative algorithm to implement the proposed procedure in a computationally efficient way. Theoretically, we establish the convergence properties of the algorithm and asymptotic theoretical properties of the proposed procedure. Since parameter tuning is crucial to the performance of the GSELO procedure, we also propose an extended Bayesian information criteria (EBIC) parameter selector for the GSELO procedure. Simulated and real data studies have demonstrated the prediction performance and effectiveness of the proposed method over several state‐of‐the‐art methods. [ABSTRACT FROM AUTHOR]

Variable selection approaches are often employed in high-dimensionality and multicollinearity problems. Since lasso selects variables by shrinking the coefficients, it has extensive use in many fields. On the other, we may sometime have extra information on the model. In this case, the extra information should be considered in the estimation procedure. In this paper, we propose a stochastic restricted lasso estimator in linear regression model which uses the extra information as stochastic linear restrictions. The estimator is a generalization of mixed estimator with L1 type penalization. We give the coordinate descent algorithm to estimate the coefficient vector of the proposed method and strong rules for the coordinate descent algorithm to discard variables from the model. Also, we propose a method to estimate the tuning parameter. We conduct two real data analyses and simulation studies to compare the new estimator with several estimators including the ridge, lasso and stochastic restricted ridge. The real data analyses and simulation studies show that the new estimator enjoys the automatic variable selection property of the lasso while outperforms standard methods, achieving lower test mean squared error. [ABSTRACT FROM AUTHOR]

In this paper, we propose a MIXED penalty for the LAD regression model, which can estimate parameters and select important variables efficiently and stably. The proposed method has a good performance in the case of dependent variable with heavy tail and outliers, so this estimator is robust and efficient for tackling the problem of heterogeniety. We show that the proposed estimator possesses the good properties by applying certain assumptions. In the part of numerical simulation, we give several simulation studies to examine the asymptotic results, which shows that the method we proposed behaves better. [ABSTRACT FROM AUTHOR]

AbstractThe Hawkes process models have been recently become a popular tool for modeling and analysis of neural spike train data. Despite this popularity, existing methodological and theoretical work has mainly focused on models without assumption of time-varying covariates, such as stimulus drive. In this article, motivated by neuronal spike trains study in the presence of stimuli, we propose a new Hawkes process model, where covariates are included in the intensity function. We consider the problem of simultaneous variable selection and estimation for the Hawkes process in the high-dimensional regime. Estimation of the intensity function of the high-dimensional point process is considered within a nonparametric framework, applying B-splines and the SCAD penalty for matters of sparsity. We apply the Doob-Kolmogorov inequality to establish the consistency of the resulting estimators. Finally, we illustrate the performance of our proposal through simulation and demonstrate its utility by applying it to the neuron spike train data set. [ABSTRACT FROM AUTHOR]

AbstractAn initial screening of which covariates are relevant is a common practice in high-dimensional regression models. The classic feature screening selects only a subset of covariates correlated with the response variable. However, many important features might have a relevant albeit highly nonlinear relation with the response. One screening approach that handles nonlinearity is to compute the correlation between the response and nonparametric functions of each covariate. Wavelets are powerful tools for nonparametric and functional data analysis but are still seldom used in the feature screening literature. We propose a wavelet feature screening method that can be easily implemented, and we prove that, under suitable conditions, it captures the true covariates with high probability. Simulation results also show that our approach outperforms other screening methods in highly nonlinear models. We apply feature screening to two datasets about ozone concentration and epilepsy. In both applications, the proposed method selects features that match findings in the literature of their respective research fields, illustrating the applicability of feature screening. Supplementary material for this article is available online. [ABSTRACT FROM AUTHOR]

The study aimed to measure the carotenoid (Car) and pH contents of carrots using hyperspectral imaging. A total of 300 images were collected using a hyperspectral imaging system, covering 472 wavebands from 400 to 1000 nm. Regions of interest (ROIs) were defined to extract average spectra from the hyperspectral images (HIS). We developed two models: least squares support vector machine (LS-SVM) and partial least squares regression (PLSR) to establish a quantitative analysis between the pigment amounts and spectra. The spectra and pigment contents were predicted and correlated using these models. The selection of EWs for modeling was done using the Successive Projections Algorithm (SPA), regression coefficients (RC) from PLSR models, and LS-SVM. The results demonstrated that hyperspectral imaging could effectively evaluate the internal attributes of carrot cortex and xylem. Moreover, these models accurately predicted the Car and pH contents of the carrot parts. This study provides a valuable approach for variable selection and modeling in hyperspectral imaging studies of carrots. [ABSTRACT FROM AUTHOR]

With the rapid development of modern technology, massive amounts of data with complex pattern are generated. Gaussian process models that can easily fit the nonlinearity in data become more and more popular nowadays. It is often the case that in some data only a few features are important or active. However, unlike classical linear models, it is challenging to identify active variables in Gaussian process models. One of the most commonly used methods for variable selection in Gaussian process models is automatic relevance determination, which is known to be open-ended. There is no rule of thumb to determine the threshold for dropping features, which makes the variable selection in Gaussian process models ambiguous. In this work, we propose two variable selection algorithms for Gaussian process models, which use the artificial nuisance columns as baseline for identifying the active features. Moreover, the proposed methods work for both regression and classification problems. The algorithms are demonstrated using comprehensive simulation experiments and an application to multi-subject electroencephalography data that studies alcoholic levels of experimental subjects. for this article are available online. [ABSTRACT FROM AUTHOR]

Advancement in high‐throughput sequencing technologies has stimulated intensive research interests to identify specific microbial taxa that are associated with disease conditions. Such knowledge is invaluable both from the perspective of understanding biology and from the biomedical perspective of therapeutic development, as the microbiome is inherently modifiable. Despite availability of massive data, analysis of microbiome compositional data remains difficult. The nature that relative abundances of all components of a microbial community sum to one poses challenges for statistical analysis, especially in high‐dimensional settings, where a common research theme is to select a small fraction of signals from amid many noisy features. Motivated by studies examining the role of microbiome in host transcriptomics, we propose a novel approach to identify microbial taxa that are associated with host gene expressions. Besides accommodating compositional nature of microbiome data, our method both achieves FDR‐controlled variable selection, and captures heterogeneity due to either heteroscedastic variance or non‐location‐scale covariate effects displayed in the motivating dataset. We demonstrate the superior performance of our method using extensive numerical simulation studies and then apply it to real‐world microbiome data analysis to gain novel biological insights that are missed by traditional mean‐based linear regression analysis. [ABSTRACT FROM AUTHOR]

The modeling of mineral deposits has been improved over the years with the incorporation of mineralogical and metallurgical information obtained from drilling samples that make up the pillars for the construction of resource models. However, sampling data is being made available in large quantities, causing current databases to grow exponentially. The use of machine learning (ML) algorithms has been applied to deal with multidimensional data problems. Principal component analysis (PCA) is a multivariate analysis (MA) technique whose aim is to reduce the dimension of multivariate data. Studies show that results obtained with the reduction of variables were satisfactory in different areas of activity. The purpose of this article is to test variable selection criteria using PCA for geometallurgical data and to check the feasibility of the technique for simplifying variable types and defining typological domains. [ABSTRACT FROM AUTHOR]

This article investigates the estimation and variable selection procedures of single-index models when the data have non ignorable response. For estimation of the index coefficients, a new procedure based on martingale difference divergence is developed. Consistency and asymptotic normality of the proposed estimators are established. The finite sample performance of the proposed procedure is assessed through extensive simulation studies. Finally, we illustrate the proposed method through a real body fat data set. [ABSTRACT FROM AUTHOR]

Zero-inflated explanatory variables, as opposed to outcome variables, are common, for example, in environmental sciences. In this article, we address the problem of having excess of zero values in some continuous explanatory variables, which are subject to multi-outcome lasso-regularized variable selection. In short, the problem results from the failure of the lasso-type of shrinkage methods to recognize any difference between zero value occurring either in the regression coefficient or in the corresponding value of the explanatory variable. This kind of confounding will obviously increase the number of false positives – all non-zero regression coefficients do not necessarily represent true outcome effects. We present here the adaptive LAD-lasso for multiple outcomes, which extends the earlier work of multi-outcome LAD-lasso with adaptive penalization. In addition to well-known property of having less biased regression coefficients, we show that the adaptivity also improves method's ability to recover from influences of excess of zero values measured in continuous covariates. [ABSTRACT FROM AUTHOR]

Two new non convex penalty functions – Laplace and arctan – were recently introduced in the literature to obtain sparse models for high-dimensional statistical problems. In this article, we study the theoretical properties of Laplace and arctan penalized ordinary least squares linear regression models. We first illustrate the near-unbiasedness of the non zero regression weights obtained by the new penalty functions, in the orthonormal design case. In the general design case, we present theoretical results in two asymptotic settings: (a) the number of features, p fixed, but the sample size, n → ∞ , and (b) both n and p tend to infinity. The theoretical results shed light onto the differences between the solutions based on the new penalty functions and those based on existing convex and non convex Bridge penalty functions. Our theory also shows that both Laplace and arctan penalties satisfy the oracle property. Finally, we also present results from a brief simulations study illustrating the performance of Laplace and arctan penalties based on the gradient descent optimization algorithm. [ABSTRACT FROM AUTHOR]

AbstractLongitudinal studies are indispensable to study the change over time in a response variable. The main challenge of such studies is the presence of missing values. Another challenge in these studies is that covariates may be subject to measurement error. In such studies, variable selection, especially if the data are subject to measurement error and missingness, is crucial. Variable selection may lead to biased results in case of ignoring the missing values. Also, measurement error in covariates can negatively affect the accuracy of the estimates if not treated properly. Variable selection for longitudinal data that suffers from missing values and measurement error in covariates is not well explored in literature. In this article, we propose and develop a simultaneous variable selection and parameter estimation method for longitudinal data that suffers from intermittent missing values and covariates measurement error. The penalized weighted generalized estimating equations is used to account for the missingness in the longitudinal response, and simulation selection extrapolation techniques is used to account for the covariate measurement error. A simulation study is conducted to assess the performance of the proposed method. Also, the applicability of the proposed method is demonstrated using the Longitudinal Internet Studies for Social sciences data. [ABSTRACT FROM AUTHOR]

This paper investigates variable selection in semiparametric spatial autoregressive panel data model with random effects. A penalized profile maximum-likelihood method is proposed with adaptive lasso penalty which achieves parameter estimation and variable selection at the same time. Under some regular conditions, we prove the theoretical properties of the estimators, including consistency and oracle property. In addition, we develop a feasible logarithm and carry out numerical simulations to examine the finite sample performance of this method. At last, a real data study about the investment influencing factors of the "Belt and Road" initiative is presented for illustration purpose. [ABSTRACT FROM AUTHOR]

The penalized regression is an important technique for high-dimensional data analysis, but penalized estimation method hinge on finding a suitable choice of tuning parameter. In this paper, a simple modified L curve method is proposed to select the tuning parameter for penalized estimation including Lasso, SCAD and MCP in linear regression models. Through data simulation and actual data analysis, we find that the modified L curve method can be simpler and more accurate than traditional tuning parameter selection schemes such as CV and BIC. Furthermore, the method is able to identify the true model consistently and has the less model error, especially for the cases where there is a high correlation between predictors. [ABSTRACT FROM AUTHOR]

For the functional linear models in which the dependent variable is functional and the predictors are scalar, robust regularization for simultaneous variable selection and regression parameter estimation is an important yet challenging issue. In this paper, we propose two types of regularized robust estimation methods. The first estimator adopts the ideas of reproducing kernel Hilbert space, least absolute deviation and group Lasso techniques. Based on the first method, the second estimator applies the pre-whitening technique and estimates the error covariance function by using functional principal component analysis. Simulation studies are conducted to examine the performance of the proposed methods in small sample sizes. The method is also applied to the Canadian weather data set, which consists of the daily average temperature and precipitation observed by 35 meteorological stations across Canada from 1960 to 1994. Numerical simulations and real data analysis show a good performance of the proposed robust methods for function-on-scalar models. [ABSTRACT FROM AUTHOR]

In this paper, we study the sparse estimation under the semiparametric linear transformation models for the current status data, also called type I interval-censored data. For the problem, the failure time of interest may be dependent on the censoring time and the association parameter between them is left unspecified. To address this, we employ the copula model to describe the dependence between them and a two-stage estimation procedure to estimate both the association parameter and the regression parameter. In addition, we propose a penalized maximum likelihood estimation procedure based on the broken adaptive ridge regression, and Bernstein polynomials are used to approximate the nonparametric functions involved. The oracle property of the proposed method is established and the numerical studies suggest that the method works well for practical situations. Finally, the method is applied to an Alzheimer's disease study that motivated this investigation. [ABSTRACT FROM AUTHOR]

This paper develops a Bayesian method to detect heterogeneity in the relationship between covariates and the outcome in models with ordered responses. To this end, we construct an efficient Markov chain Monte Carlo algorithm for a hierarchical Bayesian model that selects random coefficients in ordered models. This method extends an approach for selecting random coefficients in linear mixed models into the ordered setting by adding two enhancements that are relevant to the latter category of models. First, we construct steps to efficiently estimate cut-points by addressing identification and ordering constraints. Second, we develop a framework to evaluate marginal effects that combine the fixed and random effects of each covariate. The marginal effects additionally allow for model uncertainty by averaging across models visited by the selection algorithm. Simulation studies demonstrate that this method detects random effects when they are present, estimates parameters accurately and efficiently samples from the posterior with low autocorrelations across successive draws. On applying this method on data from the survey of consumer expectations, we find clear support for the presence of household-level heterogeneity in relationships between demographic variables, and current as well as expected financial conditions. [ABSTRACT FROM AUTHOR]

AbstractInterval-censored data frequently arise in various biomedical areas involving periodical follow-ups where the failure or event time of interest cannot be observed exactly but is only known to fall into a time interval. This article considers a semiparametric probit regression model, a valuable alternative to other commonly used semiparametric models in survival analysis, to investigate potential risk factors for the interval-censored failure time of interest. We develop an expectation-maximization (EM) algorithm to conduct the pseudo maximum likelihood estimation (MLE) using the working independence strategy for general or mixed-case interval-censored data. The resulting estimators of regression parameters are shown to be consistent and asymptotically normal with the empirical process techniques. In addition, we propose a novel penalized EM algorithm for simultaneously achieving variable selection and parameter estimation in the case of high-dimensional covariates. The proposed variable selection method can be readily implemented with some existing software and considerably reduces the estimation error of the proposed pseudo-MLE approach. Simulation studies demonstrate the satisfactory performance of the proposed methods. An application to a set of interval-censored data on prostate cancer further confirms the utility of the methodology. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

AbstractDue to the difficulties inherent in conducting controlled experiments, recent causal inference studies have focused on developing treatment effect estimation using observational data. One major difficulty in causal inference from observational data is that the underlying causal structure is unknown. This may result in the misidentification of potential sources of causal estimation bias, such as confounders, which must be controlled for accurate estimation. To consider all possible confounders and other relevant information, conventional methods predominantly use all observed covariates indiscriminately. However, previous studies have shown that including all covariates without considering their causal relationships may deteriorate the estimation performance. Although several data-driven variable selection methods have been proposed for treatment effect estimation, they cannot distinguish the confounders from other outcome-related covariates and are limited to simple regression forms. In this study, we propose a method called the Variable Selection Causal Estimation Network (VSCEN) that performs treatment effect estimation and causal variable selection simultaneously. Through end-to-end differentiable training, the VSCEN selects only the covariates beneficial for effect estimation and uses only those selected for effect estimation. Experimental evaluations on fully synthetic, semi-synthetic, and real datasets demonstrate the VSCEN’s superior performance in conditional average treatment effect estimation and competitive performance in average treatment effect estimation along with its accurate variable selection capabilities. [ABSTRACT FROM AUTHOR]

AbstractObtaining excellent prediction accuracy in the high-dimensional partially linear model is particularly important. However, it is difficult to achieve due to the complex relationship between nonparametric covariates and the response. Irrelevant covariates and unimportant data also commonly attenuate the prediction performance of the model. Further, high-dimensional data analysis is challenging for modern statistical studies. To overcome these difficulties, we propose the Double Sparsity Garrotized Kernel Machine (DSGKM) method with an efficient algorithm and its adjusted version for prediction. Specifically, we estimate the nonparametric components using the kernel machine technique, and impose L1-norm penalties simultaneously to select relevant covariates and retain the representative data in the final model. Besides, the convergence analysis of the adjusted algorithm is conducted. The advantages of our method are: (i) to sufficiently capture the complex relationship between nonparametric covariates and the response; (ii) to identify relevant covariates and select representative data; and (iii) to achieve higher computational efficiency, especially the situations when both parametric and nonparametric components are high-dimensional. Results on both simulated and real data show that the proposed method outperforms existing methods, even when outliers exist. [ABSTRACT FROM AUTHOR]

AbstractDue to rapid advancements in computer technology, high-dimensional, big, and complex data, such as functional data, where observations are considered as curves, have emerged from many applications in various disciplines of sciences, such as biomedicine, chemometrics, engineering, and social sciences. Variable selection has consequently become one of the most important problems in statistical research in recent years. As the functional data are inherently infinite dimensional, variable selection problem in multiple functional regression model is, therefore, challenging and difficult. In this study, we extend Sure Independence Screening (SIS) method to a multiple functional regression model with a scalar response and functional predictors. We show that the SIS-based procedure for multiple functional regression model retaining important functional predictors while screening out unimportant functional predictors, and reduce the dimension efficiently. Our methodologies are validated by simulation studies as well as some applications to data. [ABSTRACT FROM AUTHOR]

Misclassified current status data arise when the failure time of interest is observed or known only to be either smaller or larger than an observation time rather than observed exactly, and the failure status is examined by a diagnostic test with testing error. Such data commonly occur in various scientific fields, including clinical trials, demographic studies and epidemiological surveys. This paper discusses regression analysis of such data with the focus on variable selection or identifying predictable and important covariates associated with the failure time of interest. For the problem, a penalized maximum likelihood approach is proposed under the Cox proportional hazards model and the smoothly clipped absolute deviation penalty. More specifically, we develop a penalized EM algorithm to relieve the computational burden in maximizing the resulting, complex penalized likelihood function. A simulation study is conducted to examine the empirical performance of the proposed approach in finite samples, and an illustration to a set of real data on chlamydia is also provided. [ABSTRACT FROM AUTHOR]

Group testing has been verified as a cost-effective and time-efficient approach, where the individual samples are pooled with a predefined group size for subsequent testing. Recent research has explored the integration of covariate information to improve the modeling of the group testing data. While existing works for high-dimensional data primarily focus on parametric models, this study considers a more flexible generalized nonparametric additive model. Nonlinear components are approximated using B-splines and model estimation under the sparsity assumption is derived employing group lasso. Theoretical results demonstrate that our method selects the true model with a high probability and provides consistent estimates. Numerical studies are conducted to illustrate the good performance of our proposed method, using both simulated and real data. [ABSTRACT FROM AUTHOR]

Model selection uncertainty has drawn a lot of attention from academics recently because it significantly affects parameter estimation and prediction. Scholars are currently addressing and quantifying uncertainty in model selection by concentrating on model combining and model confidence sets. In this paper, we present a new approach for building model confidence sets, which we call AMac. We provide a theoretical lower bound on the degree of confidence in the model confidence sets that AMac has built. Furthermore, we discuss how the implementation of current model confidence set construction methods becomes difficult when dealing with high-dimensional variables. To address this problem, we suggest building model selection paths (MSP) as a solution. We develop an algorithm for building MSP and show its effectiveness by utilizing the theories of adaptive lasso and lars. We perform an extensive set of simulation experiments to compare the performances of Mac and AMac methods. According to the results, AMac is more stable when there are fluctuations in noise levels. The model confidence sets built by AMac, in particular, achieve coverage rates that are closer to the desired confidence level, especially in the presence of high noise levels. To further confirm that MSP can successfully generate model confidence sets that maintain the given confidence level as the sample size increases, we conduct extensive simulation tests with high-dimensional variables. Ultimately, we hope that the strategies and concepts discussed in this work will improve results in subsequent research on the uncertainty of model selection. [ABSTRACT FROM AUTHOR]

Recently, there has been a growing interest in variable selection for causal inference within the context of high-dimensional data. However, when the outcome exhibits a skewed distribution, ensuring the accuracy of variable selection and causal effect estimation might be challenging. Here, we introduce the generalized median adaptive lasso (GMAL) for covariate selection to achieve an accurate estimation of causal effect even when the outcome follows skewed distributions. A distinctive feature of our proposed method is that we utilize a linear median regression model for constructing penalty weights, thereby maintaining the accuracy of variable selection and causal effect estimation even when the outcome presents extremely skewed distributions. Simulation results showed that our proposed method performs comparably to existing methods in variable selection when the outcome follows a symmetric distribution. Besides, the proposed method exhibited obvious superiority over the existing methods when the outcome follows a skewed distribution. Meanwhile, our proposed method consistently outperformed the existing methods in causal estimation, as indicated by smaller root-mean-square error. We also utilized the GMAL method on a deoxyribonucleic acid methylation dataset from the Alzheimer's disease (AD) neuroimaging initiative database to investigate the association between cerebrospinal fluid tau protein levels and the severity of AD. [ABSTRACT FROM AUTHOR]

Drought is defined as a prolonged regional precipitation deficiency. Significant drought conditions often occur during spring in South Korea since abundant water resources are only available in summer, and failed water management is often reversed in the following spring. Since precipitation is the major driver of spring drought over South Korea, forecasting spring precipitation is essential for drought management. However, spring precipitation forecasting is difficult due to the difficulty of finding explanatory variables and the lack of long hydrological records for application to complex models, including nonlinear or deep learning models. When the teleconnection between the spring precipitation and globally gridded climate variables is used, there are many available candidates, and an appropriate procedure must be used to select reliable predictors. Therefore, this study aims to develop a simple forecasting model, finding reliable predictors for spring precipitation in South Korea, and using the teleconnection to the globally gridded climate variable of mean sea level pressure (MSLP). First, the accumulated spring precipitation (ASP), the median of 93 weather stations in South Korea, was identified, and two-pair combinations of teleconnected global winter MSLP, denoted Df4m, were considered candidate predictors. Then, exhaustive searching for explanatory variables was performed with correlation analysis between the ASP and Df4m to find reliable predictors. The 37 Df4m variables have high correlations of 0.55. The 37 Df4m variables were categorized into three regions: Arctic Ocean, South Pacific, and South Africa. Variables in the same region are similar, and the multicollinearity problem is unavoidable. Thus, the least absolute shrinkage and selection operator (LASSO) model was applied by choosing five Df4m variables, and the forecasting value agreed with the observed value (R2 = 0.72). Other models, including the multiple regression model and ElasticNet, did not perform significantly superior to the LASSO model. Therefore, by exhaustively searching many globally gridded candidates, the LASSO model is a good alternative to forecasting spring precipitation over South Korea with the ASP by finding reliable predictors and producing skillful results with limited records. The forecasting result using the LASSO model and Df4m variables can be widely used for drought management. [ABSTRACT FROM AUTHOR]

Neural demyelination and brain damage accumulated in white matter appear as hyperintense areas on T2-weighted MRI scans in the form of lesions. Modeling binary images at the population level, where each voxel represents the existence of a lesion, plays an important role in understanding aging and inflammatory diseases. We propose a scalable hierarchical Bayesian spatial model, called BLESS, capable of handling binary responses by placing continuous spike-and-slab mixture priors on spatially varying parameters and enforcing spatial dependency on the parameter dictating the amount of sparsity within the probability of inclusion. The use of mean-field variational inference with dynamic posterior exploration, which is an annealing-like strategy that improves optimization, allows our method to scale to large sample sizes. Our method also accounts for underestimation of posterior variance due to variational inference by providing an approximate posterior sampling approach based on Bayesian bootstrap ideas and spike-and-slab priors with random shrinkage targets. Besides accurate uncertainty quantification, this approach is capable of producing novel cluster size based imaging statistics, such as credible intervals of cluster size, and measures of reliability of cluster occurrence. Lastly, we validate our results via simulation studies and an application to the UK Biobank, a large-scale lesion mapping study with a sample size of 40,000 subjects. for this article are available online. [ABSTRACT FROM AUTHOR]

Sufficient dimension reduction (SDR) methods target finding lower-dimensional representations of a multivariate predictor to preserve all the information about the conditional distribution of the response given the predictor. The reduction is commonly achieved by projecting the predictor onto a low-dimensional subspace. The smallest such subspace is known as the Central Subspace (CS) and is the key parameter of interest for most SDR methods. In this article, we propose a unified and flexible framework for estimating the CS in high dimensions. Our approach generalizes a wide range of model-based and model-free SDR methods to high-dimensional settings, where the CS is assumed to involve only a subset of the predictors. We formulate the problem as a quadratic convex optimization so that the global solution is feasible. The proposed estimation procedure simultaneously achieves the structural dimension selection and coordinate-independent variable selection of the CS. Theoretically, our method achieves dimension selection, variable selection, and subspace estimation consistency at a high convergence rate under mild conditions. We demonstrate the effectiveness and efficiency of our method with extensive simulation studies and real data examples. for this article are available online. [ABSTRACT FROM AUTHOR]

Although log‐likelihood is widely used in model selection, the log‐likelihood ratio has had few applications in this area. We develop a log‐likelihood ratio based method for selecting regression models by focusing on the set of models deemed plausible by the likelihood ratio test. We show that when the sample size is large and the significance level of the test is small, there is a high probability that the smallest model in this set is the true model; thus, we select this smallest model. The significance level of the test serves as a tuning parameter of this method. We consider three levels of this parameter in a simulation study and compare this method with the Akaike information criterion (AIC) and Bayesian information criterion (BIC) to demonstrate its excellent accuracy and adaptability to different sample sizes. This method is a frequentist alternative and a strong competitor to AIC and BIC for selecting regression models. [ABSTRACT FROM AUTHOR]

As a popular tool for nonlinear models, additive models work efficiently with nonparametric estimation. However, naively applying the existing regularization method can result in misleading outcomes because of the basis sparsity in each variable. In this article, we consider variable selection in additive models via a combination of variable selection and basis selection, yielding a joint selection of variables and basis functions. A novel penalty function is proposed for basis selection to address the hierarchical structure as well as the sparsity assumption. Under some mild conditions, we establish theoretical properties including the support recovery consistency. We also derive the necessary and sufficient conditions for the estimator and develop an efficient algorithm based on it. Our new methodology and results are supported by simulation and real data examples. [ABSTRACT FROM AUTHOR]

Supersaturated designs (SSDs) refer to those designs in which the run size is much smaller than the main effects to be estimated. They are commonly used to identify a few, but critical, active factors from a large set of potentially active ones, keeping the cost as low as possible. In this regard, the development of new construction and analysis methods has recently seen a rapid increase. In this paper, we provide some methods to construct equi- and mixed-level E (f   N O D ) optimal SSDs with a large number of inert factors using the substitution method. The proposed methods are easy to implement, and many new SSDs can then be constructed from them. We also study a variable selection method based on the screening-selection network (SSnet) method for regression problems. A real example is analyzed to illustrate that it is able to effectively identify active factors. Eight different analysis methods are used to analyze the data generated from the proposed designs. Three scenarios with different setups of parameters are designed, and the performances of each method are illustrated by extensive simulation studies. Among all these methods, SSnet produced the most satisfactory results according to power. [ABSTRACT FROM AUTHOR]

Understanding how populations respond to climate is fundamentally important to many questions in ecology, evolution, and conservation biology. Climate is complex and multifaceted, with aspects affecting populations in different and sometimes unexpected ways. Thus, when measuring the changing climate it is important to consider the complexity of the phenomenon and the number of ways it can be characterized through different metrics. We used a Bayesian sparse modeling approach to select among 80 metrics of climate and applied the approach to 19 datasets of bird, insect, and plant population responses to abiotic conditions as case studies of how the method can be applied for climate variable selection in a time series context. For phenological datasets, mean spring temperature was frequently selected as an important climate driver, while selected predictors were more diverse for population metrics such as abundance or reproductive success. The climate variable selection approach presented here can help to identify potential climate metrics when there is limited physiological or mechanistic information to make an a priori variable selection, and is broadly applicable across studies on population responses to climate. [ABSTRACT FROM AUTHOR]