Search

Machine Learning Requires Probability and Statistics The contemporary practice of machine learning often involves the application of deterministic, computationally intensive algorithms to iteratively minimize a criterion of fit between a discriminant and sample data. There is often little interest in using probability to model the uncertainty in the problem and statistics to characterize the behavior of predictors derived from data, with the emphasis being on computation and coding. It follows that little can be stated about performance on future data, beyond perhaps a simple error count on a given test set. In this article, we argue that the knowledge imparted by deterministic computational methods is not rigorously related to the real world and, in particular, future events. This connection requires rigorous probabilistic modeling and statistical inference as well as an understanding of the proper role of computation and an appreciation of epistemological issues.

Uncovering links between processing conditions, microstructure, and properties is a central tenet of materials analysis. It is well known that microstructure determines properties, but expressing these structural features in a universal quantitative fashion has proved to be extremely difficult. Recent efforts have focused on training supervised learning algorithms to place microstructure images into predefined classes, but this approach assumes a level of a priori knowledge that may not always be available. In this paper, we expand this idea to the semi-supervised context in which class labels are known with confidence for only a fraction of the microstructures that represent the material system. It is shown that classifiers which perform well on both the high-confidence labeled data and the unlabeled, ambiguous data can be constructed by relying on the labeling consensus of a collection of semi-supervised learning methods. We also demonstrate the use of novel error estimation approaches for unlabeled data to establish robust confidence bounds on the classification performance over the entire microstructure space.

For most diseases, building large databases of labeled genetic data is an expensive and time-demanding task. To address this, we introduce genetic Generative Adversarial Networks (gGAN), a semi-supervised approach based on an innovative GAN architecture to create large synthetic genetic data sets starting with a small amount of labeled data and a large amount of unlabeled data. Our goal is to create a mechanism able to increase the sample size of the labeled data and generalize learning over different populations while keeping awareness of the quality of its own predictions. The proposed model achieved satisfactory results using real genetic data from different datasets and populations, in which the test populations may not have the same genetic profiles. The proposed model is self-aware and capable of determining whether a new genetic profile has enough compatibility with the data on which the network was trained and is thus suitable for prediction. The code and datasets used can be found at https://github.com/caio-davi/gGAN

Dengue has become one of the most important worldwide arthropod-borne diseases. Dengue phenotypes are based on laboratorial and clinical exams, which are known to be inaccurate. Objective : We present a machine learning approach for the prediction of dengue fever severity based solely on human genome data. Methods : One hundred and two Brazilian dengue patients and controls were genotyped for 322 innate immunity single nucleotide polymorphisms (SNPs). Our model uses a support vector machine algorithm to find the optimal loci classification subset and then an artificial neural network (ANN) is used to classify patients into dengue fever or severe dengue. Results : The ANN trained on 13 key immune SNPs selected under dominant or recessive models produced median values of accuracy greater than 86%, and sensitivity and specificity over 98% and 51%, respectively. Conclusion : The proposed classification method, using only genome markers, can be used to identify individuals at high risk for developing the severe dengue phenotype even in uninfected conditions. Significance : Our results suggest that the genetic context is a key element in phenotype definition in dengue. The methodology proposed here is extendable to other Mendelian based and genetically influenced diseases.

Control of gene regulatory networks (GRNs) to shift gene expression from undesirable states to desirable ones has received much attention in recent years. Most of the existing methods assume that the cost of intervention at each state and time point, referred to as the immediate cost function, is fully known. In this paper, we employ the Partially-Observed Boolean Dynamical System (POBDS) signal model for a time sequence of noisy expression measurement from a Boolean GRN and develop a Bayesian Inverse Reinforcement Learning (BIRL) approach to address the realistic case in which the only available knowledge regarding the immediate cost function is provided by the sequence of measurements and interventions recorded in an experimental setting by an expert. The Boolean Kalman Smoother (BKS) algorithm is used for optimally mapping the available gene-expression data into a sequence of Boolean states, and then the BIRL method is efficiently combined with the Q-learning algorithm for quantification of the immediate cost function. The performance of the proposed methodology is investigated by applying a state-feedback controller to two GRN models: a melanoma WNT5A Boolean network and a p53-MDM2 negative feedback loop Boolean network, when the cost of the undesirable states, and thus the identity of the undesirable genes, is learned using the proposed methodology.

Motor imagery brain computer interface designs are considered difficult due to limitations in subject-specific data collection and calibration, as well as demanding system adaptation requirements. Recently, subject-independent (SI) designs received attention because of their possible applicability to multiple users without prior calibration and rigorous system adaptation. SI designs are challenging and have shown low accuracy in the literature. Two major factors in system performance are the classification algorithm and the quality of available data. This paper presents a comparative study of classification performance for both SS and SI paradigms. Our results show that classification algorithms for SS models display large variance in performance. Therefore, distinct classification algorithms per subject may be required. SI models display lower variance in performance but should only be used if a relatively large sample size is available. For SI models, LDA and CART had the highest accuracy for small and moderate sample size, respectively, whereas we hypothesize that SVM would be superior to the other classifiers if large training sample-size was available. Additionally, one should choose the design approach considering the users. While the SS design sound more promising for a specific subject, an SI approach can be more convenient for mentally or physically challenged users.

Optimal classification requires full knowledge of the feature-label distribution. In practice, that is a relatively rare scenario, and a combination of distributional knowledge and sample data must be employed to obtain a classifier. In this chapter, we introduce the basic concepts related to samplebased classification, including designed classifiers and error rates, and consistency. The chapter includes a section showing that distribution-free classification rules have important limitations. The material in this chapter provides the foundation for the next several chapters on sample-based classification.

All the classification rules seen so far were plug-in rules, that is, they could be viewed as distribution estimation using training data. We consider now a different idea: iteratively adjusting a discriminant (decision boundary) to the training data by optimizing an error criterion. This is reminiscent in some ways to the process of human learning. Not surprisingly, this is the basic idea behind many popular classification rules: support vector machines, neural networks, decision trees, and rankbased classifiers. We examine each of these classification rules in this chapter.

In this chapter, we address an obvious question about classification: which classification rule should one choose for a given problem? A related question is how to pick the free parameters of a classification rule.

If one knew the feature-label distribution of a problem, then one could in principle compute the exact error of a classifier. In many practical cases, such knowledge is not available, and it is necessary to estimate the error of a classifier using sample data. If the sample size is large enough, it is possible to divide the available data into training and testing samples, design a classifier on the training set, and evaluate it on the test set. In this chapter, it is shown that this produces a very accurate error estimator, provided that there is an abundance of test points. However, large test samples are not possible if the overall sample size is small, in which case training and testing has to be performed on the same data. This chapter provides a comprehensive survey of different classification error estimation procedures and how to assess their performance.

We present a framework for the simultaneous estimation of state and parameters of partially observed Boolean dynamical systems (POBDS). Simultaneous state and parameter estimation is achieved through the combined use of the Boolean Kalman filter and Boolean Kalman smoother, which provide the minimum mean-square error state estimators for the POBDS model, and maximum-likelihood (ML) parameter estimation; in the presence of continuous parameters, ML estimation is performed using the expectation–maximization algorithm. The performance of the proposed ML adaptive filter is demonstrated by numerical experiments with a POBDS model of gene regulatory networks observed through noisy next-generation sequencing (RNA-seq) time series data using the well-known p53-MDM2 negative-feedback loop gene regulatory model.

Background Expression-based phenotype classification using either microarray or RNA-Seq measurements suffers from a lack of specificity because pathway timing is not revealed and expressions are averaged across groups of cells. This paper studies expression-based classification under the assumption that single-cell measurements are sampled at a sufficient rate to detect regulatory timing. Thus, observations are expression trajectories. In effect, classification is performed on data generated by an underlying gene regulatory network. Results Network regulation is modeled via a Boolean network with perturbation, regulation not fully determined owing to inherent biological randomness. The binary assumption is not critical because the resulting Markov chain characterizes expression trajectories. We assume a partially known Gaussian observation model belonging to an uncertainty class of models. We derive the intrinsically Bayesian robust classifier to discriminate between wild-type and mutated networks based on expression trajectories. The classifier minimizes the expected error across the uncertainty class relative to the prior distribution. We test it using a mammalian cell-cycle model, discriminating between the normal network and one in which gene p27 is mutated, thereby producing a cancerous phenotype. Tests examine all model aspects, including trajectory length, perturbation probability, and the hyperparameters governing the prior distribution over the uncertainty class. Conclusions Simulations show the rates at which the expected error is diminished by smaller perturbation probability, longer trajectories, and hyperparameters that tighten the prior distribution relative to the unknown true network. For average-expression measurement, methods have been proposed to obtain prior distributions. These should be extended to the more mathematically difficult, but more informative, expression trajectories. Electronic supplementary material The online version of this article (10.1186/s12918-018-0549-y) contains supplementary material, which is available to authorized users.

This paper studies classification of gene-expression trajectories coming from two classes, healthy and mutated (cancerous) using Boolean networks with perturbation (BNps) to model the dynamics of each class at the state level. Each class has its own BNp, which is partially known based on gene pathways. We employ a Gaussian model at the observation level to show the expression values of the genes given the hidden binary states at each time point. We use expectation maximization (EM) to learn the BNps and the unknown model parameters, derive closed-form updates for the parameters, and propose a learning algorithm. After learning, a plug-in Bayes classifier is used to classify unlabeled trajectories, which can have missing data. Measuring gene expressions at different times yields trajectories only when measurements come from a single cell. In multiple-cell scenarios, the expression values are averages over many cells with possibly different states. Via the central-limit theorem, we propose another model for expression data in multiple-cell scenarios. Simulations demonstrate that single-cell trajectory data can outperform multiple-cell average expression data relative to classification error, especially in high-noise situations. We also consider data generated via a mammalian cell-cycle network, both the wild-type and with a common mutation affecting p27.

We propose a novel feature selection approach based on the Bayesian Top Scoring Pairs (BTSP) method. We compare its performance against well-known feature selection methods, under SVM, k-NN and NB classification rules, by means of an extensive numerical experiment using real gene-expression data sets. Results demonstrate the promise of the BTSP feature selection approach in the analysis of high-dimensional biological data.

This paper studies expression-based classification under the assumption that single-cell measurements are sampled at a sufficient rate to detect regulatory timing. Observations are expression trajectories. In effect, classification is performed on data generated by an underlying gene regulatory network. Network regulation is modeled via a Boolean network with perturbation, regulation not being fully determined owing to inherent biological randomness, and we assume a partially known Gaussian observation model belonging to an uncertainty class of models. We derive the intrinsically Bayesian robust classifier to discriminate between wild-type and mutated networks based on expression trajectories. We test it using the mammalian cell-cycle model, discriminating between wild-type and mutated networks. Tests examine all model aspects, including trajectory length, perturbation probability, and the hyperparameters governing the prior distribution over the uncertainty class.

This paper studies the classification of gene regulatory networks (GRNs) modeled by probabilistic Boolean networks (PBNs). After observing Gaussian expression values of n genes at m consecutive time points, with consideration of missing data, an algorithm based on expectation maximization (EM) is proposed to estimate the parameters and infer the unknown parts of the networks in the maximum likelihood (ML) sense. Then the estimated values are plugged in to the Bayes classifier, which is optimal, and the performance of the classifier is investigated through various simulations.

We extend the popular Top Scoring Pair (TSP) classification rule to a Bayesian setting, with the purpose of obtaining robust and effective classifiers for small-sample, high-dimensional data. We employ the Bradley-Terry model for rank data, and infer its parameters using a previously proposed Gibbs sampling algorithm. The parameters are then used to define a Bayesian TSP score, which is used to select the gene pairs to define the proposed Bayesian TSP classifiers. Accuracy of the proposed Bayesian classification rules is evaluated against those of the conventional TSP classifiers as well as other well-known machine learning methods, using a total of 12 gene-expression data sets. The results indicate that the Bayesian k-TSP classifier obtained the best overall average accuracy rate and the best accuracy rate over the majority of the individual data sets.

Error estimation is a crucial part of classification methodology and it becomes problematic with small samples. We demonstrate here that the complexity of the decision boundary plays a key role on the performance of error estimation methods. First, a model is developed which quantifies the complexity of a classification problem purely in terms of the geometry of the decision boundary, without relying on the Bayes error. Then, this model is used in a simulation study to analyze the bias and root-mean-square (RMS) error of a few widely used error estimation methods relative to the complexity of the decision boundary: resubstitution, leave-one-out, 10-fold cross-validation with repetition, 0.632 bootstrap, and bolstered resubstitution, in two- and three-dimensional spaces. Each estimator is implemented with three classification rules: quadratic discriminant analysis (QDA), 3-nearest-neighbor (3NN) and two-layer neural network (NNet). The results show that all the estimation methods lose accuracy as complexity increases.

Gene-expression-based phenotype classification is used for disease diagnosis and prognosis relating to treatment strategies. The present paper considers classification based on sequential measurements of multiple genes using gene regulatory network (GRN) modeling. There are two networks, original and mutated, and observations consist of trajectories of network states. The problem is to classify an observation trajectory as coming from either the original or mutated network. GRNs are modeled via probabilistic Boolean networks, which incorporate stochasticity at both the gene and network levels. Mutation affects the regulatory logic. Classification is based upon observing a trajectory of states of some given length. We characterize the Bayes classifier and find the Bayes error for a general PBN and the special case of a single Boolean network affected by random perturbations (BNp). The Bayes error is related to network sensitivity, meaning the extent of alteration in the steady-state distribution of the original network owing to mutation. Using standard methods to calculate steady-state distributions is cumbersome and sometimes impossible, so we provide an efficient algorithm and approximations. Extensive simulations are performed to study the effects of various factors, including approximation accuracy. We apply the classification procedure to a p53 BNp and a mammalian cell cycle PBN.

Background DENV infection can induce different clinical manifestations varying from mild forms to dengue fever (DF) or the severe hemorrhagic fever (DHF). Several factors are involved in the progression from DF to DHF. No marker is available to predict this progression. Such biomarker could allow a suitable medical care at the beginning of the infection, improving patient prognosis. Objectives The aim of this study was to compare the serum expression levels of acute phase proteins in a well-established cohort of dengue fever (DF) and dengue hemorrhagic fever (DHF) patients, in order to individuate a prognostic marker of diseases severity. Study design The serum levels of 36 cytokines, chemokines and acute phase proteins were determined in DF and DHF patients and compared to healthy volunteers using a multiplex protein array and near-infrared (NIR) fluorescence detection. Serum levels of IL-1ra, IL-23, MIF, sCD40 ligand, IP-10 and GRO-α were also determined by ELISA. Results At the early stages of infection, GRO-α and IP-10 expression levels were different in DF compared to DHF patients. Besides, GRO-α was positively correlated with platelet counts and IP-10 was negatively correlated with total protein levels. Conclusions These findings suggest that high levels of GRO-α during acute DENV infection may be associated with a good prognosis, while high levels of IP-10 may be a warning sign of infection severity.

From September 2005 to March 2007, 238 individuals being vaccinated for the first time with the yellow fever (YF) -17DD vaccine were enrolled in a cohort established in Recife, Brazil. A prospective study indicated that, after immunization, anti-YF immunoglobulin M (IgM) and anti-YF IgG were present in 70.6% (IgM) and 98.3% (IgG) of the vaccinated subjects. All vaccinees developed protective immunity, which was detected by the plaque reduction neutralization test (PRNT) with a geometric mean titer of 892. Of the 238 individuals, 86.6% had IgG antibodies to dengue virus; however, the presence of anti-dengue IgG did not interfere significantly with the development of anti-YF neutralizing antibodies. In a separate retrospective study of individuals immunized with the 17DD vaccine, the PRNT values at 5 and 10 years post-vaccination remained positive but showed a significant decrease in neutralization titer (25% with PRNT titers < 100 after 5 years and 35% after 10 years).

Error estimation must be used to find the accuracy of a designed classifier, an issue that is critical in biomarker discovery for disease diagnosis and prognosis in genomics and proteomics. This paper presents, for what is believed to be the first time, the analytical formulation for the joint sampling distribution of the actual and estimated errors of a classification rule. The analysis presented here concerns the linear discriminant analysis (LDA) classification rule and the resubstitution and leave-one-out error estimators, under a general parametric Gaussian assumption. Exact results are provided in the univariate case, and a simple method is suggested to obtain an accurate approximation in the multivariate case. It is also shown how these results can be applied in the computation of condition bounds and the regression of the actual error, given the observed error estimate. In contrast to asymptotic results, the analysis presented here is applicable to finite training data. In particular, it applies in the small-sample settings commonly found in genomics and proteomics applications. Numerical examples, which include parameters estimated from actual microarray data, illustrate the analysis throughout.

Canalizing genes possess broad regulatory power over a wide swath of regulatory processes. On the other hand, it has been hypothesized that the phenomenon of intrinsically multivariate prediction (IMP) is associated with canalization. However, applications have relied on user-selectable thresholds on the IMP score to decide on the presence of IMP. A methodology is developed here that avoids arbitrary thresholds, by providing a statistical test for the IMP score. In addition, the proposed procedure allows the incorporation of prior knowledge if available, which can alleviate the problem of loss of power due to small sample sizes. The issue of multiplicity of tests is addressed by family-wise error rate (FWER) and false discovery rate (FDR) controlling approaches. The proposed methodology is demonstrated by experiments using synthetic and real gene-expression data from studies on melanoma and ionizing radiation (IR) responsive genes. The results with the real data identified DUSP1 and p53, two well-known canalizing genes associated with melanoma and IR response, respectively, as the genes with a clear majority of IMP predictor pairs. This validates the potential of the proposed methodology as a tool for discovery of canalizing genes from binary gene-expression data. The procedure is made available through an R package.

Bolstered error estimation has been shown to perform better than cross-validation and competitively with bootstrap in small-sample settings. However, its performance can deteriorate in the high-dimensional settings prevalent in Genomic Signal Processing. We propose here a modification of Bolstered error estimation that is based on the principle of Naive Bayes. Rather than attempting to estimate a single variance parameter for a spherical bolstering kernel in high-dimensional spaces from a small sample, we assume an ellipsoidal kernel and estimate each univariate variance separately along each variable. In simulation results based on a model for gene-expression data and a linear SVM classification rule, the new bolstered estimator clearly outperformed the old one, as well as cross-validation and resubstitution, and was also superior to the 0.632 bootstrap except in the case where a large feature set is selected.

Motivation: Ranking gene feature sets is a key issue for both phenotype classification, for instance, tumor classification in a DNA microarray experiment, and prediction in the context of genetic regulatory networks. Two broad methods are available to estimate the error (misclassification rate) of a classifier. Resubstitution fits a single classifier to the data, and applies this classifier in turn to each data observation. Cross-validation (in leave-one-out form) removes each observation in turn, constructs the classifier, and then computes whether this leave-one-out classifier correctly classifies the deleted observation. Resubstitution typically underestimates classifier error, severely so in many cases. Cross-validation has the advantage of producing an effectively unbiased error estimate, but the estimate is highly variable. In many applications it is not the misclassification rate per se that is of interest, but rather the construction of gene sets that have the potential to classify or predict. Hence, one needs to rank feature sets based on their performance. Results: A model-based approach is used to compare the ranking performances of resubstitution and cross-validation for classification based on real-valued feature sets and for prediction in the context of probabilistic Boolean networks (PBNs). For classification, a Gaussian model is considered, along with classification via linear discriminant analysis and the 3-nearest-neighbor classification rule. Prediction is examined in the steady-distribution of a PBN. Three metrics are proposed to compare feature-set ranking based on error estimation with ranking based on the true error, which is known owing to the model-based approach. In all cases, resubstitution is competitive with cross-validation relative to ranking accuracy. This is in addition to the enormous savings in computation time afforded by resubstitution.

Convex bootstrap error estimation is a popular tool for classifier error estimation in gene expression studies. A basic question is how to determine the weight for the convex combination between the basic bootstrap estimator and the resubstitution estimator such that the resulting estimator is unbiased at finite sample sizes. The well-known 0.632 bootstrap error estimator uses asymptotic arguments to propose a fixed 0.632 weight, whereas the more recent 0.632+ bootstrap error estimator attempts to set the weight adaptively. In this paper, we study the finite sample problem in the case of linear discriminant analysis under Gaussian populations. We derive exact expressions for the weight that guarantee unbiasedness of the convex bootstrap error estimator in the univariate and multivariate cases, without making asymptotic simplifications. Using exact computation in the univariate case and an accurate approximation in the multivariate case, we obtain the required weight and show that it can deviate significantly from the constant 0.632 weight, depending on the sample size and Bayes error for the problem. The methodology is illustrated by application on data from a well-known cancer classification study. Electronic supplementary material The online version of this article (doi:10.1186/s13637-014-0015-0) contains supplementary material, which is available to authorized users.

Human leukocyte antigen (HLA) alleles have been correlated with susceptibility or resistance to severe dengue; however, few immunogenetic studies have been performed in Latin American (LA) populations. We have conducted immunogenetic studies of HLA class I and II alleles in a cohort of 187 patients with DENV-3 infection and confirmed clinical diagnosis of either severe dengue, known as dengue hemorrhagic fever (DHF), or the less severe form, dengue fever (DF), in Recife, Pernambuco, Brazil. An association analysis was performed using Fisher’s association test, with odds ratios (ORs) calculated using conditional maximum likelihood estimates. HLA-B*44 (P=0.047, OR = 2.025, 95% CI = 0.97–4.24) was found to be associated with increased susceptibility to DHF in response to DENV-3 infection. In addition, HLA-B*07 (P=0.048, OR = 0.501, one-sided 95% CI = 0–0.99) and HLA-DR*13 (P=0.028, OR = 0.511, one-sided 95% CI = 0–0.91) were found to be associated with resistance to secondary dengue infection by DENV-3. These results suggest that HLA-B*44 supertype alleles and their respective T-cell responses might be involved in susceptibility to severe dengue infections, whereas the HLA-B*07 supertype alleles and DR*13 might be involved in cross-dengue serotype immunity.

Canalizing genes possess broad regulatory power over biological processes. In a previous publication, it was hypothesized that canalizing genes may be identified as those that possess a large number of intrinsically multivariate predictive (IMP) gene sets. An IMP set is a predictor set that predicts well the target, which cannot however be predicted by any of the proper subsets in the IMP set. The IMP property is defined mathematically in terms of a score based on the binary Coefficient of Determination (CoD), but there was until now no rigorous statistical procedure to obtain a test for a nonzero IMP score. In this paper, we address this by assuming a stochastic regulation model and providing an Intersection-Union Test (IUT) based on likelihood-ratio tests for the individual model parameters. We derive exact analytical formulas for the rejection region and p-value for this test. We address the issue of multiplicity of tests due to a large number of candidate predictor sets and predictive logics by means of FWER- and FDR-controlling approaches. The methodology is demonstrated by application to a real melanoma gene-expression data set, which shows that DUSP1, a canalizing gene over important biological processes in melanoma, displayed a large number of significant IMP sets, after correction by either of the multiple testing procedures.

Typically, it is common to observe peaking phenomenon in the classification error when the feature size increases. In this paper, we study linear support vector machine classifiers where the data is non-separable. A simulation based on synthetic data is implemented to study the possibility of observing peaking phenomenon. However, no peaking in the expected true error is observed. We also present the performance of three different error estimators as a function of feature and sample size. Based on our study, one might conclude that when using linear support vector machines, the size of feature set can increase safely.

Error estimation is a crucial part of any classification problem and it becomes problematic with small samples. In this paper, we analyze the performance of some widely used error estimation methods relative to the complexity of the feature-label distribution: resubstitution, 10-fold cross validation with repetition (CV10r), leave-one-out (LOO), bootstrap .632, and bolstered resubstitution. Our definition of complexity takes into account both the complexity of the Bayes decision surface and the Bayes error. We define the complexity of distribution for a class of Gaussian mixture models. In this class, the Bayes classifier is a piecewise linear classifier and its complexity is included in our definition. Based on the defined measure of complexity, we perform experiments for 2-dimensional and 3-dimensional problems and apply different error estimation methods for distributions of different complexities. The Bias and root-mean-squared (RMS) error of the error estimators are used to analyze their performances. The simulation results show that all the estimation methods lose accuracy as the complexity increases and this performance loss is quantified as a function of distribution complexity.

Application of ensemble classification rules in genomics and proteomics has become increasingly common. However, the problem of error estimation for these classification rules, particularly for bagging under the small-sample settings prevalent in genomics and proteomics, is not well understood. Breiman proposed the "out-of-bag" method for estimating statistics of bagged classifiers, which was subsequently applied by other authors to estimate the classification error. In this paper, we give an explicit definition of the out-of-bag estimator that is intended to remove estimator bias, by formulating carefully how the error count is normalized. We also report the results of an extensive simulation study of bagging of common classification rules, including LDA, 3NN, and CART, applied on both synthetic and real patient data, corresponding to the use of common error estimators such as resubstitution, leave-one-out, cross-validation, basic bootstrap, bootstrap 632, bootstrap 632 plus, bolstering, semi-bolstering, in addition to the out-of-bag estimator. The results from the numerical experiments indicated that the performance of the out-of-bag estimator is very similar to that of leave-one-out; in particular, the out-of-bag estimator is slightly pessimistically biased. The performance of the other estimators is consistent with their performance with the corresponding single classifiers, as reported in other studies.

It has been argued on an empirical basis that ensemble classification by bagging cannot improve the performance of stable classification rules, such as linear discriminant analysis. We have proved that this this is indeed the case: the expected classification error of the bagged linear discriminant is always larger or equal than that of the original linear discriminant, for all sample sizes. This result was proved for the univariate case, under a general Gaussian assumption. In the multivariate case, we provide an exact expression for the expected error of the bagged classifier, which is compared to the exact expected error of the original classifier for several different model parameters. For all models and sample sizes considered, bagging produced a larger expected classification error than the original classifier. We believe that this is the first time that such results are established for bagging in continuous feature spaces.

Error estimation is fundamental in GSP applications, such as the discovery of biomarkers to classify disease, or the construction of genetic regulatory networks, especially in small sample settings. Braga-Neto and Dougherty proposed a kernel-based technique of error estimation, called bolstered error estimation, which was shown empirically to work well in low-dimensional spaces (Braga-Neto and Dougherty, 2004). We present in this paper preliminary results of a simulation study on how bolstering performs in high-dimensional spaces.

Error estimation is a key aspect of statistical pattern recognition. The true classification error rate is usually unavailable since it depends on the unknown feature-label distribution. Hence, one needs to estimate the error rate from the available sample data. This paper presents a concise, mathematically rigorous review of the subject of error estimation in statistical pattern recognition, pointing to the pitfalls that arise in small-sample settings due to the use of "rules of thumb" and a neglect for proper mathematical understanding of the problem.

This paper introduces a novel approach for image analysis based on the notion of multiscale connectivity. We use the proposed approach to design several novel tools for object-based image representation and analysis, which exploit the connectivity structure of images in a multiscale fashion. More specifically, we propose a nonlinear pyramidal image representation scheme, which decomposes an image at different scales by means of multiscale grain filters. These filters gradually remove connected components from an image that fail to satisfy a given criterion. We also use the concept of multiscale connectivity to design a hierarchical data partitioning tool. We employ this tool to construct another image representation scheme, based on the concept of component trees, which organizes partitions of an image in a hierarchical multiscale fashion. In addition, we propose a geometrically-oriented hierarchical clustering algorithm which generalizes the classical single-linkage algorithm. Finally, we propose two object-based multiscale image summaries, reminiscent of the well-known (morphological) pattern spectrum, which can be useful in image analysis and image understanding applications.

In [5], a novel concept of connectivity for grayscale images was introduced, which is called grayscale level connectivity. In that framework, a grayscale image is connected if all its threshold sets below a given level are connected. It was shown that grayscale level connectivity defines a connection, in the sense introduced by Jean Serra in [10]. In the present paper, we extend grayscale level connectivity to the case where different connectivities are used for different threshold sets, a concept we call grayscale level multiconnectivity. In particular, this leads to the definition of a new operator, called the multiconnected grayscale reconstruction operator. We show that grayscale level multiconnectivity defines a connection, provided that the connectivities used for the threshold sets obey a nesting condition. Multiconnected grayscale reconstruction is illustrated with an example of scale-space representation.

Reconstructing a topologically correct representation of the brain cortex surface from magnetic resonance images is important in several medical and neuroscience applications. Most previous methods have either made drastic changes to the underlying anatomy or relied on hand-editing. Recently, a new technique due to Shattuck and Leahy yields a fully-automatic procedure with little distortion of the underlying segmentation. The present paper can be considered as an extension of this approach to include arbitrary cut directions and arbitrary digital connectivities. A detailed analysis of the method's performance on 15 magnetic resonance brain images is provided.

Automatic mine detection is a critical issue in battle field management. This is expected to lead to better technologies that provide accurate and reliable detection of mines embedded in clutter. In this paper, we review a procedure for automatic mine detection in multispectral data provided by the Coastal Battlefield Reconnaissance and Analysis (COBRA) program. Our procedure is essentially a two-step method that employs the Maximum Noise Fraction (MNF) transform, a powerful enhancement tool for multispectral data, combined with nonlinear morphological operators that do the actual detection. Mathematical morphology is also used to account for the critical step of clutter estimation required by the MNF transform. Results obtained with available, truthed data, show the high success of the proposed method in meeting performance requirements. A low number of midsections is observed, whereas only a small number of false alarms is introduced by the algorithm. The results are better than the ones obtained by means of a constant false alarm rate (CFAR) algorithm provided along with the data.

A statistic tool for the detection of multivariate Boolean relationships is presented, with applications in the inference of gene regulatory mechanisms. A statistical test is developed for the detection of a nonzero discrete coefficient of determination (CoD) between predictor and target variables. This is done by framing the problem in the context of a stochastic logic model that naturally allows the inclusion of prior knowledge if available. The rejection region, p-value, statistical power, and confidence interval are derived and analyzed. Furthermore, the issue of multiplicity of tests due to presence of numerous candidate genes and logic relationships is addressed via FWER- and FDR-controlling approaches. The methodology is demonstrated by experiments using synthetic data and real data from a study on ionizing radiation (IR)-responsive genes. The results indicate that the proposed methodology is a promising tool for detection of gene regulatory relationships from gene-expression data. Software that implements the COD test is available online as an R package.

There has been considerable interest recently in the application of bagging in the classification of both gene-expression data and protein-abundance mass spectrometry data. The approach is often justified by the improvement it produces on the performance of unstable, overfitting classification rules under small-sample situations. However, the question of real practical interest is whether the ensemble scheme will improve performance of those classifiers sufficiently to beat the performance of single stable, nonoverfitting classifiers, in the case of small-sample genomic and proteomic data sets. To investigate that question, we conducted a detailed empirical study, using publicly-available data sets from published genomic and proteomic studies. We observed that, under t-test and RELIEF filter-based feature selection, bagging generally does a good job of improving the performance of unstable, overfitting classifiers, such as CART decision trees and neural networks, but that improvement was not sufficient to beat the performance of single stable, nonoverfitting classifiers, such as diagonal and plain linear discriminant analysis, or 3-nearest neighbors. Furthermore, as expected, the ensemble method did not improve the performance of these classifiers significantly. Representative experimental results are presented and discussed in this work.

We propose a method for automatic target detection and tracking in forward-looking infrared (FLIR) image sequences. We use morphological connected operators to extract and track targets of interest and remove undesirable clutter. The design of these operators is based on general size, connectivity, and motion criteria, using spatial intraframe and temporal interframe information. In a first step, an image sequence is filtered on a frame-by-frame basis to remove background and residual clutter and to enhance the presence of targets. Detections extracted from the first step are passed to a second step for motion-based analysis. This step exploits the spatiotemporal correlation of the data, stated in terms of a connectivity criterion along the time dimension. The proposed method is suitable for piplined implementation or time progressive coding/transmission, since only a few frames are considered at a time. Experimental results, obtained with real FLIR image sequences, illustrating a wide variety of target and clutter variability, demonstrate the effectiveness and robustness of the proposed method.