Search

Lung cancer causes more deaths globally than any other type of cancer. To determine the best treatment, detecting EGFR and KRAS mutations is of interest. However, non-invasive ways to obtain this information are not available. Furthermore, many times there is a lack of big enough relevant public datasets, so the performance of single classifiers is not outstanding. In this paper, an ensemble approach is applied to increase the performance of EGFR and KRAS mutation prediction using a small dataset. A new voting scheme, Selective Class Average Voting (SCAV), is proposed and its performance is assessed both for machine learning models and CNNs. For the EGFR mutation, in the machine learning approach, there was an increase in the sensitivity from 0.66 to 0.75, and an increase in AUC from 0.68 to 0.70. With the deep learning approach, an AUC of 0.846 was obtained, and with SCAV, the accuracy of the model was increased from 0.80 to 0.857. For the KRAS mutation, both in the machine learning models (0.65 to 0.71 AUC) and the deep learning models (0.739 to 0.778 AUC), a significant increase in performance was found. The results obtained in this work show how to effectively learn from small image datasets to predict EGFR and KRAS mutations, and that using ensembles with SCAV increases the performance of machine learning classifiers and CNNs. The results provide confidence that as large datasets become available, tools to augment clinical capabilities can be fielded.

Deep learning training typically starts with a random sampling initialization approach to set the weights of trainable layers. Therefore, different and/or uncontrolled weight initialization prevents learning the same model multiple times. Consequently, such models yield different results during testing. However, even with the exact same initialization for the weights, a lack of repeatability, replicability, and reproducibility may still be observed during deep learning for many reasons such as software versions, implementation variations, and hardware differences. In this article, we study repeatability when training deep learning models for segmentation and classification tasks using U-Net and LeNet-5 architectures in two development environments Pytorch and Keras (with TensorFlow backend). We show that even with the available control of randomization in Keras and TensorFlow, there are uncontrolled randomizations. We also show repeatable results for the same deep learning architectures using the Pytorch deep learning library. Finally, we discuss variations in the implementation of the weight initialization algorithm across deep learning libraries as a source of uncontrolled error in deep learning results.

© 2013 IEEE. This paper addresses the problem of active learning on multiple tasks, where labeled data are expensive to obtain for each individual task but the learning problems share some commonalities across multiple related tasks. To leverage the benefits of jointly learning from multiple related tasks and making active queries, we propose a novel active multitask learning approach based on trace norm regularized least squares. The basic idea is to induce an optimal classifier which has the lowest risk and at the same time which is closest to the true hypothesis. Toward this aim, we devise a new active selection criterion that takes into account not only the risk but also the excess risk, which measures the distance to the true hypothesis. Based on this criterion, our proposed algorithm actively selects the instance to query for its label based on the combination of the two risks. Experiments on both synthetic and real-world datasets show that our proposed algorithm provides superior performance as compared to other state-of-The-Art active learning methods.

This paper introduces a decomposition approach to address the problem of predicting different user activities at hour granularity over a long period of time. Our approach involves two steps. First, we used a temporal neural network ensemble to predict the number of each type of activity that occurred in a day. Second, we used a set of neural networks to assign the events to a user-repository pair in a particular hour. We focused this work on a subset of the public GitHub dataset that records the activities of over 2 million users on over 400,000 software repositories. Our experiments show we were able to predict hourly user-repo activity with reasonably low error. Our simulations are accurate for 1–3 weeks (168–504 hours) after inception, with accuracy gradually falling off. It was shown that activity on Twitter and Reddit increases the accuracy of activity prediction on GitHub for most events.

Mislabeled examples in the training data can severely affect the performance of supervised classifiers. In this paper, we present an approach to remove any mislabeled examples in the dataset by selecting suspicious examples as targets for inspection. We show that the large margin and soft margin principles used in support vector machines (SVM) have the characteristic of capturing the mislabeled examples as support vectors. Experimental results on two character recognition datasets show that one-class and two-class SVMs are able to capture around 85% and 99% of label noise examples, respectively, as their support vectors. We propose another new method that iteratively builds two-class SVM classifiers on the non-support vector examples from the training data followed by an expert manually verifying the support vectors based on their classification score to identify any mislabeled examples. We show that this method reduces the number of examples to be reviewed, as well as providing parameter independence of this method, through experimental results on four data sets. So, by (re-)examining the labels of the selective support vectors, most noise can be removed. This can be quite advantageous when rapidly building a labeled data set. HighlightsNovel method for label noise removal from data is introduced.It significantly reduces the required number of examples to be reviewed.Support vectors of SVM classifier can capture around 99% of label noise examples.Two-class SVM captures more label noise examples than one-class SVM classifierCombination of one-class and two-class SVM produces a marginal improvement.

Computed tomography (CT) is widely used during diagnosis and treatment of Non-Small Cell Lung Cancer (NSCLC). Current computer-aided diagnosis (CAD) models, designed for the classification of malignant and benign nodules, use image features, selected by feature selectors, for making a decision. In this paper, we investigate automated selection of different image features informed by different nodule size ranges to increase the overall accuracy of the classification. The NLST dataset is one of the largest available datasets on CT screening for NSCLC. We used 261 cases as a training dataset and 237 cases as a test dataset. The nodule size, which may indicate biological variability, can vary substantially. For example, in the training set, there are nodules with a diameter of a couple millimeters up to a couple dozen millimeters. The premise is that benign and malignant nodules have different radiomic quantitative descriptors related to size. After splitting training and testing datasets into three subsets based on the longest nodule diameter (LD) parameter accuracy was improved from 74.68% to 81.01% and the AUC improved from 0.69 to 0.79. We show that if AUC is the main factor in choosing parameters then accuracy improved from 72.57% to 77.5% and AUC improved from 0.78 to 0.82. Additionally, we show the impact of an oversampling technique for the minority cancer class. In some particular cases from 0.82 to 0.87.

Mislabeled examples are difficult to avoid while building large scale datasets. In this paper we discuss an efficient approach for finding those mislabeled examples. Our approach involves selecting a small number of potentially mislabeled examples for review by an expert. We demonstrate the utility of our method by finding some mislabeled examples in one large scale dataset (ImageNet). We found 92 errors by automatically selecting 3607 examples to review out of 22951 images from 18 classes. This requires reviewing 9 times fewer examples than the random sampling method to find an equivalent number of mislabels.

Prediction of survival time from brain tumor magnetic resonance images (MRI) is not commonly performed and would ordinarily be a time consuming process. However, current cross-sectional imaging techniques, particularly MRI, can be used to generate many features that may provide information on the patient’s prognosis, including survival. This information can potentially be used to identify individuals who would benefit from more aggressive therapy. Rather than using pre-defined and hand-engineered features as with current radiomics methods, we investigated the use of deep features extracted from pre-trained convolutional neural networks (CNNs) in predicting survival time. We also provide evidence for the power of domain specific fine-tuning in improving the performance of a pre-trained CNN’s, even though our data set is small. We fine-tuned a CNN initially trained on a large natural image recognition dataset (Imagenet ILSVRC) and transferred the learned feature representations to the survival time prediction task, obtaining over 81% accuracy in a leave one out cross validation.

Very large (VL) data or big data are any data that you cannot load into your computer's working memory. This is not an objective definition, but a definition that is easy to understand and one that is practical, because there is a dataset too big for any computer you might use; hence, this is VL data for you. Clustering is one of the primary tasks used in the pattern recognition and data mining communities to search VL databases (including VL images) in various applications, and so, clustering algorithms that scale well to VL data are important and useful. This paper compares the efficacy of three different implementations of techniques aimed to extend fuzzy c-means (FCM) clustering to VL data. Specifically, we compare methods that are based on 1) sampling followed by noniterative extension; 2) incremental techniques that make one sequential pass through subsets of the data; and 3) kernelized versions of FCM that provide approximations based on sampling, including three proposed algorithms. We use both loadable and VL datasets to conduct the numerical experiments that facilitate comparisons based on time and space complexity, speed, quality of approximations to batch FCM (for loadable data), and assessment of matches between partitions and ground truth. Empirical results show that random sampling plus extension FCM, bit-reduced FCM, and approximate kernel FCM are good choices to approximate FCM for VL data. We conclude by demonstrating the VL algorithms on a dataset with 5 billion objects and presenting a set of recommendations regarding the use of different VL FCM clustering schemes.

Lung cancer is caused by abnormal and uncontrolled growth of cells in the lungs and the mortality rate of lung cancer is the highest among all types of cancer. It can be identified and treated with the help of computed tomography (CT) images. For an automated classifier, identifying good features from an image is a key concern. Deep feature extraction using pre-trained convolutional neural networks has been successful for some image domains recently. In our study, we apply a pre-trained convolutional neural network (CNN) to extract deep features from lung cancer CT images and then train classifiers to predict short and long term survivors. The best accuracy of 77.5% was with a cropping approach using a decision tree classifier in a leave one out cross validation with ten features chosen using symmetric uncertainty feature ranking. We mixed extracted deep neural network features along with quantitative (traditional image) features and obtained the best accuracy of 82.5% with a nearest neighbor classifier in a leave one out cross validation using the symmetric uncertainty feature ranking algorithm.

Clustering is typically applied for data exploration when there are no or very few labeled data available. The goal is to find groups or clusters of like data. The clusters will be of interest to the person applying the algorithm. An objective function-based clustering algorithm tries to minimize (or maximize) a function such that the clusters that are obtained when the minimum/maximum is reached are homogeneous. One needs to choose a good set of features and the appropriate number of clusters to generate a good partition of the data into maximally homogeneous groups. Objective functions for clustering are introduced. Clustering algorithms generated from the given objective functions are shown, with a number of examples of widely used approaches discussed. © 2012 Wiley Periodicals, Inc. © 2012 Wiley Periodicals, Inc.

We describe an ensemble approach to learning salient regions from arbitrarily partitioned data. The partitioning comes from the distributed processing requirements of large-scale simulations. The volume of the data is such that classifiers can train only on data local to a given partition. Since the data partition reflects the needs of the simulation, the class statistics can vary from partition to partition. Some classes will likely be missing from some or even most partitions. We combine a fast ensemble learning algorithm with scaled probabilistic majority voting in order to learn an accurate classifier from such data. Since some simulations are difficult to model without a considerable number of false positive errors, and since we are essentially building a search engine for simulation data, we order predicted regions to increase the likelihood that most of the top-ranked predictions are correct (salient). Results from simulation runs of a canister being torn and from a casing being dropped show that regions of interest are successfully identified in spite of the class imbalance in the individual training sets. Lift curve analysis shows that the use of data driven ordering methods provides a statistically significant improvement over the use of the default, natural time step ordering. Significant time is saved for the end user by allowing an improved focus on areas of interest without the need to conventionally search all of the data.

Learning from imbalanced data sets presents a convoluted problem both from the modeling and cost standpoints. In particular, when a class is of great interest but occurs relatively rarely such as in cases of fraud, instances of disease, and regions of interest in large-scale simulations, there is a correspondingly high cost for the misclassification of rare events. Under such circumstances, the data set is often re-sampled to generate models with high minority class accuracy. However, the sampling methods face a common, but important, criticism: how to automatically discover the proper amount and type of sampling? To address this problem, we propose a wrapper paradigm that discovers the amount of re-sampling for a data set based on optimizing evaluation functions like the f-measure, Area Under the ROC Curve (AUROC), cost, cost-curves, and the cost dependent f-measure. Our analysis of the wrapper is twofold. First, we report the interaction between different evaluation and wrapper optimization functions. Second, we present a set of results in a cost- sensitive environment, including scenarios of unknown or changing cost matrices. We also compared the performance of the wrapper approach versus cost-sensitive learning methods--MetaCost and the Cost-Sensitive Classifiers--and found the wrapper to outperform the cost-sensitive classifiers in a cost-sensitive environment. Lastly, we obtained the lowest cost per test example compared to any result we are aware of for the KDD-99 Cup intrusion detection data set.

We describe an ensemble approach to learning from arbitrarily partitioned data. The partitioning comes from the distributed processing requirements of a large scale simulation. The volume of the data is such that classifiers can train only on data local to a given partition. As a result of the partition reflecting the needs of the simulation, the class statistics can vary from partition to partition. Some classes will likely be missing from some partitions. We combine a fast ensemble learning algorithm with probabilistic majority voting in order to learn an accurate classifier from such data. Results from simulations of an impactor bar crushing a storage canister and from facial feature recognition show that regions of interest are successfully identified in spite of the class imbalance in the individual training sets.

Accurate computer-aided prediction of survival time for brain tumor patients requires a thorough understanding of clinical data, since it provides useful prior knowledge for learning models. However, to simplify the learning process, traditional settings often assume datasets with equally distributed classes, which clearly does not reflect a typical distribution. In this paper, we investigate the problem of mining knowledge from an imbalanced dataset (i.e., a skewed distribution) to predict survival time. In particular, we propose an algorithmic framework to predict survival groups of brain tumor patients using multi-modality MRI data. Both an imbalanced distribution and classifier design are jointly considered: 1) We used the Synthetic Minority Over-sampling Technique to compensate for the imbalanced distribution; 2) A predictive linear regression model was adopted to learn a pair of class-specific dictionaries, which were derived from reformulated balanced data. We tested the proposed framework using a dataset of 42 patients with Glioblastoma Multiforme (GBM) tumors whose scans were obtained from the cancer genome atlas (TCGA). Experimental results showed that the proposed method achieved 95.24% accuracy.

This paper presents a new approach for relational data partitioning using the notion of dominant sets. A dominant set is a subset of data points satisfying the constraints of internal homogeneity and external in-homogeneity, i.e. a cluster. However, since any subset of a dominant set cannot be a dominant set itself, dominant sets tend to be compact sets. Hence, in this paper, we present a novel approach to enumerate well distributed clusters where the number of clusters need not be known. When the number of clusters is known, in order to search the solution space appropriately, after finding each dominant set, data points are partitioned into two disjoint subsets of data points using spectral graph image segmentation methods to enumerate the other well distributed dominant sets. For the latter case, we introduce a new hierarchical approach for relational data partitioning using a new class of evolutionary game theory dynamics called InImDynamics which is very fast and linear, in computational time, with the number of data points. In this regard, at each level of the proposed hierarchy, Dunn's index is used to find the appropriate number of clusters. Then the objects are partitioned based on the projected number of clusters using game theoretic relations. The same method is applied to each partition to extract its underlying structure. Although the resulting clusters exist in their equivalent partitions, they may not be clusters of the entire data. Hence, they are checked for being an actual cluster and if they are not, they are extended to an existing cluster of the data. The approach can also be used to assign unseen data to existing clusters, as well.

The diversity of an ensemble of classifiers can be calculated in a variety of ways. Here a diversity metric and a means for altering the diversity of an ensemble, called “thinning”, are introduced. We evaluate thinning algorithms created by several techniques on 22 publicly available datasets. When compared to other methods, our percentage correct diversity measure shows a greatest correlation between the increase in voted ensemble accuracy and the diversity value. Also, the analysis of different ensemble creation methods indicates that they generate different levels of diversity. Finally, the methods proposed for thinning show that ensembles can be made smaller without loss in accuracy.

Error based pruning can be used to prune a decision tree and it does not require the use of validation data. It is implemented in the widely used C4.5 decision tree software. It uses a parameter, the certainty factor, that affects the size of the pruned tree. Several researchers have compared error based pruning with other approaches, and have shown results that suggest that error based pruning results in larger trees that give no increase in accuracy. They further suggest that as more data is added to the training set, the tree size after applying error based pruning continues to grow even though there is no increase in accuracy. It appears that these results were obtained with the default certainty factor value. Here, we show that varying the certainty factor allows significantly smaller trees to be obtained with minimal or no accuracy loss. Also, the growth of tree size with added data can be halted with an appropriate choice of certainty factor. Methods of determining the certainty factor are discussed for both small and large data sets. Experimental results support the conclusion that error based pruning can be used to produce appropriately sized trees with good accuracy when compared with reduced error pruning.

Clustering is a useful approach in image segmentation, data mining, and other pattern recognition problems for which unlabeled data exist. Fuzzy clustering using fuzzy c-means or variants of it can provide a data partition that is both better and more meaningful than hard clustering approaches. The clustering process can be quite slow when there are many objects or patterns to be clustered. This paper discusses the algorithm brFCM, which is able to reduce the number of distinct patterns which must be clustered without adversely affecting the partition quality. The reduction is done by aggregating similar examples and then using a weighted exemplar in the clustering process. The reduction in the amount of clustering data allows a partition of the data to be produced faster. The algorithm is applied to the problem of segmenting 32 magnetic resonance images into different tissue types and the problem of segmenting 172 infrared images into trees, grass and target. Average speed-ups of as much as 59-290 times a traditional implementation of fuzzy c-means were obtained using brFCM, while producing partitions that are equivalent to those produced by fuzzy c-means.

Bagging forms a committee of classifiers by bootstrap aggregation of training sets from a pool of training data. A simple alternative to bagging is to partition the data into disjoint subsets. Experiments with decision tree and neural network classifiers on various datasets show that, given the same size partitions and bags, disjoint partitions result in performance equivalent to, or better than, bootstrap aggregates (bags). Many applications (e.g., protein structure prediction) involve use of datasets that are too large to handle in the memory of the typical computer. Hence, bagging with samples the size of the data is impractical. Our results indicate that, in such applications, the simple approach of creating a committee of n classifiers from disjoint partitions each of size 1/n (which will be memory resident during learning) in a distributed way results in a classifier which has a bagging-like performance gain. The use of distributed disjoint partitions in learning is significantly less complex and faster than bagging.

Soft tissue sarcomas (STS) are a heterogenous group of malignant tumors comprised of more than 50 histologic subtypes. Based on spatial variations of the tumor, predictions of the development of necrosis in response to therapy as well as eventual progression to metastatic disease are made. Optimization of treatment, as well as management of therapy-related side effects, may be improved using progression information earlier in the course of therapy. Multimodality pre- and post-gadolinium enhanced magnetic resonance images (MRI) were taken before and after treatment for 30 patients. Regional variations in the tumor bed were measured quantitatively. The voxel values from the tumor region were used as features and a fuzzy clustering algorithm was used to segment the tumor into three spatial regions. The regions were given labels of high, intermediate and low based on the average signal intensity of pixels from the post-contrast T1 modality. These spatially distinct regions were viewed as essential meta-features to predict the response of the tumor to therapy based on necrosis (dead tissue in tumor bed) and metastatic disease (spread of tumor to sites other than primary). The best feature was the difference in the number of pixels in the highest intensity regions of tumors before and after treatment. This enabled prediction of patients with metastatic disease and lack of positive treatment response (i.e. less necrosis). The best accuracy, 73.33%, was achieved by a Support Vector Machine in a leave-one-out cross validation on 30 cases predicting necrosis < 90% post treatment and metastasis.

A fuzzy expert system must do rule chaining differently than a nonfuzzy expert system. In particular, any rule that can fire with a particular linguistic variable in its consequent must fire before any rule whose antecedent conditions depend upon the resultant fuzzy set value of the consequent linguistic variable is allowed to fire. The dependent rules would be considered in a chain with the fuzzy rules which generate or assert the needed fuzzy linguistic variable. A recent paper by J. Pan et al. (1998) points out that a version of the FuzzyCLIPS expert system shell does not operate with chained fuzzy rules as one would expect. They introduce FuzzyShell which is described as the only known shell to have the expected fuzzy rule chaining performance. We show several approaches to obtaining the desired behavior in FuzzyCLIPS. Further, a potential pitfall with the FuzzyShell approach to dealing with chaining is pointed out.

A method for chemical process fault diagnosis using semi-quantitative model generated behavior envelopes is described. The method generates a sequence of rules for each fault class, with any rule in a sequence valid within the bounds of its time interval. This can be viewed as a qualitative description of the trend of numerical sensor measurements. For each variable in each fault class two sequences of episodic fuzzy rules are automatically generated, one for the lower and one for the upper numerical behavior envelope. The diagnostic system monitors a process via the measured sensors. The measurements are matched against the fuzzy rules for the current time in the rule base. In the case of an overlapping region defined by behavior envelopes, the distance introduced and time based fault belief scaling allows ranking of fault candidates. A novel abnormal situation will not pass the introduced system undetected due to a novel class detection mechanism. In two case studies, the system detected the correct fault even in cases of nearly total overlapped fault regions bounded by behavior envelopes.

The paper introduces two ways to develop fuzzy rules, using decision trees, from data with continuous valued inputs and outputs. A key problem is dealing with continuous outputs. Output classes are created, then a crisp decision tree is created using a set of fuzzy output classes and letting each training example to partially belong to classes. Alternatively, a discrete set of fuzzy outputs classes is created that includes a selected group of overlaps, such as class A.75/class B.25. Training examples are then provided to a standard decision tree learning program, such as C4.5. In both cases, fuzzy rules are extracted from the resulting decision tree. Output classes must be created for a case in which examples belong to discrete but overlapping classes. We discuss tradeoffs of the two approaches to output class creation. An example of system performance uses a discrete set of overlapping classes on the Box-Jenkins gas furnace prediction problem and a function approximation problem. The learned rules provide effective control and function approximation.

This paper presents a multistage random sampling fuzzy c-means-based clustering algorithm, which significantly reduces the computation time required to partition a data set into c classes. A series of subsets of the full data set are used to create initial cluster centers in order to provide an approximation to the final cluster centers. The quality of the final partitions is equivalent to those created by fuzzy c-means. The speed-up is normally a factor of 2–3 times, which is especially significant for high-dimensional spaces and large data sets. Examples of the improved speed of the algorithm in two multi-spectral domains, magnetic resonance image segmentation and satellite image segmentation, are given. The results are compared with fuzzy c-means in terms of both the time required and the final resulting partition. Significant speedup is shown in each example presented in the paper. Further, the convergence properties of fuzzy c-means are preserved.

A novel hybrid symbolic-connectionist approach to machine learning is introduced and applied to fault diagnosis of a hydrocarbon chlorination plant. The learning algorithm addresses the knowledge acquisition problem by developing and maintaining the knowledge base through instance based inductive learning. The performance of the learning system is discussed in terms of the knowledge extracted from example cases and its classification accuracy on the test cases. Results indicate that the introduced system is a promising alternative to neural networks for fault diagnosis and a complement to expert systems.

Automated prediction of patient-specific disease progression can significantly contribute to clinical treatment. This paper presents a computer-assisted framework to tackle the survival time prediction problem. Inspired by the assumption that niche tumor regions may play a significant role in cancer diagnosis, we explore local visual variations from multiple MRI sequences. The research consists of three parts: 1) the extraction of multi-scale Local Binary Patterns (LBP) to describe the visual variations, 2) a supervised forward feature selection approach, called the Feature Ranking Model (FRM) which captures single feature predictive ability efficiently, and combines the top features to form a feature subset, 3) We cast the clinical survival time prediction task as a binary category classification problem. We tested the framework using a dataset of 32 cases collected from The Cancer Genome Atlas (TCGA). We obtained a 93.75% accuracy rate for the prediction of survival time.

Clustering large data sets has become very importantas the amount of available unlabeled data increases. Single Pass Fuzzy C Means (SPFCM) is useful when memory is too limited to load the whole data set. The main idea is to divide dataset into several chunks and to apply FCM to each chunk. SPFCM uses the weighted cluster centers of the previous chunk in the next chunks. Although when the number of chunks is increased, the algorithm shows sensitivity to the order the data processed. Hence, we improved SPFCM by recognizing boundary and noisy data in each chunk and using it to influence clustering in the next chunks. In this regard, the proposed approach transfers the boundary and noisy data as well as the weighted cluster centers to the next chunks. We show that our proposed approach is significantly less sensitive to the order in which the data is loaded in each chunk.

Sky surveys for Astronomy are expected to generate 2.5 petabytes a year. Electronic medical records hold the promise of treatment comparisons, grouping patients by outcomes but will be contained in petabyte data storage. We can store lots of data and much of it wont have labels. How can we analyze or explore the data? Unsupervised clustering, fuzzy, possibilistic or probabilistic will allow us to group data. However, the algorithms scale poorly in terms of computation time as the data gets large and are impractical without modification when the data exceeds the size of memory. We will explore distributed clustering, stream data clustering and subsampling approaches to enable scalable clustering. Examples will show that one can scale to build good models of the data without necessarily seeing all the data and, if needed, modified algorithms can be applied to terabytes and more of data.

All clustering algorithms process unlabeled data and, consequently, suffer from two problems: (P1) choosing and validating the correct number of clusters and (P2) insuring that algorithmic labels correspond to meaningful physical labels. Clustering algorithms such as hard and fuzzy c-means, based on optimizing sums of squared errors objective functions, suffer from a third problem: (P3) a tendency to recommend solutions that equalize cluster populations. The semi-supervised c-means algorithms introduced in this paper attempt to overcome these problems domains where a few data from each clas can be labeled. Segmentation of magnetic resonance images is a problem of this type and we use it to illustrate the new algorithm. Our examples show that the semi-supervised approach provides MRI segmentations that are superior to ordinary fuzzy c-means and to the crisp k-nearest neighbor rule and further, that the new method ameliorates (P1)-(P3).

A novel hybrid symbolic approach to machine learning is illustrated for fault diagnosis of a hydrocarbon chlorination plant. The learning algorithm addressed the knowledge acquisition problem by developing and maintaining the knowledge base through inductive learning. The performance of the learning system is discussed in terms of the knowledge extracted from example cases and its clasiffication accuracy on the test cases. Results indicate that the introduced system is a promising alternative to neural networks for fault diagnosis and a complement to expert systems.

“Radiomics” refers to the extraction and analysis of large amounts of advanced quantitative imaging features with high throughput from medical images obtained with computed tomography (CT), positron emission tomography (PET) or magnetic resonance imaging (MRI). Importantly, these data are designed to be extracted from standard-of-care images, leading to a very large potential subject pool. Radiomic data are in a mineable form that can be used to build descriptive and predictive models relating image features to phenotypes or gene-protein signatures. The core hypothesis of radiomics is that these models, which can include biological or medical data, can provide valuable diagnostic, prognostic or predictive information. The radiomics enterprise can be divided into distinct processes, each with its own challenges that need to be overcome: (i) image acquisition and reconstruction (ii) image segmentation and rendering (iii) feature extraction and feature qualification (iv) databases and data sharing for eventual (v) ad hoc informatic analyses. Each of these individual processes poses unique challenges. For example, optimum protocols for image acquisition and reconstruction have to be identified and harmonized. Also, segmentations have to be robust and involve minimal operator input. Features have to be generated that robustly reflect the complexity of the individual volumes, but cannot be overly complex or redundant. Furthermore, informatics databases that allow incorporation of image features and image annotations, along with medical and genetic data have to be generated. Finally, the statistical approaches to analyze these data have to be optimized, as radiomics is not a mature field of study. Each of these processes will be discussed in turn, as well as some of their unique challenges and proposed approaches to solve them. The focus of this article will be on images of non-small cell lung cancer, NSCLC.

Fuzzy c-means (FCM) is a well-known algorithm for clustering data, but for large datasets termination takes significant time. As a result, a number of scalable algorithms based on FCM have been developed. In this paper, four scalable variants of FCM are compared to the base algorithm. Runtime and three quality metrics are calculated. Experimental results using five data sets are analyzed. We show that the scalable algorithms are consistent with regard to speedup, but less consistent when quality is considered. The three quality measures are shown to have little correlation and vary in magnitude across datasets. Selection of a scalable algorithm must consider a tradeoff between the quality of results and speed. Of the variants, single pass FCM (SPFCM) is fastest with good fidelity to FCM, and extensible fast FCM (eFFCM) is almost as fast as SPFCM (as implemented) with very good fidelity to FCM. Random FCM is the fastest overall and often close in quality to FCM. The results showed that scalable algorithms occasionally produce better optimized results than FCM.

The authors' main contribution is to build upon their earlier efforts by expanding the tissue model concept to cover a brain volume. Furthermore, processing time is reduced and accuracy is enhanced by the use of knowledge propagation, where information derived from one slice is made available to succeeding slices as additional knowledge. The system is organized as follows. Each MR slice is initially segmented by an unsupervised fuzzy c-means clustering algorithm. Next, an expert system uses model-based recognition techniques to locate a landmark, called a focus-of attention tissue. Qualitative models of slices of brain tissue are defined and matched with their instances from imaged slices. If a significant deformation is detected in a tissue, the slice is classified to be abnormal and volume processing halts. Otherwise, the expert system locates the next focus-of-attention tissue, based on a hierarchy of expected tissues. This process is repeated until either a slice is classified as abnormal or all tissues of the slice are labeled. If the slice is determined to be abnormal, the entire volume is also considered abnormal and processing halts. Otherwise, the system will proceed to the next slice and repeat the classification steps until all slices that comprise the volume are processed. A rule-based expert system tool, CLIPS, is used to organize the system. Low level modules for image processing and high level modules for image analysis, all written in the C language, are called as actions from the right hand sides of the rules. The system described here is an attempt to provide completely automatic segmentation and labeling of normal volunteer brains. The absolute accuracy of the segmentations has not yet been rigorously established. The relative accuracy appears acceptable. Efforts have been made to segment an entire volume (rather than merging a set of segmented slices) using supervised pattern recognition techniques or unsupervised fuzzy clustering. However, there is sometimes enough data nonuniformity between slices to prevent satisfactory segmentation. >

Typically, thousands of gene expression levels are recorded for a group of patients, leading to the situation where the number of features far exceeds the number of examples. To combat this, researchers would want to combine gene expression data collected at different sites into one data set to reduce the magnitude of the difference between the number of features (genes) and examples (samples). This makes gene selection a critical component of any process to build models using gene expression data. For instance, in the domain of ordering cancer patients based on survival time, one might assume that utilizing genes related to cancer development and progression will allow the best model to be built. In this paper, we explore two different gene selection techniques and examine how well the genes selected compare between methods. We also check gene set consistency between data sets collected using the same protocols at different research institutions. It is shown that gene selection can result in very different sets given different training data.

Support vector machines are binary classifiers that can implement multi-class classifiers by creating a classifier for each possible combination of classes or for each class using a one class versus all strategy. Feature selection algorithms often search for a single set of features to be used by each of the binary classifiers. This ignores the fact that features that may be good discriminators for two particular classes might not do well for other class combinations. As a result, the feature selection process may not include these features in the common set to be used by all support vector machines. It is shown that by selecting features for each binary class combination, overall classification accuracy can be improved (as much as 2.1%), feature selection time can be significantly reduced (speed up of 3.2 times), and time required for training a multi-class support vector machine is reduced. Another benefit of this approach is that considerably less time is required for feature selection when additional classes are added to the training data. This is because the features selected for the existing class combinations are still valid, so that feature selection only needs to be run for the new class combinations created.

We show that using random forests and distance-based outlier partitioning with ensemble voting methods for supervised learning of anomaly detection provide similar accuracy results when compared to the same methods without partitioning. Further, distance-based outlier and one-class support vector machine partitioning and ensemble methods for semi-supervised learning of anomaly detection also compare favorably to the corresponding nonensemble methods. Partitioning and ensemble methods would be required for very large datasets that need distributed computing approaches. ROC curves often show significant improvement from increased true positives in the low false positive range for ensemble methods used on several datasets.

This paper describes the SC-net system that has been developed to provide expert systems capability augmented with learning in a hybrid connectionist/symbolic approach. A distributed connectionist representation of cells connected by links is used to represent symbolic knowledge. Rules may be directly encoded in the connectionist network or learned from examples. The learning method is a form of instance-based learning in which some of the individual instances in the training set are encoded by adding structure to the network and others cause modifications to biases in the network. Both continuous and nominal attributes are directly represented in the network structure. A limited form of variables in the form of attribute value bindings on the right-hand side of rules is supported. Relational comparators in the form of cell groups are also supported. Relational comparators and attribute value structures are represented by groups of connected cells in the network. The learning algorithm is presented and methods for providing generalization in an instance-based connectionist environment are presented. Empirical results are presented, which include learning in domains (fevers and gems) that contain uncertainty and the well-known iris, and soybean data sets together with a real world domain for semiconductor wafer fault diagnosis. The generalization ability of the learned network is shown to be good in several domains including iris. The system is shown to compare favorably with a nonneural instance-based learning algorithm IBL.

Clustering large data has the problem of not having all the data fit in the memory at one time. It is a challenge to apply fuzzy clustering algorithms to get a partition in a timely manner. In this paper, we compare the online fuzzy clustering and single pass fuzzy clustering algorithms, which can be used to cluster very large data sets which might be treated as streaming data, with fuzzy c-means. We introduce more meaningful partition comparison measurements based on cluster center location instead of using the difference in R m value. We obtained results on several large volumes of magnetic resonance images which indicate that the online FCM algorithm produces partitions which are very close to what you could get if you clustered all the data at one time. We also show online FCM outperforms single pass FCM and it can process streaming data as it comes without degradation in most cases.

Many genes and a small number of samples are problematic characteristics of microarray datasets. We investigated the impact on classification accuracy of gene selection approaches on filtered-to-200-gene datasets. Four datasets were used with 3 filters: Student's t-test, information gain, and reliefF. We applied Iterative Feature Perturbation (IFP) and Recursive Feature Elimination (SVM-RFE) for further gene selection. Both approaches resulted in accuracy improvement when used with the t-test-filtered datasets, but not when information gain or reliefF were used. An AUC analysis of the IFP and SVM-RFE accuracy curves indicated that both methods reached the highest AUC values after t-test filtering. A statistical analysis of accuracy across the best 50 genes using the Friedman/Holm test showed that IFP and SVM-RFE were significantly more accurate more often when applied to the t-test-filtered gene sets. Surprisingly, the simple t-test, applied as a filter, results in the best overall SVM accuracy and is at least as accurate as the other, more complicated filter methods.

An ensemble of clustering solutions or partitions may be generated for a number of reasons. If the data set is very large, clustering may be done on tractable size disjoint subsets. The data may be distributed at different sites for which a distributed clustering solution with a final merging of partitions is a natural fit. In this paper, two new approaches to combining partitions, represented by sets of cluster centers, are introduced. The advantage of these approaches is that they provide a final partition of data that is comparable to the best existing approaches, yet scale to extremely large data sets. They can be 100,000 times faster while using much less memory. The new algorithms are compared against the best existing cluster ensemble merging approaches, clustering all the data at once and a clustering algorithm designed for very large data sets. The comparison is done for fuzzy and hard-k-means based clustering algorithms. It is shown that the centroid-based ensemble merging algorithms presented here generate partitions of quality comparable to the best label vector approach or clustering all the data at once, while providing very large speedups.

There are many applications which may be done by an expert system in real time, if the system is capable of real time response. the first Lisp- and Prolog-based expert systems have typically been too slow for real time response. This has lead to an effort to use other languages, the development of fast pattern matching techniques, and other methods of improving the speed of expert systems. Another approach to developing faster expert systems is to make use of the emerging parallel processing computer technology. A further use for parallelism is to allow reasonable response time for large knowledge bases. the size of knowledge bases may become as large as 20,000 chunks of knowledge (and more) in the near future in medical and space applications. This article describes the use of parallel processing in the EMYCIN backward chained rule-based model. Performance on two examples of shared memory multiprocessors is presented and contrasted with earlier simulations.

In this paper we present an unpolished expert system development tool, based on a connectionist architecture for knowledge representation. Our work is centered around a connectionist expert system, which can be expanded, and updated through learning of sample domain specific cases [1,4]. A cell recruitment learning algorithm [2] capable of forgetting previously learned facts by learning new ones is incorporated. Using this learning mechanism, we let the system learn a knowledge base on classifying the creditworthiness of credit applicants. The knowledge base consisted of 50 credit cases and was obtained from [11]. It will be shown that the fuzznet system is capable of learning such a model, with only a small amount of the cases being presented to it for learning. In all examples learned, an acceptable model was derived (based on the average prognostic error of actual and expected output for creditworthiness). The input features and their corresponding outputs (which were learned) are all fuzzy (uncertain) at the time of learning. So far implementations of connectionist expert systems, either only allowed for crisp inputs when placed in the learning mode, or only supported uncertainty when simulated [1, 8, 9, 10]. This example will further demonstrate the versatility of the fuzznet system, when dealing with uncertainty in the inputs and outputs when being placed in the learn mode, or while being simulated.

Support vector machines can be trained to be very accurate classifiers and have been used in many applications. However, the training and to a lesser extent prediction time of support vector machines on very large data sets can be very long. This paper presents a fast compression method to scale up support vector machines to large data sets. A simple bit reduction method is applied to reduce the cardinality of the data by weighting representative examples. We then develop support vector machines trained on the weighted data. Experiments indicate that the bit reduction support vector machine produces a significant reduction in the time required for both training and prediction with minimum loss in accuracy. It is also shown to, typically, be more accurate than random sampling when the data are not over-compressed.

Complex simulations can generate very large amounts of data stored disjointedly across many local disks. Learning from this data can be problematic due to the difficulty of obtaining labels for the data. We present an algorithm for the application of semi-supervised learning on disjoint data generated by complex simulations. Our semi-supervised technique shows a statistically significant accuracy improvement over supervised learning using the same underlying learning algorithm and requires less labeled data for comparable results.

We describe an ensemble approach to learning salient regions from data partitioned according to the distributed processing requirements of large-scale simulations. The volume of the data is such that classifiers can train only on data local to a given partition. Classes will likely be missing from some, or even most, partitions. We combine a fast ensemble learning algorithm with scaled probabilistic majority voting in order to learn an accurate classifier from such data. We order predicted regions to increase the likelihood that most of the initial set of presented regions are salient. Results from a simulated casing being dropped show that regions of interest are successfully identified and ordered. This approach is much faster than manually browsing and visualizing terabyte or larger simulations to find regions of interest.

Microarray datasets are often limited to a small number of samples with a large number of gene expressions. Therefore, dimensionality reduction through a feature/gene selection process is highly important for classification purposes. In this paper, a feature perturbation method we previously introduced is applied to do gene selection from microarray data. A publicly available colon cancer dataset is used in our experiments. In comparison with SVM-RFE, our method is better with feature sets of between 10 and 80, however for less than 10 features SVM-RFE results in higher accuracy. An analysis of the area under the curve of the feature perturbation method for the top 50 and 25 features is performed, aiming to determine the proper amount of noise to be applied. We show that a good set of small features/genes can be found using the feature perturbation method.

Clustering streaming data presents the problem of not having all the data available at one time. Further, the total size of the data may be larger than will fit in the available memory of a typical computer. If the data is very large, it is a challenge to apply fuzzy clustering algorithms to get a partition in a timely manner. In this paper, we present an online fuzzy clustering algorithm which can be used to cluster streaming data, as well as very large data sets which might be treated as streaming data. Results on several large volumes of magnetic resonance images show that the new algorithm produces partitions which are very close to what you could get if you clustered all the data at one time. So, the algorithm is an accurate approach for online clustering.

Clustering is the most typical way to group unlabeled data. Today, there are very large unlabeled data sets available. Many of these data sets are too large to fit in the memory of a typical computer. Some of these data sets are so large that they can only be treated as data streams because not all of the data can be stored in a cost-effective manner. Fuzzy clustering algorithms are known to be very useful on small to medium-size data sets. This talk focuses on how to make some well understood classic fuzzy clustering algorithms scale to very large data sets and streaming data sets. The goal is to be able to create a data partition that reflects the whole data set, but requires practical computation times. In particular, we show that the fuzzy c-means families of algorithms can be scaled to provide data partitions that are very close and potentially identical to what you would get if you were able to cluster all the data. The general idea is to cluster subsets of the data and create weighted examples from the subsets. The weighted examples from a previous partition(s) are used with new data to create a new partition which reflects the examples currently loaded in memory and those partitioned previously. This process can be repeated until all the data has been clustered. Several variations on the theme of summarizing previous partitions with a set of weighted examples are given. Some history can be ignored, for example, in time changing data streams. One could also choose to cluster summarizations. Experimental data sets include several which contain tens of millions of examples, as well as streaming data sets. Results from real-world data sets show excellent partitions are obtained. For tractable size data sets it is shown that the partitions are comparable to those from fuzzy c-means when it clusters all the data.

Before new medical treatments become available to the public, clinicians must conduct extensive trials to determine the efficacy of the novel therapy. In order for the clinical trial to be successful, a significant number of patients with an appropriate set of medical conditions must be accrued. We have implemented a web-based expert system at the H. Lee Moffitt Cancer Center & Research Institute in the Gastrointestinal Tumor Clinic (GITC) to help physicians screen patients for phase II trials. Our system allows physicians to screen a patient for multiple trials simultaneously. Our experiments have shown that adaptation of the system into a clinical environment and the success of the system are related to the amount of time physicians are willing to spend entering data. We also found significant regulatory issues (HIPAA) that make implementation challenging.