Search

Computer-aided image analysis for better understanding of images has been time-honored approaches in the medical computing field. In the conventional machine learning approach, the domain experts in medical images are mandatory for image annotation that subsequently to be used for feature engineering. However, in deep learning, a big jump has been made to help the researchers do segmentation, feature extraction, classification, and detection from raw medical images obtained using digital breast tomosynthesis, digital mammography, magnetic resonance imaging, and ultrasound imaging modalities. As a result, deep learning (DL) has gained a state-of-the-art in many application areas, for example, breast cancer image analysis. In this survey paper, we reviewed the most common breast cancer imaging modalities, public, most cited and recently updated breast cancer databases, histopathological based breast cancer image analysis, and DL application types in medical image analysis. We finally conclude by pointing out the research gaps to be addressed in the future.

Recognition of isolated handwritten digits is a well-studied research problem and several models show high recognition accuracy on different standard datasets. But the same is not true while we consider recognition of handwritten digit strings although it has many real-life applications like bank cheque processing, postal code recognition, and numeric field understanding from filled-in form images. The problem becomes more difficult when digits in the string are not neatly written which is commonly seen in freestyle handwriting. The performance of any such model primarily suffers due to the presence of touching digits in the string. To handle these issues, in the present work, we first use a deep autoencoder based segmentation technique for isolating the digits from a handwritten digit string, and then we pass the isolated digits to a Residual Network (ResNet) based recognition model to obtain the machine-encoded digit string. The proposed model has been evaluated on the Computer Vision Lab (CVL) Handwritten Digit Strings (HDS) database, used in HDSRC 2013 competition on handwritten digit string recognition, and a competent result with respect to state-of-the-art techniques has been achieved.

Several approaches have been proposed for the analysis of pain-related facial expressions. These approaches range from common classification architectures based on a set of carefully designed handcrafted features, to deep neural networks characterised by an autonomous extraction of relevant facial descriptors and simultaneous optimisation of a classification architecture. In the current work, an end-to-end approach based on attention networks for the analysis and recognition of pain-related facial expressions is proposed. The method combines both spatial and temporal aspects of facial expressions through a weighted aggregation of attention-based neural networks��� outputs, based on sequences of Motion History Images (MHIs) and Optical Flow Images (OFIs). Each input stream is fed into a specific attention network consisting of a Convolutional Neural Network (CNN) coupled to a Bidirectional Long Short-Term Memory (BiLSTM) Recurrent Neural Network (RNN). An attention mechanism generates a single weighted representation of each input stream (MHI sequence and OFI sequence), which is subsequently used to perform specific classification tasks. Simultaneously, a weighted aggregation of the classification scores specific to each input stream is performed to generate a final classification output. The assessment conducted on both the BioVid Heat Pain Database (Part A) and SenseEmotion Database points at the relevance of the proposed approach, as its classification performance is on par with state-of-the-art classification approaches proposed in the literature., publishedVersion

Deep learning has attracted a lot of attention in various fields over the past few years, including facial expression recognition. However, applying deep learning techniques to facial expression recognition is not straightforward. There are several drawbacks for successful deep expression recognition systems. Besides the lack of sufficient training data, facial expressions convey various inter-personal morphological and character differences. Therefore, an expression recognition network often suffers from overfitting and missing generalizability. However, multiple learning techniques, generally known as domain adaptation, have been proposed to address the lack of sufficient data and missing variance. Consequently, facial expression recognition may profit from domain adaptation. In this paper, we evaluate the applicability of deep domain adaptation for facial expression recognition. We describe two domain adaptation frameworks, one for single frame facial expression analysis and one for sequence-based facial expression analysis based on the Self-Ensembling method defined in [1]. The former is evaluated on the CK+ dataset [2], [3], the latter on the SenseEmotion database [4] of the University of Ulm. Our results indicate that domain adaptation is mostly applicable for person-specific facial expression recognition.

The main concept behind designing a multiple classifier system is to combine a number of classifiers such that the resulting system succeeds to topple the individual classifiers by pooling together the decisions of all classifiers. Uniting relatively simple pattern recognition models with limited performance is commonly found in the literature. It performs better when each learner be trained well, but different learners have different working principles which adds diversity in the ensemble. In this paper, we first select three optimal subsets of features using three different filter methods namely Mutual Information (MI), Chi-square, and Anova F-Test. Then with the selected features we build three learning models using Multi-layer Perceptron (MLP) based classifier. Class membership values provided by these three classifiers for each sample are concatenated which is then fed to next MLP based classifier. Experimentation performed on five UCI Machine Learning Repository, namely Arrhythmia, Ionosphere, Hill-Valley, Waveform, Horse Colic shows the effectiveness of the proposed ensemble model.

The paper presents a probabilistic echo-state network (π-ESN) for density estimation over variable-length sequences of multivariate random vectors. The π-ESN stems from the combination of the reservoir of an ESN and a parametric density model based on radial basis functions. A constrained maximum likelihood training algorithm is introduced, suitable for sequence classification. Extensions of the algorithm to unsupervised clustering and semi-supervised learning (SSL) of sequences are proposed. Experiments in emotion recognition from speech signals are conducted on the WaSeP? dataset. Compared with established techniques, the π-ESN yields the highest recognition accuracies, and shows interesting clustering and SSL capabilities.

Supervised learning algorithms restrict the training of classification models to the classes of interest. Other related classes are typically neglected in this process and are not involved in the final decision rule. Nevertheless, the analysis of these foreign samples and their labels might provide additional information on the classes of interest. By revealing common patterns in foreign classification tasks it might lead to the identification of structures suitable for the original classes. This principle is used in the field of transfer learning. In this work, we investigate the use of foreign classes for the feature selection process of binary classifiers. While the final classification model is trained according to the traditional supervised learning scheme, its feature signature is designed for separating a pair of foreign classes. We systematically analyse these classifiers in \(10 \times 10\) cross-validation experiments on microarray datasets with multiple diagnostic classes. For each evaluated classification model, we observed foreign feature combinations that outperformed at least 90% of those feature sets designed for the original diagnostic classes on at least 88.9% of all datasets.

The individual nature of physiological measurements of human affective states makes it very difficult to transfer statistical classifiers from one subject to another. In this work, we propose an approach to incorporate unlabeled data into a supervised classifier training in order to conduct an emotion classification. The key idea of the method is to conduct a density estimation of all available data (labeled and unlabeled) to create a new encoding of the problem. Based on this a supervised classifier is constructed. Further, numerical evaluations on the EmoRec II corpus are given, examining to what extent additional data can improve classification and which parameters of the density estimation are optimal.

A continuous multimodal human affect recognition for both arousal and valence dimensions in a non-acted spontaneous scenario is investigated in this paper. Different regression models based on Random Forests and Echo State Networks are evaluated and compared in terms of robustness and accuracy. Moreover, an extension of Echo State Networks to a bi-directional model is introduced to improve the regression accuracy. A hybrid method using Random Forests, Echo State Networks and linear regression fusion is developed and applied on the test subset of the AVEC16 challenge. Finally, the label shift and prediction delay is discussed and an annotator specific regression model, as well as fusion architecture, is proposed for future work.

It is essential for the advancement of human-centered multimodal interfaces to be able to infer the current user's state or communication state. In order to enable a system to do that, the recognition and interpretation of multimodal social signals (i.e., paralinguistic and nonverbal behavior) in real-time applications is required. Since we believe that laughs are one of the most important and widely understood social nonverbal signals indicating affect and discourse quality, we focus in this work on the detection of laughter in natural multiparty discourses. The conversations are recorded in a natural environment without any specific constraint on the discourses using unobtrusive recording devices. This setup ensures natural and unbiased behavior, which is one of the main foci of this work. To compare results of methods, namely Gaussian Mixture Model (GMM) supervectors as input to a Support Vector Machine (SVM), so-called Echo State Networks (ESN), and a Hidden Markov Model (HMM) approach, are utilized in online and offline detection experiments. The SVM approach proves very accurate in the offline classification task, but is outperformed by the ESN and HMM approach in the online detection (F 1 scores: GMM SVM 0.45, ESN 0.63, HMM 0.72). Further, we were able to utilize the proposed HMM approach in a cross-corpus experiment without any retraining with respectable generalization capability (F 1 score: 0.49). The results and possible reasons for these outcomes are shown and discussed in the article. The proposed methods may be directly utilized in practical tasks such as the labeling or the online detection of laughter in conversational data and affect-aware applications.

Multi-instance multi-label learning (MIML) introduced by Zhou and Zhang is a comparatively new framework in machine learning with two special characteristics: Firstly, each instance is represented by a set of feature vectors (a bag of instances), and secondly, bags of instances may belong to many classes (a Multi-Label). Thus, an MIML classifier receives a bag of instances and produces a Multi-Label. For classifier training, the training set is also of this MIML structure. Labeling a data set is always cost-intensive, especially in an MIMIL framework. In order to reduce the labeling costs it is important to restructure the annotation process in such a way that the most informative examples are labeled in the beginning, and less or non-informative data more to the end of the annotation phase. Active learning is a possible approach to tackle this kind of problems in this work we focus on the MIMLSVM algorithm in combination with the k-Medoids clustering algorithm to transform the Multi-Instance to a Single-Instance representation. For the clustering distance measure we consider variants of the Hausdorff distance, namely Median- and Average-Based Hausdorff distance. Finally, active learning strategies derived from the single-instance scenario have been investigated in the MIML setting and evaluated on a benchmark data set.

The paper categorizes and reviews the state-of-the-art approaches to the partially supervised learning (PSL) task. Special emphasis is put on the fields of pattern recognition and clustering involving partially (or, weakly) labeled data sets. The major instances of PSL techniques are categorized into the following taxonomy: (i) active learning for training set design, where the learning algorithm has control over the training data; (ii) learning from fuzzy labels, whenever multiple and discordant human experts are involved in the (complex) data labeling process; (iii) semi-supervised learning (SSL) in pattern classification (further sorted out into: self-training, SSL with generative models, semi-supervised support vector machines; SSL with graphs); (iv) SSL in data clustering, using additional constraints to incorporate expert knowledge into the clustering process; (v) PSL in ensembles and learning by disagreement; (vi) PSL in artificial neural networks. In addition to providing the reader with the general background and categorization of the area, the paper aims at pointing out the main issues which are still open, motivating the on-going investigations in PSL research.

In this paper a novel approach to fuzzy support vector machines (SVM) in multi-class classification problems is presented. The proposed algorithm has the property to benefit from fuzzy labeled data in the training phase and can determine fuzzy memberships for input data. The algorithm can be considered as an extension of the traditional multi-class SVM for crisp labeled data, and it also extents the fuzzy SVM approach for fuzzy labeled training data in the two-class classification setting. Its behavior is demonstrated on three benchmark data sets, the achieved results motivate the inclusion of fuzzy labeled data into the training set for various tasks in pattern recognition and machine learning, such as the design of aggregation rules in multiple classifier systems, or in partially supervised learning.

User identification and tracking are definitely the basic tasks in any human computer interaction (HCI) scenario. For these tasks we propose a multi-view approach utilizing multi-camera systems and audio processing systems. Face detectors and face recognizers are based on orientation histogram and eigenface techniques, and Mel Frequency Cepstral Coefficients (MFCC) are applied for speaker identification. In order to achieve a robust user identification and localization spatio-temporal classifier fusion methods have been integrated into the overall classifier system, support vector machines (SVM) and k nearest neighbor (kNN) models are used as base classifiers. A general office environment with up to six persons was the test bed for data collection and numerical evaluation.

Collecting unlabelled data is often effortless while labelling them can be difficult. Either the amount of data is too large or samples cannot be assigned a specific class label with certainty. In semi-supervised clustering the aim is to set the cluster centres close to their label-matching samples and unlabelled samples. Kernel based clustering methods are known to improve the cluster results by clustering in feature space. In this paper we propose a semi-supervised kernel based clustering algorithm that minimizes convergently an error function with sample-to-cluster weights. These sample-to-cluster weights are set dependent on the class label, i.e. matching, not-matching or unlabelled. The algorithm is able to use many kernel based clustering methods although we suggest Kernel Fuzzy C-Means, Relational Neural Gas and Kernel K-Means. We evaluate empirically the performance of this algorithm on two real-life dataset, namely Steel Plates Faults and MiniBooNE.

In this paper, a partially supervised machine learning approach is proposed for the recognition of emotional user states in HCI from bio-physiological data. To do so, an unsupervised learning preprocessing step is integrated into the training of a classifier. This makes it feasible to utilize unlabeled data or --- as it is conducted in this study --- data that is labeled in others than the considered categories. Thus, the data is transformed into a new representation and a standard classifier approach is subsequently applied. Experimental evidences that such an approach is beneficial in this particular setting is provided using classification experiments. Finally, the results are discussed and arguments when such an partially supervised approach is promising to yield robust and increased classification performances are given.

User generated content from fora, weblogs and other social networks is a very fast growing data source in which different information extraction algorithms can provide a convenient data access. Hierarchical clustering algorithms are used to provide topics covered in this data on different levels of abstraction. During the last years, there has been some research using hierarchical fuzzy algorithms to handle comments not dealing with one topic but many different topics at once. The used variants of the well-known fuzzy c-means algorithm are nondeterministic and thus the cluster results are irreproducible. In this work, we present a deterministic algorithm that fuzzifies currently available agglomerative hierarchical crisp clustering algorithms and therefore allows arbitrary multi-assignments. It is shown how to reuse well-studied linkage metrics while the monotonic behavior is analyzed for each of them. The proposed algorithm is evaluated using collections of the RCV1 and RCV2 corpus.

In this paper, the classification power of the eigenvalues of six graph-associated matrices is investigated and evaluated on a benchmark dataset for optical character recognition. The extracted eigenvalues were utilized as feature vectors for multi-class classification using support vector machines. Each graph-associated matrix contains a certain type of geometric/spacial information, which may be important for the classification process. Classification results are presented for all six feature types, as well as for classifier combinations at decision level. For the decision level combination probabilistic output support vector machines have been applied. The eigenvalues of the weighted adjacency matrix provided the best classification rate of 89.9 %. Here, almost half of the misclassified letters are confusion pairs, such as I-L and N-Z. This classification performance can be increased by decision fusion, using the sum rule, to 92.4 %.

Affective computing aim to provide simpler and more natural interfaces for human-computer interaction applications, e.g. recognizing automatically the emotional status of the user based on facial expressions or speech is important to model user as complete as possible in order to develop human-computer interfaces that are able to respond to the user's action or behavior in an appropriate manner. In this paper we focus on audio-based emotion recognition. Data sets employed for the statistical evaluation have been collected through Wizard-of-Oz experiments. The emotional labels have been are defined through the experimental set up therefore given on a relatively coarse temporal scale (a few minutes) which This global labeling concept might lead to miss-labeled data at smaller time scales, for instance for window sizes uses in audio analysis (less than a second). Manual labeling at these time scales is very difficult not to say impossible, and therefore our approach is to use the globally defined labels in combination with instance/sample selection methods. In such an instance selection approach the task is to select the most relevant and discriminative data of the training set by using a pre-trained classifier. Mel-Frequency Cepstral Coefficients (MFCC) features are used to extract relevant features, and probabilistic support vector machines (SVM) are applied as base classifiers in our numerical evaluation. Confidence values to the samples of the training set are assigned through the outputs of the probabilistic SVM.

Support vector machines (SVMs) often contain a large number of support vectors which reduce the run-time speeds of decision functions. In addition, this might cause an over fitting effect where the resulting SVM adapts itself to the noise in the training set rather than the true underlying data distribution and will probably fail to correctly classify unseen examples. To obtain more fast and accurate SVMs, many methods have been proposed to prune SVs in trained SVMs. In this paper, we propose a multi-objective genetic algorithm to reduce the complexity of support vector machines as well as to improve generalization accuracy by the reduction of over fitting. Experiments on four benchmark datasets show that the proposed evolutionary approach can effectively reduce the number of support vectors included in the decision functions of SVMs without sacrificing their classification accuracy.

Classifier selection aims to reduce the size of an ensemble of classifiers in order to improve its efficiency and classification accuracy. Recently an information-theoretic view was presented for feature selection. It derives a space of possible selection criteria and show that several feature selection criteria in the literature are points within this continuous space. The contribution of this paper is to export this information-theoretic view to solve an open issue in ensemble learning which is classifier selection. We investigated a couple of information-theoretic selection criteria that are used to rank classifiers.

The Gaussian mixture model (GMM) super vector approach is a well known technique in the domain of speech processing, e.g. speaker verification and audio segmentation. In this paper we apply this approach to video data in order to recognize human facial expressions. Three different image feature types (optical ???ow histograms, orientation histograms and principal components) from four pre-selected regions of the human’s face image were extracted and GMM super-vectors of the feature channels per sequence were constructed. Support vector machines (SVM) were trained using these super vectors for every channel separately and its results were combined using classifier fusion techniques. Thus, the performance of the classifier could be improved compared to the best individual classifier.

Emotion recognition is a relevant task in human-computer interaction. Several pattern recognition and machine learning techniques have been applied so far in order to assign input audio and/or video sequences to specific emotional classes. This paper introduces a novel approach to the problem, suitable also to more generic sequence recognition tasks. The approach relies on the combination of the recurrent reservoir of an echo state network with a connectionist density estimation module. The reservoir realizes an encoding of the input sequences into a fixed-dimensionality pattern of neuron activations. The density estimator, consisting of a constrained radial basis functions network, evaluates the likelihood of the echo state given the input. Unsupervised training is accomplished within a maximum-likelihood framework. The architecture can then be used for estimating class-conditional probabilities in order to carry out emotion classification within a Bayesian setup. Preliminary experiments in emotion recognition from speech signals from the WaSeP© dataset show that the proposed approach is effective, and it may outperform state-of-the-art classifiers.

Supervised learning requires a large amount of labeled data but the data labeling process can be expensive and time consuming, as it requires the efforts of human experts. Semi-supervised learning methods that can reduce the amount of required labeled data through exploiting the available unlabeled data to improve the classification accuracy. Here, we propose a learning framework to exploit the unlabeled data by decomposing multi-class problems into a set of binary problems and apply Co-Training to each binary problem. A probabilistic version of Tri-Class Support Vector Machine is proposed (SVM) that can discriminate between ignorance and uncertainty and an updated version of Sequential Minimal Optimization (SMO) algorithm is used for fast learning of Tri-Class SVMs. The proposed framework is applied to facial expressions recognition task. The results show that Co-Training can exploit effectively the independent views and the unlabeled data to improve the recognition accuracy of facial expressions.

Semi-supervised learning reduces the cost of labeling the training data of a supervised learning algorithm through using unlabeled data together with labeled data to improve the performance. Co-Training is a popular semi-supervised learning algorithm, that requires multiple redundant and independent sets of features (views). In many real-world application domains, this requirement can not be satisfied. In this paper, a single-view variant of Co-Training, CoBC (Co-Training by Committee), is proposed, which requires an ensemble of diverse classifiers instead of the redundant and independent views. Then we introduce two new learning algorithms, QBC-then-CoBC and QBC-with-CoBC, which combines the merits of committee-based semi-supervised learning and committee-based active learning. An empirical study on handwritten digit recognition is conducted where the random subspace method (RSM) is used to create ensembles of diverse C4.5 decision trees. Experiments show that these two combinations outperform the other non committee-based ones.

In this paper classifier fusion approaches are investigated through numerical evaluation. For this purpose a multi classifier architecture for the recognition of human facial expressions in image sequences has been constructed on characteristic facial regions and three different feature types (principal components, orientation histograms of static images and temporal features based on optical flow). Classifier fusion is applied to the individual channels established by feature principle and facial region, which are addressed to by individual classifiers. The available combinations of classifier outputs are examined and it is investigated how combining classifiers can lead to more appropriate results. The stability of fusion regarding varying classifier combinations is studied and the fused classifier output is compared to the human view on the data.

Multiclass pattern recognition problems (K > 2) can be decomposed by a tree-structured approach. It constructs an ensemble of K -1 individually trained binary classifiers whose predictions are combined to classify unseen instances. A key factor for an effective ensemble is how to combine its member outputs to give the final decision. Although there are various methods to build the tree structure and to solve the underlying binary problems, there is not much work to develop new combination methods that can best combine these intermediate results. We present here a trainable fusion method that integrates statistical information about the individual outputs (clustered decision templates) into a Radial Basis Function (RBF) network. We compare our model with the decision templates combiner and the existing nontrainable tree ensemble fusion methods: classical decision tree-like approach, product of the unique path and Dempster-Shafer evidence theory based method.

Semi-supervised learning is a paradigm that exploits the unlabeled data in addition to the labeled data to improve the generalization error of a supervised learning algorithm. Although in real-world applications regression is as important as classification, most of the research in semi-supervised learning concentrates on classification. In particular, although Co-Training is a popular semi-supervised learning algorithm, there is not much work to develop new Co-Training style algorithms for semi-supervised regression. In this paper, a semi-supervised regression framework, denoted by CoBCReg is proposed, in which an ensemble of diverse regressors is used for semi-supervised learning that requires neither redundant independent views nor different base learning algorithms. Experimental results show that CoBCReg can effectively exploit unlabeled data to improve the regression estimates.

For many data mining applications, it is necessary to develop algorithms that use unlabeled data to improve the accuracy of the supervised learning. Co-Training is a popular semi-supervised learning algorithm. It assumes that each example is represented by two or more redundantly sufficient sets of features (views) and these views are independent given the class. However, these assumptions are not satisfied in many real-world application domains. Therefore, we present a framework called co-training by committee (CoBC), in which a set of diverse classifiers are used to learn each other. The framework is a simple, general single-view semi-supervised learner that can use any ensemble learner to build diverse committees. Experimental studies on CoBC using bagging, AdaBoost and the random subspace method (RSM) as ensemble learners demonstrate that error diversity among classifiers leads to an effective co-training that requires neither redundant and independent views nor different learning algorithms.

Supervised learning requires a large amount of labeled data but the data labeling process can be expensive and time consuming. Co-training is a semi-supervised learning method that reduces the amount of required labeled data through exploiting the available unlabeled data in supervised learning to boost the accuracy. It assumes that the patterns are described by multiple independent feature sets and each feature set is sufficient for classification. On the other hand, most of the real-world pattern recognition tasks involve a large number of categories which may make the task difficult. The tree-structured approach is a multi-class decomposition strategy where a complex multi-class problem is decomposed into tree structured binary sub-problems. In this paper, we propose a framework that combines the tree-structured approach with Co-training. We show that our framework is especially useful for classification tasks involving a large number of classes and a small amount of labeled data where the tree-structured approach does not perform well by itself and when combined with Co-training, the unlabeled data boosts its accuracy.

We present a novel approach for Fuzzy-Input Fuzzy-Output classification. One-Against-All Support Vector Machines are adapted to deal with the fuzzy memberships encoded in fuzzy labels, and to also give fuzzy classification answers. The mathematical background for the modifications is given. In a benchmark application, the recognition of emotions in human speech, the accuracy of our F2-SVM approach is clearly superior to that of fuzzy MLP and fuzzy K-NN architectures.

In this paper we present a method to recognize human faces based on histograms of local orientation. Orientation histograms were used as input feature vectors for a k-nearest neigbour classifier. We present a method to calculate orientation histograms of n×n subimages partitioning the 2D-camera image with the segmented face. Numerical experiments have been made utilizing the Olivetti Research Laboratory (ORL) database containing 400 images of 40 subjects. Remarkable recognition rates of 98% to 99% were achieved with this extremely simple approach.

Supervised learning models most commonly use crisp labels for classifier training. Crisp labels fail to capture the data characteristics when overlapping classes exist. In this work we attempt to compare between learning using soft and hard labels to train K-nearest neighbor classifiers. We propose a new technique to generate soft labels based on fuzzy-clustering of the data and fuzzy relabelling of cluster prototypes. Experiments were conducted on five data sets to compare between classifiers that learn using different types of soft labels and classifiers that learn with crisp labels. Results reveal that learning with soft labels is more robust against label errors opposed to learning with crisp labels. The proposed technique to find soft labels from the data, was also found to lead to a more robust training in most data sets investigated.

We explore the use of constraints with divisive hierarchical clustering. We mention some considerations on the effects of the inclusion of constraints into the hierarchical clustering process. Furthermore, we introduce an implementation of a semi-supervised divisive hierarchical clustering algorithm and show the influence of including constraints into the divisive hierarchical clustering process. In this task our main interest lies in building stable dendrograms when clustering with different subsets of data.

We consider the problem of multiclass decomposition schemes for Support Vector Machines with Linear, Polynomial and RBF kernels. Our aim is to compare and discuss popular multiclass decomposing approaches such as the One versus the Rest, One versus One, Decision Directed Acyclic Graphs, Tree Structured, Error Correcting Output Codes. We conducted our experiments on benchmark datastes consisting of camera images of 3D objects. In our experiments we found that all the multiclass decomposing schemes for SVMs performed comparably very well with no significant statistical differences in cases of nonlinear kernels. In case of linear kernels the multiclass schemes OvR, OvO and DDAG outperform Tree Structured and ECOC.

In this work a new approach to the support vector machine (SVM) method is taken. Not in developing a new algorithm, but rather in analyzing the result of the performed classification tasks. The SVM approach provides efficient and powerful classification algorithms. SM-Classifiers have a few meta parameters to be tuned, are easy to implement, and are trained through optimization of a quadratic cost function, which ensures the uniqueness of the SVM solution. The SVM solution is given through a linear combination of the training samples which are selected by the SVM optimization procedure. This subset of borderline samples close to the decision boundary can be separated into the samples which are misclassified and those samples that are just classified correctly. The potential drawback of the SVMs being restricted to samples from the dataset is at the same time an advantage in medical applications. Here, we applied this approach to a highly selected group of 44 patients with inducible ventricular tachycardia and a group of 51 healthy subjects.

Time series classification based on decision templates is the topic of this paper. The decision templates are built over several local feature vectors which are extracted from local time windows of the time series. To learn characteristic classifier outputs a set of decision templates is determined for each class. In the classification phase class memberships based on the decision templates and the classifier outputs are calculated. These class memberships are combined to compute the final classification result. We present algorithms to calculate multiple decision templates per class, and demonstrate their behaviour on two data sets, a synthetic data set, and a data set derived from a real world pattern recognition problem, the classification of bioacoustic time series. The fuzzy-k-nearest neighbour classifier is utilized as base classifier for both data sets.

In this paper a strategy for neural network training is described. The similarities of supervised competitive and gradient descent learning are used to construct an algorithm which initializes radial basis function networks.

A strategy for neural network training is described. It combines supervised competitive and gradient descent learning. This algorithm is then applied to classify high-resolution electrograms. The combined approach gives an increase in classification accuracy of about 10%. Nevertheless the results show that more elaborate feature extraction methods have to be considered.

Support vector machines (SVM) are learning algorithms derived from statistical learning theory. The SVM approach was originally developed for binary classification problems. In this paper SVM architectures for multi-class classification problems are discussed, in particular we consider binary trees of SVMs to solve the multi-class problem. Numerical results for different classifiers on a benchmark data set of handwritten digits are presented.

In this chapter we discuss algorithms for clustering and visualization of large and multivariate data. We describe an algorithm for exploratory data analysis which combines adaptive c-means clustering and multi-dimensional scaling (ACMDS). ACMDS is an algorithm for the online visualization of clustering processes and may be considered as an alternative approach to Kohonen's self organizing feature map (SOM). Whereas SOM is a heuristic neural network algorithm, ACMDS is derived from multivariate statistical algorithms. The implications of ACMMDS are illustrated through five different data sets.

Support vector machines (SVM) are learning algorithms derived from statistical learning theory. The SVMapproac h was originally developed for binary classification problems. In this paper SVMarc hitectures for multi-class classification problems are discussed, in particular we consider binary trees of SVMs to solve the multi-class problem. Numerical results for different classifiers on a benchmark data set of handwritten digits are presented.

In this chapter we present a 3-D visual object recognition system for an autonomous mobile robot. This object recognition system performs the following three tasks: Object localization in the camera images, feature extraction, and classification of the extracted feature vectors with hierarchical radial basis function (RBF) networks.

In this paper we discuss classifier architectures to categorize time series. Three different architectures for the fusion of local classifier decisions are presented and applied to classify recordings of cricket songs. Different features from local time windows are extracted automatically from the waveform of the sound patterns. These features are used to classify the whole time series. We present results for all three classifier architectures on a data set of 28 different categories.

The classification of time series based on local features is discussed in this paper. In this context we discuss the topics data fusion, decision fusion, and temporal fusion. Three different classifier architectures for these fusion tasks are proposed. Some local features are automatically derived form the time and frequency domain and categorized through a fuzzy-k-nearest-neighbour rule. Soft decisions are combined to a crisp decision of the whole time series. Numerical results for all architectures are given for a data set (songs of crickets, recorded in Thailand and Ecuador) containing 35 different categories.

Support vector machines (SVM) are learning algorithms derived from statistical learning theory. The SVM approach was originally developed for binary classification problems. In this paper SVM architectures for multi-class classification problems are discussed, in particular we consider binary trees of SVMs to solve the multi-class pattern recognition problem. Numerical results for different classifiers on a benchmark data set handwritten digits are presented.

We describe an algorithm for exploratory data analysis which combines adaptive c-means clustering and multi-dimensional scaling (ACMDS). ACMDS is an algorithm for the online visualization of clustering processes and may be considered as an alternative approach to Kohonen’s self organizing feature map (SOM). Whereas SOM is a heuristic neural network algorithm, ACMDS is derived from multivariate statistical algorithms. The implications of ACMDS are illustrated through five different data sets.

We describe a algorithm for exploratory data analysis which combines the adaptive c-means clustering and the multi-dimensional scaling procedure (ACMMDS). ACMMDS is an algorithm for the online visualization of clustering processes and may be considered as a alternative approach to Kohonen’s self organizing feature (SOM). Whereas SOM is a heuristic neural network algorithm, ACMMDS is derived from multivariate statistic algorithms. The possible implications of ACMMDS are illustrated through two different data sets.

A hierarchical neural network architecture for classification tasks is proposed. This algorithm has been evaluated in an OCR problem. The basic components of the network architecture are LVQ and RBF networks.

Combining a hardware approach with a multiple classifier method can deeply improve system performance, since the multiple classifier system can successfully enhance the classification accuracy with respect to a single classifier, and a hardware implementation would lead to systems able to classify samples with high throughput and with a short latency. To the best of our knowledge, no paper in the literature takes into account the multiple classifier scheme as additional design parameter, mainly because of lack of efficient hardware combiner architecture. In order to fill this gap, in this paper we will first propose a novel approach for an efficient hardware implementation of the majority voting combining rule. Then, we will illustrate a design methodology to suitably embed in a digital device a multiple classifier system having Decision Trees as base classifiers and a majority voting rule as combiner. Bagging, Boosting and Random Forests will be taken into account. We will prove the effectiveness of the proposed approach on two real case studies related to Big Data issues.