Search

Classifying objects and patterns to a certain category is crucial for both humans and machines, so that learnt knowledge may be applied across similar problem instances. Although autonomous learning of patterns by machines has advanced recently, it still requires humans to set up the problem at an appropriate level for the learning technique. If the problem is too complex the system does not learn; conversely, if the problem is too simple the system does not reach its full potential to be able to classify environmental examples. In this work, an automated evolving pattern generator and pattern recognizer has been created for pattern classification problems that can be manipulated autonomously using Learning Classifier Systems (LCSs) at different levels of difficulty. Experiments confirm that both of the agents (e.g. the pattern generation and the pattern classification agent) can be evolved autonomously and co-operatively. The novel contributions in this work enable the effect of domain features on classification performance to become human readable, i.e. possibly determine what features make it difficult for the classification algorithm to learn. This work provides a foundation for a co-evolutionary approach to problem domain creation and the associated learning, such that the agents will trigger evolution when necessary.

Human beings have the ability to apply the domain knowledge learned from a smaller problem to more complex problems of the same or a related domain, but currently evolutionary computation techniques lack this ability. Hence these techniques relearn from the start when the problem scales, increasing the time required and potentially limiting capability. In order to autonomously scale in a problem domain reusable building blocks of knowledge must be extracted. A richer encoding scheme than ternary alphabet has been constructed to identify building blocks. The novel work presented here is to extract useful building blocks from smaller problems and reuse them to learn complex problems in the domain. The proposed system has been compared with ternary alphabet based XCS for three different problem domains, i.e. multiplexer, carry, and even-parity problems. Autonomous scaling is shown possible for the first time in learning classifier systems. It improves effectiveness and reduces the number of training instances required in large problems, but requires more time due to more involved methods.

Extracting features from images is an important task in order to identify (classify) the patterns contained. The Evolutionary Computation and Reinforcement Learning technique of Learning Classifier Systems (LCSs) has been successfully applied to classification tasks, but rarely to image pattern classification due to the large search space associated with pixel data. Recently, a Feature Pattern Classification System (FPCS), utilising Haar-like features has been introduced with promising results in the image recognition domain. This system used a confusion-matrix to direct learning to hard to classify classes, but due to its reinforcement learning nature was required to estimate the ground truth. The novel work presented here adopts a supervised learning (UCS-based) approach into the FPCS framework. This work is compared with the original XCS-based system, updated to include the known ground-truth of the confusion matrix to aid comparison, albeit no longer reinforcement learning. Results on the 10 class MNIST numerical digits recognition task show that the XCS-based FPCS produces better classification due to its complete mapping guiding learning. However, results on the 26 class NIST character recognition task show that the UCS-based scales better as it does not require the complete mapping. The human readable rules produced by each system, coupled with the competitive classification performance compared with similar techniques, supports future work on both the XCS and UCS-based FPCS.

Feature selection (FS) is an important data preprocessing technique, which has two goals of minimising the classification error and minimising the number of features selected. Based on particle swarm optimisation (PSO), this paper proposes two multi-objective algorithms for selecting the Pareto front of non-dominated solutions (feature subsets) for classification. The first algorithm introduces the idea of non-dominated sorting based multi-objective genetic algorithm II into PSO for FS. In the second algorithm, multi-objective PSO uses the ideas of crowding, mutation and dominance to search for the Pareto front solutions. The two algorithms are compared with two single objective FS methods and a conventional FS method on nine datasets. Experimental results show that both proposed algorithms can automatically evolve a smaller number of features and achieve better classification performance than using all features and feature subsets obtained from the two single objective methods and the conventional method. Both the continuous and the binary versions of PSO are investigated in the two proposed algorithms and the results show that continuous version generally achieves better performance than the binary version. The second new algorithm outperforms the first algorithm in both continuous and binary versions.

Decision tree induction is one of the most employed methods to extract knowledge from data, since the representation of knowledge is very intuitive and easily understandable by humans. The most successful strategy for inducing decision trees, the greedy top-down approach, has been continuously improved by researchers over the years. This work, following recent breakthroughs in the automatic design of machine learning algorithms, proposes a hyper-heuristic evolutionary algorithm for automatically generating decision-tree induction algorithms, named HEAD-DT. We perform extensive experiments in 20 public data sets to assess the performance of HEAD-DT, and we compare it to traditional decision-tree algorithms such as C4.5 and CART. Results show that HEAD-DT can generate algorithms that significantly outperform C4.5 and CART regarding predictive accuracy and F-Measure.

This paper proposes a constrained-syntax genetic programming (GP) algorithm-for discovering classification rules in medical data sets. The proposed GP contains several syntactic constraints to be enforced by the system using a disjunctive normal form representation, so that individuals represent valid rule sets that are easy to interpret. The GP is compared with C4.5 in a real-world medical data set. This data set represents a difficult classification problem, and a new preprocessing method was devised for mining the data.

Artificial Immune Systems are computational algorithms which take their inspiration from the way in which natural immune systems learn to respond to attacks on an organism. This paper discusses how such a system can be used as an alternative to genetic algorithms as a way of exploring program-space in a system similar to genetic programming. Some experimental results are given for a symbolic regression problem. The paper ends with a discussion of future directions for the use of artificial immune systems in program induction.

For a given data set, its set of attributes defines its data space representation. The quality of a data space representation is one of the most important factors influencing the performance of a data mining algorithm. The attributes defining the data space can be inadequate, making it difficult to discover high-quality knowledge. In order to solve this problem, this paper proposes a Genetic Programming algorithm developed for attribute construction. This algorithm constructs new attributes out of the original attributes of the data set, performing an important preprocessing step for the subsequent application of a data mining algorithm.