Search

© 1997-2012 IEEE. With a global search mechanism, particle swarm optimization (PSO) has shown promise in feature selection (FS). However, most of the current PSO-based FS methods use a fix-length representation, which is inflexible and limits the performance of PSO for FS. When applying these methods to high-dimensional data, it not only consumes a significant amount of memory but also requires a high computational cost. Overcoming this limitation enables PSO to work on data with much higher dimensionality which has become more and more popular with the advance of data collection technologies. In this paper, we propose the first variable-length PSO representation for FS, enabling particles to have different and shorter lengths, which defines smaller search space and therefore, improves the performance of PSO. By rearranging features in a descending order of their relevance, we facilitate particles with shorter lengths to achieve better classification performance. Furthermore, using the proposed length changing mechanism, PSO can jump out of local optima, further narrow the search space and focus its search on smaller and more fruitful area. These strategies enable PSO to reach better solutions in a shorter time. Results on ten high-dimensional datasets with varying difficulties show that the proposed variable-length PSO can achieve much smaller feature subsets with significantly higher classification performance in much shorter time than the fixed-length PSO methods. The proposed method also outperformed the compared non-PSO FS methods in most cases. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

© 2019 The Royal Society of New Zealand. Artificial intelligence (AI) emphasises the creation of intelligent machines/systems that function like humans. AI has been applied to many real-world applications. Machine learning is a branch of AI based on the idea that systems can learn from data, identify hidden patterns, and make decisions with little/minimal human intervention. Evolutionary computation is an umbrella of population-based intelligent/learning algorithms inspired by nature, where New Zealand has a good international reputation. This paper provides a review on evolutionary machine learning, i.e. evolutionary computation techniques for major machine learning tasks such as classification, regression and clustering, and emerging topics including combinatorial optimisation, computer vision, deep learning, transfer learning, and ensemble learning. The paper also provides a brief review of evolutionary learning applications, such as supply chain and manufacturing for milk/dairy, wine and seafood industries, which are important to New Zealand. Finally, the paper presents current issues with future perspectives in evolutionary machine learning.This is an Accepted Manuscript of an article published by Taylor & Francis in 'Journal of the Royal Society of New Zealand' on 2019-04-03, available online: https://www.tandfonline.com/10.1080/03036758.2019.1609052.

Recent research has shown that Particle Swarm Optimisation is a promising approach to feature selection. However, applying it on high-dimensional data with thousands to tens of thousands of features is still challenging because of the large search space. While filter approaches are time efficient and scalable for high-dimensional data, they usually obtain lower classification accuracy than wrapper approaches. On the other hand, wrapper methods require a longer running time than filter methods due to the learning algorithm involved in fitness evaluation. This paper proposes a new strategy of combining filter and wrapper approaches in a single evolutionary process in order to achieve smaller feature subsets with better classification performance in a shorter time. A new local search heuristic using symmetric uncertainty is proposed to refine the solutions found by PSO and a new hybrid fitness function is used to better evaluate candidate solutions. The proposed method is examined and compared with three recent PSO based methods on eight high-dimensional problems of varying difficulty. The results show that the new hybrid PSO is more effective and efficient than the other methods.© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Fall events in elderly populations often result in serious injury and may lead to long-term disability and/or death. While many fall detection systems have been developed using wearable sensors to distinguish falls from other daily activities, detection sensitivity and specificity decreases when exposed to more rigorous activities such as running and jumping. This research uses time-frequency analysis of accelerometer-only activity data to develop a strategy for improving fall detection accuracy. In this study, a wireless sensor system (WSS) consisting of a three-axis accelerometer, microprocessor and wireless communications module is used to collect daily activities performed following a script in the laboratory setting. Experiments were conducted on 36 healthy human subjects performing four types of falls (i.e., forward, backward, and left/right sideway falls) as well as normal movements such as standing, walking, stand-to-sit, sit-to-stand, stepping, running and jumping. In total, 1227 different activities were collected and analyzed. The developed algorithm computes the magnitude of three-axis accelerometer data to detect if a critical fall threshold is passed, then analyzes the power spectral density within a critical fall duration window (500 ms) to differentiate fall events from other rigorous activities. Fall events were observed with high energy in the 2&ndash, 3.5 Hz range and distinct from other rigorous activities such as running (3.5&ndash, 5.5 Hz) and jumping (1&ndash, 2 Hz). Preliminary results indicate the power spectral density (PSD)-based algorithm can detect falls with high sensitivity (98.4%) and specificity (98.6%) using lab-based daily activity data. The proposed algorithm has the benefit of improved accuracy over existing time-domain only strategies and multisensor strategies.

This paper aims to produce classifiers that are not only accurate but also interpretable to decision makers. The classifiers are represented in the form of risk scores, i.e. simple linear classifiers where coefficient vectors are sparse and bounded integer vectors which are then optimised by a novel and scalable discrete particle swarm optimisation algorithm. In contrast to past studies which usually use particle swarm optimisation as a pre-processing step, the proposed algorithm incorporates particle swarm optimisation into the classification process. A penalty-based fitness function and a local search heuristic based on symmetric uncertainty are developed to efficiently identify classifiers with high classification performance and a preferred model size or complexity. Experiments with 10 benchmark datasets show that the proposed swarm-based algorithm is a strong candidate to develop effective linear classifiers. Comparisons with other interpretable machine learning algorithms that produce rule-based and tree-based classifiers also demonstrate the competitiveness of the proposed algorithm. Further analyses also confirm the interpretability of the produced classifiers. Finally, the proposed algorithm shows excellent speed-up via parallelisation, which gives it a great advantage when coping with large scale problems.

Growing big data has posed a great challenge for machine learning algorithms. To cope with big data, the algorithm has to be both efficient and accurate. Although evolutionary computation has been successfully applied to many complex machine learning tasks, its ability to handle big data is limited. In this paper, we proposed a dynamic self-organising swarm algorithm to learn an effective set of prototypes for big high-dimensional datasets in an unsupervised manner. The novelties of this new algorithm are the energy-based fitness function, the adaptive topological neighbourhood, the growing/shrinking capability, and the efficient learning scheme. Experiments with well-known datasets show that the proposed algorithm can maintain a very compact set of prototypes and achieve competitive predictive performance as compared to other algorithms in the literature. The analyses also show that prototypes generated by the proposed algorithms have a stronger separatability compared to those from other prototype generation algorithms.

Feature space is an important factor influencing the performance of any machine learning algorithm including classification methods. Feature selection aims to remove irrelevant and redundant features that may negatively affect the learning process especially on high-dimensional data, which usually suffers from the curse of dimensionality. Feature ranking is one of the most scalable feature selection approaches to high-dimensional problems, but most of them fail to automatically determine the number of selected features as well as detect redundancy between features. Particle swarm optimisation (PSO) is a population-based algorithm which has shown to be effective in addressing these limitations. However, its performance on high-dimensional data is still limited due to the large search space and high computation cost. This study proposes the first adaptive multi-swarm optimisation (AMSO) method for feature selection that can automatically select a feature subset of high-dimensional data more effectively and efficiently than the compared methods. The subswarms are automatically and dynamically changed based on their performance during the evolutionary process. Experiments on ten high-dimensional datasets of varying difficulties have shown that AMSO is more effective and more efficient than the compared PSO-based and traditional feature selection methods in most cases.

BackgroundMindfulness training shows promise for improving smoking cessation and lapse recovery, and between-session mobile health messages could enhance treatment engagement and effectiveness. Personalized, in-the-moment text messaging support could be particularly useful for low-income smokers with fewer smoking cessation resources. ObjectiveThis pilot study examined the feasibility of a text messaging program (iQuit Mindfully) as an adjunct to in-person Mindfulness-Based Addiction Treatment (MBAT) for smoking cessation. MethodsA total of 71 participants were randomly assigned to MBAT (n=33) or iQuit Mindfully (n=38; MBAT + between-session text messages); of these, 70% (50/71) were African American, and 61% (43/71) had an annual household income of US $30,000 or less. All participants received 8 weekly therapist-led group counseling sessions, nicotine patches, and self-help materials. Outcomes were feasibility (attrition, engagement, and participants’ ratings), participants’ feedback regarding the text messaging intervention, and smoking cessation (assessed in person). ResultsStrong retention was achieved (76% [54/71] at the end of treatment, and 89% [63/71] at 1-month follow-up). In the iQuit Mindfully group, engagement was high (88% [29/33] indicated reading all or most texts, and 89% [34/38] engaged in interactive texting), and participants provided positive ratings (on a 1-10 scale, average rating for recommending the program to others was 8.4 [SD 2.5]). Participants indicated benefiting from the texts (eg, appreciating encouraging reminders, coping strategies, and social support) and suggested improvements (eg, more personalization). Overall, biochemically confirmed smoking cessation rates were 22% (12/55) at the end of treatment and 19% (12/62) at 1-month follow-up, with no differences between conditions. Living below the poverty level predicted worse cessation outcomes at 1-month follow-up among participants receiving in-person only treatment (P=.03) but not among those receiving iQuit Mindfully. ConclusionsText messaging appears to be a feasible and acceptable modality for supporting mindfulness-based smoking cessation treatment. The availability of 24/7 text messaging might be particularly helpful for low-income smokers who have access to fewer cessation resources and experience significant day-to-day barriers to quitting. Trial RegistrationClinicalTrials.gov NCT03029819; https://clinicaltrials.gov/ct2/show/NCT03029819

Feature selection is an essential step in classification tasks with a large number of features, such as in gene expression data. Recent research has shown that particle swarm optimisation PSO is a promising approach to feature selection. However, it also has potential limitation to get stuck into local optima, especially for gene selection problems with a huge search space. Therefore, we developed a PSO algorithm PSO-LSRG with a fast “local search” combined with a gbest resetting mechanism as a way to improve the performance of PSO for feature selection. Furthermore, since many existing PSO-based feature selection approaches on the gene expression data have feature selection bias, i.e. no unseen test data is used, 2 sets of experiments on 10 gene expression datasets were designed: with and without feature selection bias. As compared to standard PSO, PSO with gbest resetting only, and PSO with local search only, PSO-LSRG obtained a substantial dimensionality reduction and a significant improvement on the classification performance in both sets of experiments. PSO-LSRG outperforms the other three algorithms when feature selection bias exists. When there is no feature selection bias, PSO-LSRG selects the smallest number of features in all cases, but the classification performance is slightly worse in a few cases, which may be caused by the overfitting problem. This shows that feature selection bias should be avoided when designing a feature selection algorithm to ensure its generalisation ability on unseen data.

Classification on high-dimensional data with thousands to tens of thousands of dimensions is a challenging task due to the high dimensionality and the quality of the feature set. The problem can be addressed by using feature selection to choose only informative features or feature construction to create new high-level features. Genetic programming (GP) using a tree-based representation can be used for both feature construction and implicit feature selection. This work presents a comprehensive study to investigate the use of GP for feature construction and selection on high-dimensional classification problems. Different combinations of the constructed and/or selected features are tested and compared on seven high-dimensional gene expression problems, and different classification algorithms are used to evaluate their performance. The results show that the constructed and/or selected feature sets can significantly reduce the dimensionality and maintain or even increase the classification accuracy in most cases. The cases with overfitting occurred are analysed via the distribution of features. Further analysis is also performed to show why the constructed feature can achieve promising classification performance. This is a post-peer-review, pre-copyedit version of an article published in 'Memetic Computing'. The final authenticated version is available online at: https://doi.org/10.1007/s12293-015-0173-y. The following terms of use apply: https://www.springer.com/gp/open-access/publication-policies/aam-terms-of-use.

Feature construction is a pre-processing technique to create new features with better discriminating ability from the original features. Genetic programming (GP) has been shown to be a prominent technique for this task. However, applying GP to high-dimensional data is still challenging due to the large search space. Feature clustering groups similar features into clusters, which can be used for dimensionality reduction by choosing representative features from each cluster to form the feature subset. Feature clustering has been shown promising in feature selection; but has not been investigated in feature construction for classification. This paper presents the first work of utilising feature clustering in this area. We propose a cluster-based GP feature construction method called CGPFC which uses feature clustering to improve the performance of GP for feature construction on high-dimensional data. Results on eight high-dimensional datasets with varying difficulties show that the CGPFC constructed features perform better than the original full feature set and features constructed by the standard GP constructor based on the whole feature set.

The notion of cyberspace became impossible to separate from the notions of cyber threat and cyberattack. Since cyberattacks are getting easier to run, they are also becoming more serious threats from the economic damage perspective. Consequently, we are evident of a continuous adversarial relationship between the attackers trying to mount as powerful as possible attacks and defenders trying to stop the attackers in their goals. To defend against such attacks, defenders have at their disposal a plethora of techniques but they are often falling behind the attackers due to the fact that they need to protect the whole system while the attacker needs to find only a single weakness to exploit. In this paper, we consider one type of a cyberattack – network intrusion – and investigate how to use feature construction via genetic programming in order to improve the intrusion detection accuracy. The obtained results show that feature construction offers improvements in a number of tested scenarios and therefore should be considered as an important step in defense efforts. Such improvements are especially apparent in scenario with the highly unbalanced data, which also represents the most interesting case from the defensive perspective.

Genetic Programming (GP) has shown promise in feature construction where high-level features are formed by combining original features using predefined functions or operators. Multiple feature construction methods have been proposed for high-dimensional data with thousands of features. Results of these methods show that several constructed features can maintain or even improve the discriminating ability of the original feature set. However, some particular features may have better ability than other features to distinguish instances of one class from other classes. Therefore, it may be more difficult to construct a better discriminating feature when combing features that are relevant to different classes. In this study, we propose a new GP-based feature construction method called CDFC that constructs multiple features, each of which focuses on distinguishing one class from other classes. We propose a new representation for class-dependent feature construction and a new fitness function to better evaluate the constructed feature set. Results on eight datasets with varying difficulties showed that the features constructed by CDFC can improve the discriminating ability of thousands of original features in most cases. Results also showed that CFDC is more effective and efficient than the hybrid MGPFC method which was shown to have better performance than standard GP to feature construction.

Feature construction and feature selection are common pre-processing techniques to obtain smaller but better discriminating feature sets than the original ones. These two techniques are essential in high-dimensional data with thousands or tens of thousands of features where there may exist many irrelevant and redundant features. Genetic programming (GP) is a powerful technique that has shown promising results in feature construction and feature selection. However, constructing multiple features for high-dimensional data is still challenging due to its large search space. In this paper, we propose a GP-based method that simultaneously performs multiple feature construction and feature selection to automatically transform high-dimensional datasets into much smaller ones. Experiment results on six datasets show that the size of the generated feature set is less than 4% of the original feature set size and it significantly improves the performance of K-Nearest Neighbour, Naive Bayes and Decision Tree algorithms on 15 out of 18 comparisons. Compared with the single feature construction method using GP, the proposed method has better performance on half cases and similar on the other half. Comparisons between the constructed features, the selected features and the combination of both constructed and selected features by the propose method reveal different preferences of the three learning algorithms on these feature sets.

Feature selection and discretisation have shown their effectiveness for data preprocessing especially for high-dimensional data with many irrelevant features. While feature selection selects only relevant features, feature discretisation finds a discrete representation of data that contains enough information but ignoring some minor fluctuation. These techniques are usually applied in two stages, discretisation and then selection since many feature selection methods work only on discrete features. Most commonly used discretisation methods are univariate in which each feature is discretised independently; therefore, the feature selection stage may not work efficiently since information showing feature interaction is not considered in the discretisation process. In this study, we propose a new method called PSO-DFS using bare-bone particle swarm optimisation (BBPSO) for discretisation and feature selection in a single stage. The results on ten high-dimensional datasets show that PSO-DFS obtains a substantial dimensionality reduction for all datasets. The classification performance is significantly improved or at least maintained on nine out of ten datasets by using the transformed “small” data obtained from PSO-DFS. Compared to applying the two-stage approach which uses PSO for feature selection on the discretised data, PSO-DFS achieves better performance on six datasets, and similar performance on three datasets with a much smaller number of features selected.