Search

In many cases of serious crime, images of a hand can be the only evidence available for the forensic identification of the offender. As well as placing them at the scene, such images and video evidence offer proof of the offender committing the crime. The knuckle creases of the human hand have emerged as an effective biometric trait and been used to identify the perpetrators of child abuse in forensic investigations. However, manual utilization of knuckle creases for identification is highly time consuming and can be subjective, requiring the expertise of experienced forensic anthropologists whose availability is very limited. Hence, there arises a need for an automated approach for localization and comparison of knuckle patterns. In this paper, we present a fully automatic end-to-end approach which localizes the minor, major and base knuckles in images of the hand, and effectively uses them for identification achieving state-of-the-art results. This work improves on existing approaches and allows us to strengthen cases further by objectively combining multiple knuckles and knuckle types to obtain a holistic matching result for comparing two hands. This yields a stronger and more robust multi-unit biometric and facilitates the large-scale examination of the potential of knuckle-based identification. Evaluated on two large landmark datasets, the proposed framework achieves equal error rates (EER) of 1.0-1.9%, rank-1 accuracies of 99.3-100% and decidability indices of 5.04-5.83. We make the full results available via a novel online GUI to raise awareness with the general public and forensic investigators about the identifiability of various knuckle regions. These strong results demonstrate the value of our holistic approach to hand identification from knuckle patterns and their utility in forensic investigations.

MACHINE and AI justifiably attract the attention and interest not only of the wider scientific community and industry, but also society and policy makers. However, even the most powerful (in terms of accuracy) algorithms such as deep learning (DL) can give a wrong output, which may be fatal. Due to the opaque and cumbersome model structure used by DL, some authors started to talk about a dystopian “black box” society. Despite the success in this area, the way computers learn is still principally different from the way people acquire new knowledge, recognise objects and make decisions. People do not need a huge amount of annotated data. They learn by example, using similarities to previously acquired prototypes, not by using parametric analytical models. Current ML approaches are focused primarily on accuracy and overlook explainability, the semantic meaning of the internal model representation, reasoning and its link with the problem domain. They also overlook the efforts to collect and label training data and rely on assumptions about the data distribution that are often not satisfied. The ability to detect the unseen and unexpected and start learning this new class/es in real time with no or very little supervision is critically important and is something that no currently existing classifier can offer. The challenge is to fill this gap between high level of accuracy and the semantically meaningful solutions. The most efficient algorithms that have fuelled interest towards ML and AI recently are also computationally very hungry - they require specific hardware accelerators such as GPU, huge amounts of labeled data and time. They produce parametrised models with hundreds of millions of coefficients, which are also impossible to interpret or be manipulated by a human. Once trained, such models are inflexible to new knowledge. They cannot dynamically evolve their internal structure to start recognising new classes. They are good only for what they were originally trained for. They also lack robustness, formal guarantees about their behaviour and explanatory and normative transparency. This makes problematic use of such algorithms in high stake complex problems such as aviation, health, bailing from jail, etc. where the clear rationale for a particular decision is very important and the errors are very costly. All these challenges and identified gaps require a dramatic paradigm shift and a radical new approach. In this talk the speaker will present such a new approach towards the next generation of computationally lean ML and AI algorithms that can learn in real-time using normal CPUs on computers, laptops, smartphones or even be implemented on chip that will change dramatically the way these new technologies are being applied. It is explainable-by-design. It focuses on addressing the open research challenge of developing highly efficient, accurate ML algorithms and AI models that are transparent, interpretable, explainable and fair by design. Such systems are able to self-learn lifelong, and continuously improve without the need for complete retraining, can start learning from few training data samples, explore the data space, detect and learn from unseen data patterns, collaborate with humans or other such algorithms seamlessly.

A novel self-organizing fuzzy proportional-integral-derivative (SOF-PID) control system is proposed in this paper. The proposed system consists of a pair of control and reference models, both of which are implemented by a first-order autonomous learning multiple model (ALMMo) neuro-fuzzy system. The SOF-PID controller self-organizes and self-updates the structures and meta-parameters of both the control and reference models during the control process "on the fly". This gives the SOF-PID control system the capability of quickly adapting to entirely new operating environments without a full re-training. Moreover, the SOF-PID control system is free from user- and problem-specific parameters, and the uniform stability of the SOF-PID control system is theoretically guaranteed. Simulations and real-world experiments with mobile robots demonstrate the effectiveness and validity of the proposed SOF-PID control system., This paper has been submitted to WCCI2020 for publication

Microscopic blood cell analysis is an important methodology for medical diagnosis, and complete blood cell counts (CBCs) are one of the routine tests operated in hospitals. Results of the CBCs include amounts of red blood cells, white blood cells and platelets in a unit blood sample. It is possible to diagnose diseases such as anemia when the numbers or shapes of red blood cells become abnormal. The percentage of white blood cells is one of the important indicators of many severe illnesses such as infection and cancer. The amounts of platelets are decreased when the patient suffers hemophilia. Doctors often use these as criteria to monitor the general health conditions and recovery stages of the patients in the hospital. However, many hospitals are relying on expensive hematology analyzers to perform these tests, and these procedures are often time consuming. There is a huge demand for an automated, fast and easily used CBCs method in order to avoid redundant procedures and minimize patients’ burden on costs of healthcare. In this research, we investigate a new CBC detection method by using deep neural networks, and discuss state of the art machine learning methods in order to meet the medical usage requirements. The approach we applied in this work is based on YOLOv3 algorithm, and our experimental results show the applied deep learning algorithms have a great potential for CBCs tests, promising for deployment of deep learning methods into microfluidic point-of-care medical devices. As a case of study, we applied our blood cell detector to the blood samples of COVID-19 patients, where blood cell clots are a typical symptom of COVID-19.

Machine learning has arisen with advanced data analytics. Many factors influence crop yield, such as soil, amount of water, climate, and genotype. Determining factors that significantly influence yield prediction and identify the most appropriate predictive methods are important in yield management. It is critical to consider and study the combination of different crop factors and their impact on the yield. The objectives of this paper are: (1) to use advanced data analytic techniques to precisely predict the soybean crop yields, (2) to identify the most influential features that impact soybean predictions, (3) to illustrate the ability of Fuzzy Rule-Based (FRB) sub-systems, which are self-organizing, self-learning, and data-driven, by using the recently developed Autonomous Learning Multiple-Model First-order (ALMMo-1) system, and (4) to compare the performance with other well-known methods. The ALMMo-1 system is a transparent model, which stakeholders can easily read and interpret. The model is a data-driven and composed of prototypes selected from the actual data. Many factors affect the yield, and data clouds can be formed in the feature/data space based on the data density. The data cloud is the key to the IF part of FRB sub-systems, while the THEN part (the consequences of the IF condition) illustrates the yield prediction in the form of a linear regression model, which consists of the yield features or factors. In addition, the model can determine the most influential features of the yield prediction online. The model shows an excellent prediction accuracy with a Root Mean Square Error (RMSE) of 0.0883, and Non-Dimensional Error Index (NDEI) of 0.0611, which is competitive with state-of-the-art methods.

This paper presents an actively semi-supervised multi-layer neuro-fuzzy modeling method, ASSDRB, to classify different lighting conditions for driving scenes. ASSDRB is composed of a massively parallel ensemble of AnYa type 0-order fuzzy rules. It uses a recursive learning algorithm to update its structure when new data items are provided and, therefore, is able to cope with nonstationarities. Different lighting conditions for driving situations are considered in the analysis, which is used by self-driving cars as a safety mechanism. Differently from mainstream Deep Neural Networks approaches, the ASSDRB is able to learn from unseen data. Experiments on different lighting conditions for driving scenes, demonstrated that the deep neuro-fuzzy modeling is an efficient framework for these challenging classification tasks. Classification accuracy is higher than those produced by alternative machine learning methods. The number of algebraic calculations for the present method are significantly smaller and, therefore, the method is significantly faster than common Deep Neural Networks approaches. Moreover, DRB produced transparent AnYa fuzzy rules, which are human interpretable.

In this paper, a new deep rule-based approach using high-level ensemble feature descriptor is proposed for aerial scene classification. By creating an ensemble of three pre-trained deep convolutional neural networks for feature extraction, the proposed approach is able to extract more discriminative representations from the local regions of aerial images. With a set of massively parallel IF…THEN rules built upon the prototypes identified through a self-organizing, nonparametric, transparent and highly human-interpretable learning process, the proposed approach is able to produce the state-of-the-art classification results on the unlabeled images outperforming the alternatives. Numerical examples on benchmark datasets demonstrate the strong performance of the proposed approach.

A team of UAVs has great potential to handle real-world challenges. Knowing the environment is essential to perform in an effective manner. However, in many situations, a map of the environment will not be available. Additionally, for autonomous systems, it is necessary to have approaches that require little energy, computing, power, weight and size. To address this, we propose a light-weight, evolving, and memory efficient cooperative approach for estimating the map of an environment with a team of UAVs. Additionally, we present proof of-concept experiments with real-life flights, showing that we can estimate maps using an off-the-shelf web-camera.

Satellite scene images contain multiple sub-regions of different land use categories; however, traditional approaches usually classify them into a particular category only. In this paper, a new approach is proposed for automatically analyzing the semantic content of sub-regions of satellite images. At the core of the proposed approach is the recently introduced deep rule-based image classification method. The proposed approach includes a self-organizing set of transparent zero order fuzzy IF-THEN rules with human-interpretable prototypes identified from the training images and a pre-trained deep convolutional neural network as the feature descriptor. It requires a very short, nonparametric, highly parallelizable training process and can perform a highly accurate analysis on the semantic features of local areas of the image with the generated IF-THEN rules in a fully automatic way. Examples based on benchmark datasets demonstrate the validity and effectiveness of the proposed approach.

In this paper, we propose an approach to automatic detection of attacks on computer networks using data that combine the traffic generated with ’live’ intra-cloud virtual-machine (VM) migration. The method used in this work is the recently introduced typicality and eccentricity data analytics (TEDA) framework. We compare the results of applying TEDA with the traditionally used methods such as statistical analysis, such as k-means clustering. One of the biggest challenges in computer network analysis using statistical or numerical methods is the fact that the protocols information is composed of integer/string values and, thus, not easy to handle by traditional pattern recognition methods that deal with real values. In this study we consider as features the tuple {IP source, IP destination, Port source and Port destination} extracted from the network flow data in addition to the traditionally used real values that represent the number of packets per time or quantity of bytes per time. Using entropy of the IP data helps to convert the integer raw data into real valued signatures. The proposed solution permit to build a real-time anomaly detection system and reduce the number of information that is necessary for evaluation. In general, the systems based on traffic are fast and are used in real time but they do not produce good results in attacks that produce a flow hidden within the background traffic or within a high traffic that is produced by other application. We validate our approach an a dataset which includes attacks on the network port scan (NPS) and network scan (NS) that permit hidden flow within the normal traffic and see this attacks together with live migration which produces a higher traffic flow.

In complex environments, agents must be able to cooperate with previously unknown team-mates, and hence dynamically learn about other agents in the environment while searching for optimal actions. Previous works employ Monte Carlo Tree Search approaches. However, the search tree increases exponentially with the number of agents, and only scenarios with very small team sizes have been explored. Hence, in this paper we propose a history-based version of UCT Monte Carlo Tree Search, using a more compact representation than the original algorithm. We perform several experiments with a varying number of agents in the level-based foraging domain, an important testbed for ad-hoc teamwork. We achieve better overall performance than the state-of-the-art and better scalability with team size. Additionally, we contribute an open-source version of our system, making it easier for the research community to use the level-based foraging domain as a benchmark problern for ad-hoc teamwork.

In this paper, a fast, transparent, self-evolving, deep learning fuzzy rule-based (DLFRB) image classifier is proposed. This new classifier is a cascade of the recently introduced DLFRB classifier called MICE and an auxiliary SVM. The DLFRB classifier serves as the main engine and can identify a number of human interpretable fuzzy rules through a very short, transparent, highly parallelizable training process. The SVM based auxiliary plays the role of a conflict resolver when the DLFRB classifier produces two highly confident labels for a single image. Only the fundamental image transformation techniques (rotation, scaling and segmentation) and feature descriptors (GIST and HOG) are used for pre-processing and feature extraction, but the proposed approach significantly outperforms the state-of-art methods in terms of both time and precision. Numerical experiments based on a handwritten digits recognition problem are used to demonstrate the highly accurate and repeatable performance of the proposed approach.

In this paper, a new type of fast deep learning (DL) network for handwriting recognition is proposed. In contrast to the existing DL networks, the proposed approach has a clearly interpretable structure that is entirely data- driven and free from user- or problem-specific assumptions. Firstly, the fundamental image transformation techniques (rotation and scaling) used by other existing DL methods are used to improve the generalization. The commonly used descriptors are then used to extract the global features from the training set and based on them a bank/ensemble of zero order AnYa type fuzzy rule-based (FRB) models is built in parallel through the recently introduced Autonomous Learning Multiple Model (ALMMo) method. The final decision about the winning class label is made by a committee on the basis of the fuzzy mixture of the trained zero order ALMMo models. The training of the proposed MICE system is very efficient and highly parallelizable. It significantly outperforms the best-known methods in terms of time and is on par in terms of precision/accuracy. Critically, it offers a high level of interpretability, transparency of the classification model, full repeatability (unlike the methods that use probabilistic elements) of the results. Moreover, it allows an evolving scenario whereby the data is provided in an incremental, online manner and the system structure evolves in parallel with the classification which opens opportunities for online and real-time applications (on a sample by sample basis). Numerical examples from the well-known handwritten digits recognition problem (MNIST) were used and the results demonstrated the very high repeatable performance after a very short training process exhibiting high level of interpretability, transparency.

In this paper, we introduce an approach to classify gender and age from images of human faces which is an essential part of our method for autonomous detection of anomalous human behaviour. Human behaviour is often uncertain, and sometimes it is affected by emotion or environment. Automatic detection can help to recognise human behaviour which later can assist in investigating suspicious events. Central to our proposed approach is the recently introduced transfer learning. It was used on the basis of deep learning and successfully applied to image classification area. This paper is a continuous study from previous research on heterogeneous data in which we use images as supporting evidence. We present a method for image classification based on a pre-trained deep model for feature extraction and representation followed by a Support Vector Machine classifier. Because very few data sets with labels of gender and age exist of face images, we build one dataset named GAFace and applied our proposed method to this dataset achieving excellent results and robustness (gender classification: 90.33% and age classification: 80.17% accuracy) approaching human performance.

In this paper, a new type of 0-order multi-model classifier, called Autonomous Learning Multiple-Model (ALMMo-0), is proposed. The proposed classifier is non-iterative, feedforward and entirely data-driven. It automatically extracts the data clouds from the data per class and forms 0-order AnYa type fuzzy rule-based (FRB) sub-classifier for each class. The classification of new data is done using the “winner takes all” strategy according to the scores of confidence generated objectively based on the mutual distribution and ensemble properties of the data by the sub-classifiers. Numerical examples based on benchmark datasets demonstrate the high performance and computation-efficiency of the proposed classifier.

Human action recognition is an imperative research area in the field of computer vision due to its numerous applications. Recently, with the emergence and successful deployment of deep learning techniques for image classification, object recognition, and speech recognition, more research is directed from traditional handcrafted to deep learning techniques. This paper presents a novel method for human action recognition based on a pre-trained deep CNN model for feature extraction & representation followed by a hybrid Support Vector Machine (SVM) and K-Nearest Neighbor (KNN) classifier for action recognition. It has been observed that already learnt CNN based representations on large-scale annotated dataset could be transferred to action recognition task with limited training dataset. The proposed method is evaluated on two well-known action datasets, i.e., UCF sports and KTH. The comparative analysis confirms that the proposed method achieves superior performance over state-of-the-art methods in terms of accuracy.

In this paper, a new type of feedforward non-parametric deep learning network with automatic feature extraction is proposed. The proposed network is based on human-understandable local aggregations extracted directly from the images. There is no need for any feature selection and parameter tuning. The proposed network involves nonlinear transformation, segmentation operations to select the most distinctive features from the training images and builds RBF neurons based on them to perform classification with no weights to train. The design of the proposed network is very efficient (computation and time wise) and produces highly accurate classification results. Moreover, the training process is parallelizable, and the time consumption can be further reduced with more processors involved. Numerical examples demonstrate the high performance and very short training process of the proposed network for different applications.

This paper presents an autonomous Robust Evolving Cloud-based Controller (RECCo). The control algorithm is a fuzzy type with non-parametric (cloud-based) antecedent part and adaptive PID-R consequent part. The procedure starts with zero clouds (fuzzy rules) and the structure evolves during performing the process control. The PID-R parameters of the first cloud are initialized with zeros and furthermore, they are adapted on-line with a stable adaptation mechanism based on Lyapunov approach. The RECCo controller does not require any mathematical model of the controlled process but just basic information such as input and output range and the estimated value of the dominant time constant. Due to the problem space normalization the design parameters are fixed. The proposed controller with the same initial design parameters was tested on two different simulation examples. The experimental results show the convergence of the adaptive parameters and the effectiveness of the proposed algorithm.

In this paper, a new data partitioning algorithm, named “local modes-based data partitioning”, is proposed. This algorithm is entirely data-driven and free from any user input and prior assumptions. It automatically derives the modes of the empirically observed density of the data samples and results in forming parameter-free data clouds. The identified focal points resemble Voronoi tessellations. The proposed algorithm has two versions, namely, offline and evolving. The two versions are both able to work separately and start “from scratch”, they can also perform a hybrid. Numerical experiments demonstrate the validity of the proposed algorithm as a fully autonomous partitioning technique, and achieve better performance compared with alternative algorithms.

In this paper, a novel autonomous data-driven clustering approach, called AD_clustering, is presented for live data streams processing. This newly proposed algorithm is a fully unsupervised approach and entirely based on the data samples and their ensemble properties, in the sense that there is no need for user-predefined or problem-specific assumptions and parameters, which is a problem most of the current clustering approaches suffer from. Moreover, the proposed approach automatically evolves its structure according to the experimentally observable streaming data and is able to recursively update its self-defined parameters using only the current data sample; meanwhile, it discards all the previously processed data samples. Experimental results based on benchmark datasets exhibit the higher performance of the proposed fully autonomous approach compared with the comparative approaches requiring user- and problem-specific parameters to be predefined. This new clustering algorithm is a promising tool for further applications in the field of real-time streaming data analytics.

In this paper, a novel empirical data analysis approach (abbreviated as EDA) is introduced which is entirely data-driven and free from restricting assumptions and pre-defined problem- or user-specific parameters and thresholds. It is well known that the traditional probability theory is restricted by strong prior assumptions which are often impractical and do not hold in real problems. Machine learning methods, on the other hand, are closer to the real problems but they usually rely on problem- or user-specific parameters or thresholds making it rather art than science. In this paper we introduce a theoretically sound yet practically unrestricted and widely applicable approach that is based on the density in the data space. Since the data may have exactly the same value multiple times we distinguish between the data points and unique locations in the data space. The number of data points, k is larger or equal to the number of unique locations, l and at least one data point occupies each unique location. The number of different data points that have exactly the same location in the data space (equal value), ƒ can be seen as frequency. Through the combination of the spatial density and the frequency of occurrence of discrete data points, a new concept called multimodal typicality, τMM is proposed in this paper. It offers a closed analytical form that represents ensemble properties derived entirely from the empirical observations of data. Moreover, it is very close (yet different) from the histograms, from the probability density function (pdf) as well as from fuzzy set membership functions. Remarkably, there is no need to perform complicated pre-processing like clustering to get the multimodal representation. Moreover, the closed form for the case of Euclidean, Mahalanobis type of distance as well as some other forms (e.g. cosine-based dissimilarity) can be expressed recursively making it applicable to data streams and online algorithms. Inference/estimation of the typicality of data points that were not present in the data so far can be made. This new concept allows to rethink the very foundations of statistical and machine learning as well as to develop a series of anomaly detection, clustering, classification, prediction, control and other algorithms.

In this paper, we propose a novel approach to unsupervised and online data classification. The algorithm is based on the statistical analysis of selected features and development of a self-evolving fuzzy-rule-basis. It starts learning from an empty rule basis and, instead of offline training, it learns “on-the-fly”. It is free of parameters and, thus, fuzzy rules, number, size or radius of the classes do not need to be pre-defined. It is very suitable for the classification of online data streams with real-time constraints. The past data do not need to be stored in memory, since that the algorithm is recursive, which makes it memory and computational power efficient. It is able to handle concept-drift and concept-evolution due to its evolving nature, which means that, not only rules/classes can be updated, but new classes can be created as new concepts emerge from the data. It can perform fuzzy classification/soft-labeling, which is preferred over traditional crisp classification in many areas of application. The algorithm was validated with an industrial pilot plant, where online calculated period and amplitude of control signal were used as input to a fault diagnosis application. The approach, however, is generic and can be applied to different problems and with much higher dimensional inputs. The results obtained from the real data are very significant.

In this paper the design, implementation, and evaluation of a general purpose smartphone based intelligent surveillance system is presented. It has two main elements; i) a detection module, and ii) a classification module. The detection module is based on the recently introduced approach that combines the well-known background subtraction method with the optical flow and recursively estimated density. The classification module is based on a neural network using Deep Learning methodology. Firstly, the architecture design of the convolutional neural network is presented and analyzed in the context of the four selected architectures (two of them recent successful types) and two custom modifications specifically made for the problem at hand. The results are carefully evaluated, and the best one is selected to be used within the proposed system. In addition, the system is implemented on both a PC (using Linux type OS) and on a smartphone (using Android). In addition to the compatibility with all modern Android-based devices, most GPU-powered platforms such as Raspberry Pi, Nvidia Tegra X1 and Jetson run on Linux. The proposed system can easily be installed on any such device benefiting from the advantage of parallelisation for faster execution. The proposed system achieved a performance which surpasses that of a human (classification accuracy of the top 1 class >95.9% for automatic recognition of a detected object into one of the seven selected categories. For the top-2 classes, the accuracy is even higher (99.85%). That means, at least, one of the two top classes suggested by the system is correct. Finally, a number of visual examples are showcased of the system in use in both PC and Android devices.

It is well known that clustering is an unsupervised machine learning technique. However, most of the clustering methods need setting several parameters such as number of clusters, shape of clusters, or other user- or problem-specific parameters and thresholds. In this paper, we propose a new clustering approach which is fully autonomous, in the sense that it does not require parameters to be pre-defined. This approach is based on data density automatically derived from their mutual distribution in the data space. It is called ADD clustering (Autonomous Data Density based clustering). It is entirely based on the experimentally observable data and is free from restrictive prior assumptions. This new method exhibits highly accurate clustering performance. Its performance is compared on benchmarked data sets with other competitive alternative approaches. Experimental results demonstrate that ADD clustering significantly outperforms other clustering methods yet does not require restrictive user- or problem-specific parameters or assumptions. The new clustering method is a solid basis for further applications in the field of data analytics.

Analyzing and predicting the high frequency trading (HFT) financial data stream is very challenging due to the fast arrival times and large amount of the data samples. Aiming at solving this problem, an online evolving fuzzy rule-based prediction model is proposed in this paper. Because this prediction model is based on evolving fuzzy rule-based systems and a novel, simpler form of data density, it can autonomously learn from the live data stream, automatically build/remove its rules and recursively update the parameters. This model responds quickly to all unpredictable sudden changes of financial data and re-adjusts itself to follow the new data pattern. Experimental results show the excellent prediction performance of the proposed approach with real financial data stream regardless of quick shifts of data patterns and frequent appearances of abnormal data samples.

In this paper, we propose a new approach to fuzzy data clustering. We present a new algorithm, called TEDA-Cloud, based on the recently introduced TEDA approach to outlier detection. TEDA-Cloud is a statistical method based on the concepts of typicality and eccentricity able to group similar data observations. Instead of the traditional concept of clusters, the data is grouped in the form of granular unities called data clouds, which are structures with no pre-defined shape or set boundaries. TEDA-Cloud is a fully autonomous and self-evolving algorithm that can be used for data clustering of online data streams and applications that require real-time response. Since it is fully autonomous, TEDA-Cloud is able to “start from scratch” (from an empty knowledge basis), create, update and merge data clouds, in a fully autonomous manner, without requiring any user-defined parameters (e.g. number of clusters, size, radius) or previous training. Moreover, TEDA-Cloud, unlike most of the traditional statistical approaches, does not rely on a specific data distribution or on the assumption of independence of data samples. The results, obtained from multiple data sets that are very well known in literature, are very encouraging.

This paper presents an improved version and a modification of Robust Evolving Cloud-based Controller (RECCo). The first modification is normalization of data space in RECCo. As a consequence, some of the evolving and adaptation parameters become independent of the range of the process output signal. Thus the controller tuning is simplified which makes the approach more appealing for the use in practical applications. The data space normalization is general and is used with Euclidean norm, but other distance metrics could also be used. Beside the normalization new adaptation scheme of the controller gain is proposed which improves the control performance in the case of a negative initial error in starting phase of the evolving process. At the end, different simulation scenarios are tested and analyzed for further practical implementation of the Cloud-based controller into real environments. For that reason a detail simulation study of a plate heat exchanger is performed and different scenarios were analyzed.

In this article the novel clustering and regression methods TEDACluster and TEDAPredict methods are described additionally to recently proposed evolving classifier TEDAClass. The algorithms for classification, clustering and regression are based on the recently proposed AnYa type fuzzy rule based system. The novel methods use the recently proposed TEDA framework capable of recursive processing of large amounts of data. The framework is capable of computationally cheap exact update of data per sample, and can be used for training ‘from scratch’. All three algorithms are evolving that is they are capable of changing its own structure during the update stage, which allows to follow the changes within the model pattern.

Fault detection is a task of major importance in industry nowadays, since that it can considerably reduce the risk of accidents involving human lives, in addition to production and, consequently, financial losses. Therefore, fault detection systems have been largely studied in the past few years, resulting in many different methods and approaches to solve such problem. This paper presents a detailed study on fault detection on industrial processes based on the recently introduced eccentricity and typicality data analytics (TEDA) approach. TEDA is a recursive and non-parametric method, firstly proposed to the general problem of anomaly detection on data streams. It is based on the measures of data density and proximity from each read data point to the analyzed data set. TEDA is an online autonomous learning algorithm that does not require a priori knowledge about the process, is completely free of user- and problem-defined parameters, requires very low computational effort and, thus, is very suitable for real-time applications. The results further presented were generated by the application of TEDA to a pilot plant for industrial process.

In this paper a new density-based, non-frequentistic data analytics tool, called typicality distribution function (TDF) is proposed. It is a further development of the recently introduced typicality- and eccentricity-based data analytics (TEDA) framework. The newly introduced TDF and its standardized form offer an effective alternative to the widely used probability distribution function (pdf), however, remaining free from the restrictive assumptions made and required by the latter. In particular, it offers an exact solution for any (except a single point) amount of non-coinciding data samples. For a comparison, that the well developed and widely used traditional probability theory and related statistical learning approaches require (theoretically) an infinitely large amount of data samples/ observations, although, in practice this requirement is often ignored. Furthermore, TDF does not require the user to pre-select or assume a particular distribution (e.g. Gaussian or other) or a mixture of such distributions or to pre-define the number of such distributions in a mixture. In addition, it does not require the individual data items to be independent. At the same time, the link with the traditional statistical approaches such as the well-known “nσ” analysis, Chebyshev inequality, etc. offers the interesting conclusion that without the restrictive prior assumptions listed above to which these traditional approaches are tied up the same type of analysis can be made using TDF automatically. TDF can provide valuable information for analysis of extreme processes, fault detection and identification were the amount of observations of extreme events or faults is usually disproportionally small. The newly proposed TDF offers a non-parametric, closed form analytical (quadratic) description extracted from the real data realizations exactly in contrast to the usual practice where such distributions are being pre-assumed or approximated. For example, so called particle filters are also a non-parametric approximation of the traditional statistics; however, they suffer from computational complexity and introduce a large number of dummy data. In addition to that, for several types of proximity/similarity measures (such as Euclidean, Mahalonobis, cosine) it can be calculated recursively, thus, computationally very efficiently and is suitable for real time and online algorithms. Moreover, with a very simple example, it has been illustrated that while traditional probability theory and related statistical approaches can lead in some cases to paradoxically incorrect results and/or to the need for hard prior assumptions to be made. In contrast, the newly proposed TDF can offer a logically meaningful result and an intuitive interpretation automatically and exactly without any prior assumptions. Finally, few simple univariate examples are provided and the process of inference is discussed and the future steps of the development of TDF and TEDA are outlined. Since it is a new fundamental theoretical innovation the areas of applications of TDF and TEDA can span from anomaly detection, clustering, classification, prediction, control, regression to (Kalman-like) filters. Practical applications can be even wider and, therefore, it is difficult to list all of them.

In this paper we demonstrate a new density based clustering technique, CODSAS, for online clustering of streaming data into arbitrary shaped clusters. CODAS is a two stage process using a simple local density to initiate micro-clusters which are then combined into clusters. Memory efficiency is gained by not storing or re-using any data. Computational efficiency is gained by using hyper-spherical micro-clusters to achieve a micro-cluster joining technique that is dimensionally independent for speed. The micro-clusters divide the data space in to sub-spaces with a core region and a non-core region. Core regions which intersect define the clusters. A threshold value is used to identify outlier micro-clusters separately from small clusters of unusual data. The cluster information is fully maintained on-line. In this paper we compare CODAS with ELM, DEC, Chameleon, DBScan and Denstream and demonstrate that CODAS achieves comparable results but in a fully on-line and dimensionally scale-able manner.

A new approach to autonomously detect and track moving objects in a video captured by a moving camera from a UAV in real-time is proposed in this paper. The introduced approach replaces the need for a human operator to perform video analytics by autonomously detecting moving objects and clustering them for tracking purposes. The effectiveness of the introduced approach is tested on the footage taken from a real UAV and the evaluation results are demonstrated in this paper.

In this paper, we propose a new eccentricity- based anomaly detection principle and algorithm. It is based on a further development of the recently introduced data analytics framework (TEDA - from typicality and eccentricity data analytics). We compare TEDA with the traditional statistical approach and prove that TEDA is a generalization of it in regards to the well-known “nσ” analysis (TEDA gives exactly the same result as the traditional “nσ” analysis but it does not require the restrictive prior assumptions that are made for the traditional approach to be in place). Moreover, it offers a non-parametric, closed form analytical descriptions (models of the data distribution) to be extracted from the real data realizations, not to be pre-assumed. In addition to that, for several types of proximity/similarity measures (such as Euclidean, cosine, Mahalonobis) it can be calculated recursively, thus, computationally very efficiently and is suitable for real time and online algorithms. Building on the per data sample, exact information about the data distribution in a closed analytical form, in this paper we propose a new less conservative and more sensitive condition for anomaly detection. It is quite different from the traditional “nσ” type conditions. We demonstrate example where traditional conditions would lead to an increased amount of false negatives or false positives in comparison with the proposed condition. The new condition is intuitive and easy to check for arbitrary data distribution and arbitrary small (but not less than 3) amount of data samples/points. Finally, because the anomaly/novelty/change detection is very important and basic data analysis operation which is in the fundament of such higher level tasks as fault detection, drift detection in data streams, clustering, outliers detection, autonomous video analytics, particle physics, etc. we point to some possible applications which will be the domain of future work.

In this paper, we propose a new approach to data density estimation based on the total sum of distances from a data point, and the recently introduced Recursive Density Estimation technique. It is suitable for autonomous real-time video analytics problems, and has been specifically designed to be executed very fast; it uses integer-only arithmetic with no divisions and no floating point numbers (no FLOPs), making it particularly useful in situations where a hardware floating point unit may not be available, such as on embedded hardware and digital signal processors, allowing for high definition video to be processed for novelty detection in real-time.

Autonomously detecting novelties using background subtraction has quickly become a very important area of image analysis with many different approaches to novelty detection and the output therein. The ultimate goal of the approaches is to be robust to false detections and noise whilst using as little computational power as possible. This review focuses on some of the most prominent pixel-wise background subtraction techniques currently in use, and compares and contrasts their attributes and capabilities. The purpose of this review is to practically summarize the pixel-wise approaches and suggest a way forward from these techniques.

In this paper, a new fully unsupervised approach to fault detection and identification is proposed. It is based on a two-stage algorithm and starts with the recursive density estimation (RDE) in the feature space. The choice of the features is important and in the real world process that we consider these are control and error related variables. The basis of the proposed approach is the fully unsupervised evolving classifier AutoClass which can be seen as an extension of the earlier one, but is using data clouds and data density information. It has to be stressed that the density in the data space is not the same as the well-known and widely used in statistics probability density function (pdf) although it looks similar. The density in the data space, D is pivotal and instrumental for anomaly detection. It can be calculated recursively, which makes it very efficient in terms of memory, computational power and, thus, applicable to on-line applications. Importantly, the proposed method not only can detect anomalies, but also can identify and diagnose the fault during the second stage of the process. While the first stage is centred around RDE, the second stage is based on the evolving fuzzy rule-based (FRB) classifier AutoClass. A key advantage of AutoClass is that it is fully unsupervised (there is no need to pre-specify the fuzzy rules, number of classes) and can start learning "from scratch". AutoClass can be initialised with some prior knowledge (assuming that it does exists) and evolve/develop it further, but that is not mandatory. This new approach is generic, but in this paper without limiting the concept it is validated on a lab based control kit. In this particular example, the features are the control and error signals. The results significantly outperform alternative methods, which is in addition to the advantages that the approach is autonomous.

In this paper, a novel evolving fuzzy rule-based classifier is presented. The proposed classifier addresses the three fundamental issues of data stream learning, viz., computational efficiency in terms of processing time and memory requirements, adaptive to changes, and robustness to noise. Though, there are several online classifiers available, most of them do not take into account all the three issues simultaneously. The newly proposed classifier is inherently adaptive and can attend to any minute changes as it learns the rules in online manner by considering each incoming example. However, it should be emphasized that it can easily distinguish noise from new concepts and automatically handles noise. The performance of the classifier is evaluated using real-life data with evolving characteristic and compared with state-of-the-art adaptive classifiers. The experimental results show that the classifier attains a simple model in terms of number of rules. Further, the memory requirements and processing time per sample does not increase linearly with the progress of the stream. Thus, the classifier is capable of performing both prediction and model update in real-time in a streaming environment.

In this paper an improved robust evolving cloud-based controller (RECCo) for a class of nonlinear processes is introduced. The controller is based on parameter-free premise (IF) part. The consequence in this case is given in the form of PID-type controller. The three adjustable parameters of PID controller are updated on-line with a stable adaptation mechanism based on Lyapunov approach such that the output of the process tracks the desired model-reference trajectory. The proposed algorithm has also ability to add new rules or new clouds when this is necessary to improve the whole behaviour of the controlled process. This means that RECCo controller evolves the control structure and adjusts at the same time the parameters of the controller in an on-line manner, while performing the control of the plant. This approach is an example of almost parameter-free approach, because it does not use any off-line pre-training nor the explicit model of the plant and requires almost no parameter tuning. The proposed algorithm is tested on an artificial nonlinear first-order process and on a simulated hydraulic plant.

In this paper, a new online evolving clustering approach for streaming data is proposed, named Dynamically Evolving Clustering method. The clustering approach attempts to meet the following three key requirements of data stream clustering: (i) fast and memory efficient (ii) adaptive (iii) robust to noise. The proposed clustering approach processes one sample at a time and makes necessary changes to the model and then forgets the processed sample. This feature naturally makes it adaptive to changes in the data pattern. The clustering method considers both distance and weight before generating new clusters. This avoids generation of large number of clusters. Further, to capture the dynamics of the data stream, the weight uses an exponential decay model. Since in data streaming environment, a low density cluster can be outlier points or seed of actual cluster, DEC applies a strategy that enables detecting and removing only those low density clusters that are real outliers. To evaluate the performance of the proposed clustering approach, experiments were conducted using benchmark dataset. The results show that the Dynamically Evolving Clustering approach can separate the data well which are evolving in nature.

A new algorithm for symbol recognition is proposed in this paper. It is based on the AutoClass classifier [1], [2], which itself is a version of the evolving fuzzy rule-based classifier eClass [3] in which AnYa[1] type of fuzzy rules and data density are used. In this classifier, symbol recognition task is divided into two stages: feature extraction, and recognition based on feature vector. This approach gives flexibility, allowing us to use various feature sets for one classifier. The feature extraction is performed by means of gist image descriptors[4] augmented by several additional features. In this method, we map the symbol images into the feature space, and then we apply AutoClass classifier in order to recognise them. Unlike many of the state-of-the-art algorithms, the proposed algorithm is evolving, i.e. it has a capability of incremental learning as well as ability to change its structure during the training phase. The classifier update is performed sample by sample, and we should not memorize the training set to provide recognition or further update. It gives a possibility to adapt the classifier to the broadening and changing data sets, which is especially useful for large scale systems improvement during exploitation. More, the classifier is computationally cheap, and it has shown stable recognition time during the increase of training data set size that is extremely important for online applications.

Distributed Denial of Service (DDoS) attacks on network infrastructure are one of the major challenges facing network service providers. Despite the recent rise of low-volume application-level attacks, volume-based DDoS attacks still dominate, with peak traffic rates of 80Gbps being observed recently. This prompts the need for more efficient ways to deal with them. Meanwhile, service providers are struggling to acquire the right technology, resources and expertise to offer more resilient and reliable services. One of the solutions to help address this issue is to adopt an autonomous resilience strategy that systematically coordinates resilience related activities such as detecting and mitigating attacks. In this paper, we study an implementation of an autonomous traffic rate limiter - a function that can be used to mitigate DDoS attacks - that capitalises on the AnYa algorithm, an autonomous learning systems (ALS) algorithm that provides advanced features that are crucial to support an autonomous resilience strategy. These features include self-structuring and support for online learning. In our study, we experimentally show how remediation and recovery processes can be realized autonomously, in response to changes in the operational policy.