Search

Incremental learning is a learning paradigm in which a model is updated continuously as new data becomes available, and its main challenge is to adapt to non-stationary environments without the time-consuming re-training process. Many efforts have been made on incremental supervised learning. However, providing sufficient labeled data remains a major problem. Recently, domain adaptation methods have gained attention. These methods aim to leverage the knowledge from an auxiliary source domain to boost the performance of the model in the target domain by reducing the domain discrepancy between them. Regarding these issues, in the present paper, a proposed model aims to incrementally learn a new domain characterized by drifts due to a non-stationary environment. It utilizes an unsupervised, fuzzy-based domain adaptation to classify data streams faced with concept drift while accounting for a label-agnostic incremental setting in the target domain. Incremental learning updates occur whenever the entropy-based metric indicates uncertainty, ensuring informative samples are integrated. Also, outdated samples are forgotten during the training stage using the dynamic sample weighting strategy. Through experimentation on forty-five tasks, the superiority of the proposed model in handling dynamic adaptation on non-stationary domains is demonstrated, showcasing improvements in accuracy and computational efficiency. [ABSTRACT FROM AUTHOR]

Learning with data streams has attracted much attention in recent decades. Conventional approaches typically assume that the feature and label of a data item can be timely observed at each round. In many real-world tasks, however, it often occurs that either the feature or the label is observed firstly while the other arrives with delay. For instance, in distributed learning systems, a central processor collects training data from different sub-processors to train a learning model, whereas the feature and label of certain data items can arrive asynchronously due to network latency. The problem of learning with asynchronous feature or label in streams encompasses many applications but still lacks sound solutions. In this article, we formulate the problem and propose a new approach to alleviate the negative effect of asynchronicity and mining asynchronous data streams. Our approach carefully exploits the timely arrived information and builds an online ensemble structure to adaptively reuse historical models and instances. We provide the theoretical guarantees of our approach and conduct extensive experiments to validate its effectiveness. [ABSTRACT FROM AUTHOR]

Finding top-k items in data streams is a fundamental problem in data mining. Unbiased estimation is well acknowledged as an elegant and important property for top-k algorithms. In this paper, we propose a novel sketch algorithm, called WavingSketch, which is more accurate than existing unbiased algorithms. We theoretically prove that WavingSketchcan provide unbiased estimation, and derive its error bound. WavingSketchis generic to measurement tasks, and we apply it to five applications: finding top-k frequent items, finding top-k heavy changes, finding top-k persistent items, finding top-k Super-Spreaders, and join-aggregate estimation. Our experimental results show that, compared with the state-of-the-art Unbiased Space-Saving, WavingSketchachieves 10 × faster speed and 10 3 × smaller error on finding frequent items. For other applications, WavingSketchalso achieves higher accuracy and faster speed. All related codes are open-sourced at GitHub (https://github.com/WavingSketch/Waving-Sketch). [ABSTRACT FROM AUTHOR]

Recently, advanced AI systems equipped with sophisticated learning algorithms have emerged, enabling the processing of extensive streaming data for online decision-making in diverse domains. However, the widespread deployment of these systems has prompted concerns regarding potential ethical issues, particularly the risk of discrimination that can adversely impact certain community groups. This issue has been proven to be challenging to address in the context of streaming data, where data distribution can change over time, including changes in the level of discrimination within the data. In addition, transparent models like decision trees are favoured in such applications because they illustrate the decision-making process. However, it is essential to keep the models compact because the explainability of large models can diminish. Existing methods usually mitigate discrimination at the cost of accuracy. Accuracy and discrimination, therefore, can be considered conflicting objectives. Current methods are still limited in controlling the trade-off between these conflicting objectives. This paper proposes a method that can incrementally learn classification models from streaming data and automatically adjust the learnt models to balance multi-objectives simultaneously. The novelty of this research is to propose a multi-objective algorithm to maximise accuracy, minimise discrimination and model size simultaneously based on swarm intelligence. Experimental results using six real-world datasets show that the proposed algorithm can evolve fairer and simpler classifiers while maintaining competitive accuracy compared to existing state-of-the-art methods tailored for streaming data. [ABSTRACT FROM AUTHOR]

Deep learning-based approaches have gained popularity for many applications in recent years and have become the state-of-the-art method in machine learning applications. However, most deep learning applications manipulate static data sets that do not vary over time. This is not the case for several significant recent applications, particularly for online learning on data streams that require real-time adjustment to the latest data context. This paper presents an architecture derived from the theory of transfer learning utilizing Extreme Learning Machine (ELM) and Recurrent Neural Network (RNN) methods. Unlike most data stream applications, when receiving high volumes of data at a high velocity, the proposed method does not store and reuse previous observations. The proposed model uses a Deep Echo State Network (DeepESN) for the primary classification task to compensate for the lack of past data. More precisely, instead of the data itself, we propagate generative models so that historical training data that is not stored can be regenerated. Before the DeepESN block, a layer of pre-trained ELMs was used to have generic feature engineering. The proposed method was tested on well-known benchmark data sets, and it outperformed the state-of-the-art models by an accuracy improvement ranging from 0.02 to 3.3 percent. The impact of data set regeneration on the learning process was studied using different RNN approaches. [ABSTRACT FROM AUTHOR]

The synergy of edge computing and Machine Learning (ML) holds immense potential for revolutionizing Internet of Things (IoT) applications, particularly in scenarios characterized by high-speed, continuous data generation. Offline ML algorithms struggle with streaming data as they rely on static datasets for model construction. In contrast, Online Machine Learning (OML) adapts to changing environments by training the model with each new observation in real-time. However, developing OML algorithms introduces complexities such as bias and variance considerations, making the selection of suitable estimators challenging. In this challenging landscape, ensemble learning emerges as a promising approach, offering a strategic framework to navigate the bias-variance tradeoff and enhance prediction accuracy by amalgamating outputs from diverse ML models. This paper introduces a novel ensemble method tailored for edge computing environments, designed to efficiently operate on resource-constrained devices while accommodating various online learning scenarios. The primary objective is to enhance predictive accuracy at the edge, thereby empowering IoT applications with robust decision-making capabilities. Our study addresses the critical challenges of ML in resource-constrained edge computing environments, offering practical insights for enhancing predictive accuracy and scalability in IoT applications. To validate our ensemble's efficacy, we conducted comprehensive experimental evaluations leveraging both synthetic and real-world datasets. The results indicate that our ensemble surpassed state-of-the-art data stream algorithms and ensemble regressors across a range of regression metrics, underlining its superior predictive prowess. Furthermore, we scrutinized the ensemble's performance within the realm of auto-scaling for Virtual Network Function (VNF)-based applications situated at the network's edge, thereby elucidating its applicability and scalability in real-world scenarios. [ABSTRACT FROM AUTHOR]

Many real-world data stream applications not only suffer from concept drift but also class imbalance. Yet, very few existing studies investigated this joint challenge. Data difficulty factors, which have been shown to be key challenges in class imbalanced data streams, are not taken into account by existing approaches when learning class imbalanced data streams. In this work, we propose a drift adaptable oversampling strategy to synthesise minority class examples based on stream clustering. The motivation is that stream clustering methods continuously update themselves to reflect the characteristics of the current underlying concept, including data difficulty factors. This nature can potentially be used to compress past information without caching data in the memory explicitly. Based on the compressed information, synthetic examples can be created within the region that recently generated new minority class examples. Experiments with artificial and real-world data streams show that the proposed approach can handle concept drift involving different minority class decomposition better than existing approaches, especially when the data stream is severely class imbalanced and presenting high proportions of safe and borderline minority class examples. [ABSTRACT FROM AUTHOR]

Abstract Recently, advanced AI systems equipped with sophisticated learning algorithms have emerged, enabling the processing of extensive streaming data for online decision-making in diverse domains. However, the widespread deployment of these systems has prompted concerns regarding potential ethical issues, particularly the risk of discrimination that can adversely impact certain community groups. This issue has been proven to be challenging to address in the context of streaming data, where data distribution can change over time, including changes in the level of discrimination within the data. In addition, transparent models like decision trees are favoured in such applications because they illustrate the decision-making process. However, it is essential to keep the models compact because the explainability of large models can diminish. Existing methods usually mitigate discrimination at the cost of accuracy. Accuracy and discrimination, therefore, can be considered conflicting objectives. Current methods are still limited in controlling the trade-off between these conflicting objectives. This paper proposes a method that can incrementally learn classification models from streaming data and automatically adjust the learnt models to balance multi-objectives simultaneously. The novelty of this research is to propose a multi-objective algorithm to maximise accuracy, minimise discrimination and model size simultaneously based on swarm intelligence. Experimental results using six real-world datasets show that the proposed algorithm can evolve fairer and simpler classifiers while maintaining competitive accuracy compared to existing state-of-the-art methods tailored for streaming data.

Internet of Things (IoT) systems generate a vast volume of data to process, and the volume of data generated by the IoT will continue to expand exponentially in the future. Due to the large-scale nature of such systems and their geographically distributed nature, how to optimally place operators and brokers for data processing across cloud, fog, and edge is the main concern of this paper. Furthermore, in the past, IoT systems relied on cloud-based brokers. If the communication between operators in an IoT application has to be regulated by a broker located in the cloud, the advantages brought by the processing of nodes located at the edge and the fog may be neglected. We consider various factors, such as the impact of different network topologies on operators and brokers, the needs of IoT application users, the performance of sensor layers, and the location characteristics of IoT sensors. In diverse placement strategies, the characteristics of the broker have been given special attention. This paper considers the combined architecture of the operator and broker on the edge, fog, and cloud as an optimization problem of minimizing computational processing and data transmission cost. We propose jointly placing the IoT application's operator and broker on the edge, fog, and cloud nodes. Then we proposed two algorithms: the low-complexity heuristic algorithm named GREEDY and the genetic algorithm called GA. Simulation results show that, in most situations, the GA placement method outperforms others and provides a near-optimal solution. [ABSTRACT FROM AUTHOR]

Nowadays data streams from different sources, like blockchain‐based and traditional financial transactions, social networks, and interconnected Internet of Things (IoT) devices, are becoming increasingly large in volume and the need to recognize patterns in real time from these streams, while adapting to their velocity and veracity, is emerging. Established machine learning algorithms used for pattern recognition methods have not been designed taking under account the volume, velocity, diversity, and accuracy of the data streams. This research contributes with an approach for assessing the pattern recognition capabilities of established machine learning algorithms when handling volatile data in real time and proposes a system that adapts the algorithms to the requirements of data streams, as well as assesses their pattern recognition capabilities based on established criteria. The system was applied for assessing five machine learning algorithms with input from a data stream from Bluetooth beacons tracking consumers in a retail store. This research can support future data scientists and analysts who need to reveal data patterns in big, volatile data streams in real time in order to support effective decision‐making in the respective application domain. Copyright © 2024 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Online anomaly detection is critical for industrial safety and security monitoring but is facing challenges due to the complexity of evolving data streams from working conditions and performance degradation. Unfortunately, existing approaches fall short of such challenges, and these models may be disabled, suffering from the evolving data distribution. The paper presents a framework for online anomaly detection of data streams, of which the baseline algorithm is the incremental learning method of Growing Neural Gas (GNG). It handles complex and evolving data streams via the proposed model Bayes-Optimized Adaptive Growing Neural Gas (BOA-GNG). Firstly, novel learning rate adjustment and neuron addition strategies are designed to enhance the model convergence and data presentation capability. Then, the Bayesian algorithm is adopted to realize the fine-grained search of BOA-GNG-based hyperparameters. Finally, comprehensive studies with six data sets verify the superiority of BOA-GNG in terms of detection accuracy and computational efficiency. [ABSTRACT FROM AUTHOR]

Concept drift is a phenomenon where the distribution of data streams changes over time. When this happens, model predictions become less accurate. Hence, models built in the past need to be re-learned for the current data. Two design questions need to be addressed in designing a strategy to re-learn models: which type of concept drift has occurred, and how to utilize the drift type to improve re-learning performance. Existing drift detection methods are often good at determining when drift has occurred. However, few retrieve information about how the drift came to be present in the stream. Hence, determining the impact of the type of drift on adaptation is difficult. Filling this gap, we designed a framework based on a lazy strategy called Type-Driven Lazy Drift Adaptor (Type-LDA). Type-LDA first retrieves information about both how and when a drift has occurred, then it uses this information to re-learn the new model. To identify the type of drift, a drift type identifier is pre-trained on synthetic data of known drift types. Furthermore, a drift point locator locates the optimal point of drift via a sharing loss. Hence, Type-LDA can select the optimal point, according to the drift type, to re-learn the new model. Experiments validate Type-LDA on both synthetic data and real-world data, and the results show that accurately identifying drift type can improve adaptation accuracy. [ABSTRACT FROM AUTHOR]

Hyper-parameter tuning of machine learning models has become a crucial task in achieving optimal results in terms of performance. Several researchers have explored the optimisation task during the last decades to reach a state-of-the-art method. However, most of them focus on batch or offline learning, where data distributions do not change arbitrarily over time. On the other hand, dealing with data streams and online learning is a challenging problem. In fact, the higher the technology goes, the greater the importance of sophisticated techniques to process these data streams. Thus, improving hyper-parameter self-tuning during online learning of these machine learning models is crucial. To this end, in this paper, we present MESSPT, an evolutionary algorithm for self-hyper-parameter tuning for data streams. We apply Differential Evolution to dynamically-sized samples, requiring a single pass-over of data to train and evaluate models and choose the best configurations. We take care of the number of configurations to be evaluated, which necessarily has to be reduced, thus making this evolutionary approach a micro-evolutionary one. Furthermore, we control how our evolutionary algorithm deals with concept drift. Experiments on different learning tasks and over well-known datasets show that our proposed MESSPT outperforms the state-of-the-art on hyper-parameter tuning for data streams. [ABSTRACT FROM AUTHOR]

Change detection is of fundamental importance when analyzing data streams. Detecting changes both quickly and accurately enables monitoring and prediction systems to react, e.g., by issuing an alarm or by updating a learning algorithm. However, detecting changes is challenging when observations are high-dimensional. In high-dimensional data, change detectors should not only be able to identify when changes happen, but also in which subspace they occur. Ideally, one should also quantify how severe they are. Our approach, ABCD, has these properties. ABCD learns an encoder-decoder model and monitors its accuracy over a window of adaptive size. ABCD derives a change score based on Bernstein's inequality to detect deviations in terms of accuracy, which indicate changes. Our experiments demonstrate that ABCD outperforms its best competitor by up to 20% in F1-score on average. It can also accurately estimate changes' subspace, together with a severity measure that correlates with the ground truth. [ABSTRACT FROM AUTHOR]

Distilling actionable patterns from large-scale streaming data in the presence of concept drift is a challenging problem, especially when data is polluted with noisy labels. To date, various data stream mining algorithms have been proposed and extensively used in many real-world applications. Considering the functional complementation of classical online learning algorithms and with the goal of combining their advantages, we propose an Online Ensemble Classification (OEC) algorithm to integrate the predictions obtained by different base online classification algorithms. The proposed OEC method works by learning weights of different base classifiers dynamically through the classical Normalized Exponentiated Gradient (NEG) algorithm framework. As a result, the proposed OEC inherits the adaptability and flexibility of concept drift-tracking online classifiers, while maintaining the robustness of noise-resistant online classifiers. Theoretically, we show OEC algorithm is a low regret algorithm which makes it a good candidate to learn from noisy streaming data. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed OEC method. [ABSTRACT FROM AUTHOR]

Data streams are important sources of information nowadays, and with the popularization of mobile devices and sensor systems that collect all kinds of data, more and more information is generated at an ever increasing speed. This growth in data supply poses some problems for traditional machine learning algorithms. Tasks such as data classification, regression, or data clustering presents some limitations regarding very large datasets, data streams, or variations in data. The high cost of labeling instances for training classification algorithms makes it difficult to use fully supervised algorithms. Unbalanced datasets tend to cause algorithms to ignore one or more classes. Moreover, concept drifts in data streams require algorithms to be retrained from time to time. In order to tackle such problems mentioned, a semi-supervised and online algorithm based on Extreme Learning Machine (ELM) called SSOE-FP-ELM is proposed and detailed. Experimental results show that the proposed algorithm outperform others in the literature in accuracy, generalization ability and concept drift detection and recovery, showing suitable alternatives for data streams classification. [ABSTRACT FROM AUTHOR]

Concept drift is a difficult problem in the field of data stream learning, while the class imbalance problem existing in the data stream can seriously affect the classification performance of the algorithm. To address the joint problem of concept drift and class imbalance, an Incremental Weighted Ensemble for Imbalance Learning (IWEIL) for classifying unbalanced data streams is proposed by introducing an online update mechanism on the method based on the integration of data chunks, combined with the resampling and forgetting mechanism. The method is based on an integration framework that utilizes a variable-size window-based forgetting mechanism to determine the classification performance of base classifiers for a number of recent instances within the window, and calculates the weights of the base classifiers, updating each base classifier and its weight in IWEILonline every time a new instance is reached. An improved adaptive nearest-neighbor SMOTE method is also used to generate new minority class instances that conform to the new concept to solve the class imbalance problem in the data stream. The experimental results show that compared with the DWMIL algorithm, IWEIL improves the G-mean and Recall metrics on the synthesized HyperPlane dataset by 5.77% and 6.28%, respectively, and the two metrics on the real-world Electricity dataset by 3.25% and 6.47%,respectively. Finally, IWEIL has performed well in the Android app detection problem. [ABSTRACT FROM AUTHOR]

Machine Learning (ML) has been widely applied to cybersecurity and is considered state-of-the-art for solving many of the open issues in that field. However, it is very difficult to evaluate how good the produced solutions are, since the challenges faced in security may not appear in other areas. One of these challenges is the concept drift, which increases the existing arms race between attackers and defenders: malicious actors can always create novel threats to overcome the defense solutions, which may not consider them in some approaches. Due to this, it is essential to know how to properly build and evaluate an ML-based security solution. In this article, we identify, detail, and discuss the main challenges in the correct application of ML techniques to cybersecurity data. We evaluate how concept drift, evolution, delayed labels, and adversarial ML impact the existing solutions. Moreover, we address how issues related to data collection affect the quality of the results presented in the security literature, showing that new strategies are needed to improve current solutions. Finally, we present how existing solutions may fail under certain circumstances and propose mitigations to them, presenting a novel checklist to help the development of future ML solutions for cybersecurity. [ABSTRACT FROM AUTHOR]

Information systems age ungracefully. Once-modern systems aging into unmaintainable, buggy, meltdown-prone albatrosses is a widespread phenomenon that has received limited research attention. The received wisdom is that degenerative deterioration can be combated with refactoring or architectural improvements to their existing code. We conceptualize this phenomenon as system atrophy, and corroborate its existence by analyzing the code of over 1,300 systems as they underwent 19 million changes over 25 years. Such atrophy in systems has bread-and-butter consequences for organizations that rely on them. We show that it stunts the evolution of systems, makes them more bug-prone, and disengages developers. Atrophy in existing systems also makes it for organizations to implement other new systems because there are harder to integrate with them and cannibalize resources left over after their costlier upkeep. We then develop the idea that little increments in the modularity of their underlying code as a system evolves provide a powerful antidote to such atrophy. However, this antidote gradually loses its potency as a system ages further. Contrary to the popular belief, architectural improvements slow down atrophy but do not stop it. Our findings suggest that organizations must plan to eventually phase out these information systems, rather than just hoping to maintain them. For practice, we offer new insights on managing the tradeoff between evolution and atrophy; and how organizations can extract more useful life from aging systems. A pervasive, unbroached phenomenon is how once-modern systems age into unmaintainable albatrosses. We conceptualize this phenomenon from first-principles as system atrophy. We construct a trace data set from 190 million lines of evolving code in 1,354 systems spanning 25 years to corroborate it. Our middle-range theory introduces system atrophy into the conversation on information systems evolution, showing how small increments in modularity slow atrophy but lose potency with age. Atrophy eventually stunts systems, increases bugginess, and disengages developers. History: Yulin Fang, Senior Editor; Ning Su, Associate Editor. Funding: H. Safadi acknowledges support from the Terry-Sanford Research Award. Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2023.1218. [ABSTRACT FROM AUTHOR]

A wide variety of weather conditions, from windstorms to prolonged heat events, can substantially impact power systems, posing many risks and inconveniences due to power outages. Accurately estimating the probability distribution of the number of customers without power using data about the power utility system and environmental and weather conditions can help utilities restore power more quickly and efficiently. However, the critical shortcoming of current models lies in the difficulties of handling (i) data streams and (ii) model uncertainty due to combining data from various weather events. Accordingly, this article proposes an adaptive ensemble learning algorithm for data streams, which deploys a feature‐ and performance‐based weighting mechanism to adaptively combine outputs from multiple competitive base learners. As a proof of concept, we use a large, real data set of daily customer interruptions to develop the first adaptive all‐weather outage prediction model using data streams. We benchmark several approaches to demonstrate the advantage of our approach in offering more accurate probabilistic predictions. The results show that the proposed algorithm reduces the probabilistic predictions' error of the base learners between 4% and 22% with an average of 8%, which also result in substantially more accurate point predictions. The improvement made by our algorithm is enhanced as we exchange base learners with simpler models. [ABSTRACT FROM AUTHOR]

Most of the existing adaptive classification algorithms in non-stationary data streams require recent labelled data for their updates. Such recent labels are often missing. For stream classification under verification latency only few approaches exist. Most of them assume clustered data or homogeneous drift in all features, which limits their applicability. We address this by proposing Anticipative Bayesian stream Classifier (ABClass), an approach that is capable of integrating and automatically selecting from different components. In its Bayesian classification framework, ABClass combines density estimation techniques, extended to extrapolate drift patterns over time, with unsupervised parameter tuning and unsupervised model selection. ABClass allows for multivariate density estimation and extrapolation techniques. In this work, we assume conditional independence between features given the class label for modelling feature-specific drift patterns. ABClass is generative and can also be used for explaining and visualising concept drift patterns. It is generic, making it easy to include further types of drift models, both for the class-conditional feature distribution and for the class prior distribution. The experimental evaluation on several real-world data streams shows its competitiveness compared to other state-of-the-art approaches. ABClass is in most cases ten- to hundred-times faster than its competitors, both for model fitting and for prediction. [ABSTRACT FROM AUTHOR]

Machine Learning (ML) applications need large volumes of data to train their models so that they can make high‐quality predictions. Given digital revolution enablers such as the Internet of Things (IoT) and the Industry 4.0, this information is generated in large quantities in terms of continuous data streams and not in terms of static datasets as it is the case with most AI (Artificial Intelligence) frameworks. Kafka‐ML is a novel open‐source framework that allows the complete management of ML/AI pipelines through data streams. In this article, we present new features for the Kafka‐ML framework, such as the support for the well‐known ML/AI framework PyTorch, as well as for GPU acceleration at different points along the pipeline. This pipeline will be described by taking a real Industry 4.0 use case in the Petrochemical Industry. Finally, a comprehensive evaluation with state‐of‐the‐art deep learning models will be carried out to demonstrate the feasibility of the platform. [ABSTRACT FROM AUTHOR]

In IoT environment applications generate continuous non-stationary data streams with in-built problems of concept drift and class imbalance which cause classifier performance degradation. The imbalanced data affects the classifier during concept detection and concept adaptation. In general, for concept detection, a separate mechanism is added in parallel with the classifier to detect the concept drift called a drift detector. For concept adaptation, the classifier updates itself or trains a new classifier to replace the older one. In case, the data stream faces a class imbalance issue, the classifier may not properly adapt to the latest concept. In this survey, we study how the existing work addresses the issues of class imbalance and concept drift while learning from nonstationary data streams. We further highlight the limitation of existing work and challenges caused by other factors of class imbalance along with concept drift in data stream classification. Results of our survey found that, out of 1110 studies, by using our inclusion and exclusion criteria, we were able to narrow the pool of articles down to 35 that directly addressed our study objectives. The study found that issues such as multiple concept drift types, dynamic class imbalance ratio, and multi-class imbalance in presence of concept drift are still open for further research. We also observed that, while major research efforts have been dedicated to resolving concept drift and class imbalance, not much attention has been given to with-in-class imbalance, rear examples, and borderline instances when they exist with concept drift in multi-class data. This paper concludes with some suggested future directions. [ABSTRACT FROM AUTHOR]

Online active learning is a paradigm in machine learning that aims to select the most informative data points to label from a data stream. The problem of minimizing the cost associated with collecting labeled observations has gained a lot of attention in recent years, particularly in real-world applications where data is only available in an unlabeled form. Annotating each observation can be time-consuming and costly, making it difficult to obtain large amounts of labeled data. To overcome this issue, many active learning strategies have been proposed in the last decades, aiming to select the most informative observations for labeling in order to improve the performance of machine learning models. These approaches can be broadly divided into two categories: static pool-based and stream-based active learning. Pool-based active learning involves selecting a subset of observations from a closed pool of unlabeled data, and it has been the focus of many surveys and literature reviews. However, the growing availability of data streams has led to an increase in the number of approaches that focus on online active learning, which involves continuously selecting and labeling observations as they arrive in a stream. This work aims to provide an overview of the most recently proposed approaches for selecting the most informative observations from data streams in real time. We review the various techniques that have been proposed and discuss their strengths and limitations, as well as the challenges and opportunities that exist in this area of research. [ABSTRACT FROM AUTHOR]

The world surrounding us is subject to constant change. These changes, frequently described as concept drift, influence many industrial and technical processes. As they can lead to malfunctions and other anomalous behavior, which may be safety-critical in many scenarios, detecting and analyzing concept drift is crucial. In this study, we provide a literature review focusing on concept drift in unsupervised data streams. While many surveys focus on supervised data streams, so far, there is no work reviewing the unsupervised setting. However, this setting is of particular relevance for monitoring and anomaly detection which are directly applicable to many tasks and challenges in engineering. This survey provides a taxonomy of existing work on unsupervised drift detection. In addition to providing a comprehensive literature review, it offers precise mathematical definitions of the considered problems and contains standardized experiments on parametric artificial datasets allowing for a direct comparison of different detection strategies. Thus, the suitability of different schemes can be analyzed systematically, and guidelines for their usage in real-world scenarios can be provided.

Streaming data sequences arise from various areas in the era of big data, and it is challenging to explore efficient online models that adapt to them. To address the potential heterogeneity, we introduce a new online estimation procedure to analyze the constantly incoming streaming datasets. The underlying model structures are assumed to be the generalized linear models with dynamic regression coefficients. Our key idea lies in introducing a vector of unknown parameters to measure the differences between batch-specific regression coefficients from adjacent data blocks. This is followed by the usage of the adaptive lasso penalization methodology to accurately select nonzero components, which indicates the existence of dynamic coefficients. We provide detailed derivations to demonstrate how our proposed method not only fits within the online updating framework in which the old estimator is recursively replaced with a new one based solely on the current individual-level samples and historical summary statistics but also adaptively avoids undesirable estimation biases coming from the potential changes in model parameters of interest. Computational issues are also discussed in detail to facilitate implementation. Its practical performance is demonstrated through both extensive simulations and a real case study. In summary, we contribute to a novel online method that efficiently adapts to streaming data environment, addresses potential heterogeneity, and mitigates estimation biases from changes in coefficients. [ABSTRACT FROM AUTHOR]

In this article, we describe an imbalanced regression method for making predictions over imbalanced data streams. We present MORSTS (Multiple Output Regression for Streaming Time Series), an online ensemble regressors devoted to non‐stationary and imbalanced data streams. MORSTS relies on several multiple output regressor submodels, adopts a cost sensitive weighting technique for dealing with imbalanced datasets, and handles overfitting by means of the K‐fold cross validation. For assessment purposes, experiments have been conducted on known real datasets and compared with known base regression techniques. [ABSTRACT FROM AUTHOR]

High utility pattern has received a lot of research and attention because of their wide range of application scenarios. How to efficiently mine high utility patterns over data streams has become an important issue in the field of data mining. To solve the problem that the traditional utility list structure has too many join operations and the join operation is not efficient, which leads to the low spatio-temporal efficiency of the algorithm and the problem that the sliding window model repeatedly generates the same resultset, a new algorithm for high utility pattern mining over data streams is proposed, named HUPM_Stream. A location-indexed list structure, Ext-list, is designed to reduce the time complexity of the utility list join operation, and an improved remaining utility pruning strategy IRS is proposed to reduce the number of utility list join operations, and a hash table structure-based resultset maintenance strategy HRS is designed to effectively reduce the search space of the algorithm and avoid repeatedly generating the same resultset during the sliding process of the window. A large number of experimental results show that the proposed algorithm has better performance on dense datasets. [ABSTRACT FROM AUTHOR]