Search

Understanding the operation of complex assets such heavy-duty vehicles is essential for improving the efficiency, sustainability, and safety of future industry. Specifically, reducing energy consumption of transportation is crucially important for fleet operators, due to the impact it has on decreasing energy costs and lowering greenhouse gas emissions. Drivers have a high influence on fuel usage. However, reliably estimating driver performance is challenging. This is a key component of many eco-driving tools used to train drivers. Some key aspects of good, or efficient, drivers include being more aware of the surroundings, adapting to the road situations, and anticipating likely developments of the traffic conditions. With the development of IoT technologies and possibility of collecting high-precision and high-frequency data, even such vague concepts can be qualitatively measured, or at least approximated. In this paper, we demonstrate how the driver's degree of attention to the road can be automatically extracted from onboard sensor data. More specifically, our main contribution is introduction of a new metric, called attention horizon (AH); it can, fully automatically and based on readily-available IoT data, capture, differentiate, and evaluate a driver's behavior as the vehicle approaches a red traffic light. We suggest that our measure encapsulates complex concepts such as driver's "awareness" and "carefulness" in itself. This metric is extracted from the pedal positions in a 150 m trajectory just before stopping. We demonstrate that this metric is correlated with normalized fuel consumption rate (FCR) in the long term, making it a suitable tool for ranking and evaluating drivers. For example, over weekly periods we found a negative median correlation between AH and FCR with the absolute value of 0.156 ; while using monthly data, the value was 0.402 . [ABSTRACT FROM AUTHOR]

Ensemble learning methods combine multiple models to improve performance by exploiting their diversity. The success of these approaches relies heavily on the dissimilarity of the base models forming the ensemble. This diversity can be achieved in many ways, with well-known examples including bagging and boosting. It is the diversity of the models within an ensemble that allows the ensemble to correct the errors made by its members, and consequently leads to higher classification or regression performance. A mistake made by a base model can only be rectified if other members behave differently on that particular instance, and provide the aggregator with enough information to make an informed decision. On the contrary, lack of diversity not only lowers model performance, but also wastes computational resources. Nevertheless, in the current state of the art ensemble approaches, there is no guarantee on the level of diversity achieved, and no mechanism ensuring that each member will learn a different decision boundary from the others. In this paper, we propose a parallel orthogonal deep learning architecture in which diversity is enforced by design, through imposing an orthogonality constraint. Multiple deep neural networks are created, parallel to each other. At each parallel layer, the outputs of different base models are subject to Gram–Schmidt orthogonalization. We demonstrate that this approach leads to a high level of diversity from two perspectives. First, the models make different errors on different parts of feature space, and second, they exhibit different levels of uncertainty in their decisions. Experimental results confirm the benefits of the proposed method, compared to standard deep learning models and well-known ensemble methods, in terms of diversity and, as a result, classification performance. • We propose a novel deep learning ensemble method with the enforced diversity mechanism. • We introduce a new approach for analyzing ensemble diversity, taking into account model confidence. • We perform experimental evaluation and show improved classification performance. • We demonstrate an efficient way of building deep learning ensembles with end-to-end architecture. [ABSTRACT FROM AUTHOR]

Domain adaptation (DA) methods facilitate cross-domain learning by minimizing the marginal or conditional distribution shift between domains. However, the conditional distribution shift is not well addressed by existing DA techniques for the cross-domain regression learning task. In this paper, we propose Multi-Domain Adaptation for Regression under Conditional shift (DARC) method. DARC constructs a shared feature space such that linear regression on top of that space generalizes to all domains. In other words, DARC aligns different domains according to the task-related information encoded in the values of the dependent variable. It is achieved using a novel Pairwise Similarity Preserver (PSP) loss function. PSP incentivizes the differences between the outcomes of any two samples, regardless of their domain(s), to match the distance between these samples in the constructed space. We perform experiments in both two-domain and multi-domain settings. The two-domain setting is helpful, especially when one domain contains few available labeled samples and can benefit from adaptation to a domain with many labeled samples. The multi-domain setting allows several domains, each with limited data, to be adapted collectively; thus, multiple domains compensate for each other's lack of data. The results from all the experiments conducted both on synthetic and real-world datasets confirm the effectiveness of DARC. • We propose a new pairwise similarity preserver loss function for regression tasks. • We propose a new domain adaptation method for regression tasks. • Our proposed method adapts domains with any arbitrary shift, concept, or covariate. • Our proposed method is capable of adapting multiple domains. • The results prove our method's efficiency in solving real-world problems. [ABSTRACT FROM AUTHOR]

• We identify several pitfalls in the process of extracting and evaluating methods to extract hierarchies in the context of HMC. • We propose using a random hierarchy as a necessary benchmark to evaluate the relevance of a good hierarchy, in this way we can establish whether our extracted hierarchy is capturing an existing structure between classes or not. • We demonstrate how the relevance of the hierarchy quality depends on the complexity of the dataset and the complexity of the classifiers used. • We perform experimental evaluation of different approaches for extracting hierarchies to showcase the previous contributions. Using hierarchies of classes is one of the standard methods to solve multi-class classification problems. In the literature, selecting the right hierarchy is considered to play a key role in improving classification performance. Although different methods have been proposed, there is still a lack of understanding of what makes a hierarchy good and what makes a method to extract hierarchies perform better or worse. To this effect, we analyze and compare some of the most popular approaches to extracting hierarchies. We identify some common pitfalls that may lead practitioners to make misleading conclusions about their methods. To address some of these problems, we demonstrate that using random hierarchies is an appropriate benchmark to assess how the hierarchy's quality affects the classification performance. In particular, we show how the hierarchy's quality can become irrelevant depending on the experimental setup: when using powerful enough classifiers, the final performance is not affected by the quality of the hierarchy. We also show how comparing the effect of the hierarchies against non-hierarchical approaches might incorrectly indicate their superiority. Our results confirm that datasets with a high number of classes generally present complex structures in how these classes relate to each other. In these datasets, the right hierarchy can dramatically improve classification performance. [ABSTRACT FROM AUTHOR]

In the latent bandit problem, the learner has access to reward distributions and – for the non-stationary variant – transition models of the environment. The reward distributions are conditioned on the arm and unknown latent states. The goal is to use the reward history to identify the latent state, allowing for the optimal choice of arms in the future. The latent bandit setting lends itself to many practical applications, such as recommender and decision support systems, where rich data allows the offline estimation of environment models with online learning remaining a critical component. Previous solutions in this setting always choose the highest reward arm according to the agent's beliefs about the state, not explicitly considering the value of information-gathering arms. Such information-gathering arms do not necessarily provide the highest reward, thus may never be chosen by an agent that chooses the highest reward arms at all times. In this paper, we present a method for information-gathering in latent bandits. Given particular reward structures and transition matrices, we show that choosing the best arm given the agent's beliefs about the states incurs higher regret. Furthermore, we show that by choosing arms carefully, we obtain an improved estimation of the state distribution, and thus lower the cumulative regret through better arm choices in the future. Through theoretical analysis we show that the proposed method retains the sub-linear regret rate of previous methods while having much better problem dependent constants. We evaluate our method on both synthetic and real-world data sets, showing significant improvement in regret over state-of-the-art methods. • We investigate the use of information gathering in latent bandits. • We develop a method that uses information gathering to reduce state uncertainty, while retaining sub-linear regret. • We highlight environment conditions that benefit from information gathering. • Evaluation on synthetic and real-world data show significant improvement over state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Evolutionary Algorithms (EAs) are often challenging to apply in real-world settings since evolutionary computations involve a large number of evaluations of a typically expensive fitness function. For example, an evaluation could involve training a new machine learning model. An approximation (also known as meta-model or a surrogate) of the true function can be used in such applications to alleviate the computation cost. In this paper, we propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. We define "Approximation Usefulness" to capture the necessary conditions to ensure correctness of the EA computations when an approximation is used. Based on this definition, we propose a procedure to construct a lightweight qualitative meta-model by the active selection of data instances. We then use a meta-model to carry out the feature selection task. We apply this procedure to the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation) to create a Qualitative approXimations variant, CHC Q X. We show that CHC Q X converges faster to feature subset solutions of significantly higher accuracy (as compared to CHC), particularly for large datasets with over 100K instances. We also demonstrate the applicability of the thinking behind our approach more broadly to Swarm Intelligence (SI), another branch of the Evolutionary Computation (EC) paradigm with results of PSO Q X , a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available. 2 2 https://github.com/Ghaith81/Fast-Genetic-Algorithm-For-Feature-Selection. • A method to construct computationally efficient meta-models for feature selection. • A definition of the necessary conditions to ensure correctness of EA computation. • Our algorithm converges faster than a wrapper GA, particularly for large datasets. • The approach is useful independently of the used metaheuristic as shown with PSO. • We demonstrate the HUX crossover averaging effect when used in binary optimization. [ABSTRACT FROM AUTHOR]

Methods and results are presented for applying supervised machine learning techniques to the task of predicting the need for repairs of air compressors in commercial trucks and buses. Prediction models are derived from logged on-board data that are downloaded during workshop visits and have been collected over three years on a large number of vehicles. A number of issues are identified with the data sources, many of which originate from the fact that the data sources were not designed for data mining. Nevertheless, exploiting this available data is very important for the automotive industry as means to quickly introduce predictive maintenance solutions. It is shown on a large data set from heavy duty trucks in normal operation how this can be done and generate a profit. Random forest is used as the classifier algorithm, together with two methods for feature selection whose results are compared to a human expert. The machine learning based features outperform the human expert features, which supports the idea to use data mining to improve maintenance operations in this domain. [ABSTRACT FROM AUTHOR]

Machine Activity Recognition (MAR) can be used to monitor manufacturing processes and find bottlenecks and potential for improvement in production. Several interesting results on MAR techniques have been produced in the last decade, but mostly on construction equipment. Forklift trucks, which are ubiquitous and highly important industrial machines, have been missing from the MAR research. This paper presents a data-driven method for forklift activity recognition that uses Controller Area Network (CAN) signals and semi-supervised learning (SSL). The SSL enables the utilization of large quantities of unlabeled operation data to build better classifiers; after a two-step post-processing, the recognition results achieve balanced accuracy of 88% for driving activities and 95% for load-handling activities on a hold-out data set. In terms of the Matthews correlation coefficient for five activity classes, the final score is 0.82, which is equal to the recognition results of two non-domain experts who use videos of the activities. A particular success is that context can be used to capture the transport of small weight loads that are not detected by the forklift's built-in weight sensor. [ABSTRACT FROM AUTHOR]

• A feature-based transfer learning approach for remaining useful life prediction. • Focus on generalizing models from simple source domains to complex target domains. • Propose distance-to-peers as a feature that is transferable across different domains. • The method outperforms TCA, CORAL, SCL, and a non-transfer approach on CMAPSS data. The traditional paradigm for developing machine prognostics usually relies on generalization from data acquired in experiments under controlled conditions prior to deployment of the equipment. Detecting or predicting failures and estimating machine health in this way assumes that future field data will have a very similar distribution to the experiment data. However, many complex machines operate under dynamic environmental conditions and are used in many different ways. This makes collecting comprehensive data very challenging, and the assumption that pre-deployment data and post-deployment data follow very similar distributions is unlikely to hold. In this work, we present a feature-representation based transfer learning (TL) method for predicting Remaining Useful Life (RUL) of equipment, under scenarios that samples with previously unseen conditions are presented in the target domain and the labels are available only for the source domain, but not the target domain. This setting corresponds to generalizing from a limited number of run-to-failure experiments performed prior to deployment into making prognostics with data coming from deployed equipment that is being used under multiple new operating conditions and experiencing previously unseen faults. We employ a deviation detection method, Consensus Self-Organizing Models (COSMO), to create transferable features for building the RUL regression model. These features capture how different a particular equipment is in comparison to its peers. The efficiency of the proposed TL method is demonstrated using the NASA Turbofan Engine Degradation Simulation Data Set. Models using the COSMO transferable features show better performance than other methods on predicting RUL when the target domain is more complex than the source domain. [ABSTRACT FROM AUTHOR]

• We postulate the problem of unifying streaming anomaly detection. • We propose a meta-framework for a flexible and adaptable anomaly detection procedure. • Our framework helps to overcome the limitations of one-size-fitsall solutions. • We propose a novel anomaly-aware reservoir sampling scheme. • We conduct an extensive comparison study on 20 detectors using various datasets. In recent years, research interest in detecting anomalies in temporal streaming data has increased significantly. A variety of algorithms are being developed in the data mining community. They can be broadly divided into two categories, namely general-purpose and ad hoc ones. In most cases, general approaches assume a one-size-fits-all solution model, and strive to design a single "optimal" anomaly detector which can detect all anomalies in any domain. To date, there exists no universal method that has been shown to outperform the others across different anomaly types, use cases and datasets. In this paper, we propose SAFARI, a framework created by abstracting and unifying the fundamental tasks within the streaming anomaly detection. SAFARI provides a flexible and extensible anomaly detection procedure to overcome the limitations of one-size-fits-all solutions. Such abstraction helps to facilitate more elaborate algorithm comparisons by allowing us to isolate the effects of shared and unique characteristics of diverse algorithms on the performance. Using the framework, we have identified a research gap that motivated us to propose a novel learning strategy. We implemented twenty different anomaly detectors and conducted an extensive evaluation study, comparing their performances using real-world benchmark datasets with different properties. The results indicate that there is no single superior detector which works perfectly for every case, proving our hypothesis that "there is no free lunch" in the streaming anomaly detection world. Finally, we discuss the benefits and drawbacks of each method in-depth, drawing a set of conclusions and guidelines to guide future users of SAFARI. [ABSTRACT FROM AUTHOR]

• A data-driven approach is proposed to discover heat load patterns in district heating. • The first large-scale analysis of all the buildings in six different categories is presented. • We showcase how typical and atypical behaviors look like in the entire network in Sweden. • The results show that our method has a high potential to be deployed and used in practice. Understanding the heat usage of customers is crucial for effective district heating operations and management. Unfortunately, existing knowledge about customers and their heat load behaviors is quite scarce. Most previous studies are limited to small-scale analyses that are not representative enough to understand the behavior of the overall network. In this work, we propose a data-driven approach that enables large-scale automatic analysis of heat load patterns in district heating networks without requiring prior knowledge. Our method clusters the customer profiles into different groups, extracts their representative patterns, and detects unusual customers whose profiles deviate significantly from the rest of their group. Using our approach, we present the first large-scale, comprehensive analysis of the heat load patterns by conducting a case study on many buildings in six different customer categories connected to two district heating networks in the south of Sweden. The 1222 buildings had a total floor space of 3.4 million square meters and used 1540 TJ heat during 2016. The results show that the proposed method has a high potential to be deployed and used in practice to analyze and understand customers' heat-use habits. [ABSTRACT FROM AUTHOR]

Highlights • To extract event information by exploiting historical data such as cable asset inventory. • To estimate the failure rate of every individual cables using PHM. • To demonstrate that, although many previous works make different approximations which are designed for non-repairable systems, the results of applying these approaches are misleading for repairable power cables. Abstract In this paper Weibull parametric proportional hazard model (PHM) is used to estimate the failure rate of every individual cable based on its age and a set of explanatory factors. The required information for the proposed method is obtained by exploiting available historical cable inventory and failure data. This data-driven method does not require any additional measurements on the cables, and allows the cables to be ranked for maintenance prioritization and repair actions. Furthermore, the results of reliability analysis of power cables are compared when the cables are considered as repairable or non-repairable components. The paper demonstrates that the methods which estimate the time-to-the-first failure (for non-repairable components) lead to incorrect conclusions about reliability of repairable power cables. The proposed method is used to evaluate the failure rate of each individual Paper Insulated Lead Cover (PILC) underground cables in a distribution grid in the south of Sweden. [ABSTRACT FROM AUTHOR]

Research on machine activity recognition (MAR) is drawing more attention because MAR can provide productivity monitoring for efficiency optimization, better maintenance scheduling, product design improvement, and potential material savings. A particular challenge of MAR for human-operated machines is the overlap when transiting from one activity to another: during transitions, operators often perform two activities simultaneously, e.g., lifting the fork already while approaching a rack, so the exact time when one activity ends and another begins is uncertain. Machine learning models are often uncertain during such activity transitions, and we propose a novel ensemble-based method adapted to fuzzy transitions in a forklift MAR problem. Unlike traditional ensembles, where models in the ensemble are trained on different subsets of data, or with costs that force them to be diverse in their responses, our approach is to train a single model that predicts several activity labels, each under a different context. These individual predictions are not made by independent networks but are made using a structure that allows for sharing important features, i.e., a context ensemble. The results show that the gated recurrent unit network can provide medium or strong confident context ensembles for 95% of the cases in the test set, and the final forklift MAR result achieves accuracies of 97% for driving and 90% for load-handling activities. This study is the first to highlight the overlapping activity issue in MAR problems and to demonstrate that the recognition results can be significantly improved by designing a machine learning framework that addresses this issue. [ABSTRACT FROM AUTHOR]

In this survey, 105 papers related to interactive clustering were reviewed according to seven perspectives: (1) on what level is the interaction happening, (2) which interactive operations are involved, (3) how user feedback is incorporated, (4) how interactive clustering is evaluated, (5) which data and (6) which clustering methods have been used, and (7) what outlined challenges there are. This article serves as a comprehensive overview of the field and outlines the state of the art within the area as well as identifies challenges and future research needs. [ABSTRACT FROM AUTHOR]

As Deep Neural Networks (DNN) make their way into safety-critical decision processes, it becomes imperative to have robust and reliable uncertainty estimates for their predictions for both in-distribution and out-of-distribution (OOD) examples. This is particularly important in real-life high-risk settings such as healthcare, where OOD examples (e.g., patients with previously unseen or rare labels, i.e., diagnoses) are frequent, and an incorrect clinical decision might put human life in danger, in addition to having severe ethical and financial costs. While evidential uncertainty estimates for deep learning have been studied for multi-class problems, research in multi-label settings remains untapped. In this paper, we propose a DEep Evidential Doctor (DEED), which is a novel deterministic approach to estimate multi-label targets along with uncertainty. We achieve this by placing evidential priors over the original likelihood functions and directly estimating the parameters of the evidential distribution using a novel loss function. Additionally, we build a redundancy layer (particularly for high uncertainty and OOD examples) to minimize the risk associated with erroneous decisions based on dubious predictions. We achieve this by learning the mapping between the evidential space and a continuous semantic label embedding space via a recurrent decoder. Thereby inferring, even in the case of OOD examples, reasonably close predictions to avoid catastrophic consequences. We demonstrate the effectiveness of DEED on a digit classification task based on a modified multi-label MNIST dataset, and further evaluate it on a diagnosis prediction task from a real-life electronic health record dataset. We highlight that in terms of prediction scores, our approach is on par with the existing state-of-the-art having a clear advantage of generating reliable, memory and time-efficient uncertainty estimates with minimal changes to any multi-label DNN classifier. [ABSTRACT FROM AUTHOR]

Most existing work in information fusion focuses on combining information with well-defined meaning towards a concrete, pre-specified goal. In contradistinction, we instead aim for autonomous discovery of high-level knowledge from ubiquitous data streams. This paper introduces a method for recognition and tracking of hidden conceptual modes, which are essential to fully understand the operation of complex environments, and an important step towards building truly intelligent aware systems. We consider a scenario of analyzing usage of a fleet of city buses, where the objective is to automatically discover and track modes such as highway route, heavy traffic, or aggressive driver, based on available on-board signals. The method we propose is based on aggregating the data over time, since the high-level modes are only apparent in the longer perspective. We search through different features and subsets of the data, and identify those that lead to good clusterings, interpreting those clusters as initial, rough models of the prospective modes. We utilize Bayesian tracking in order to continuously improve the parameters of those models, based on the new data, while at the same time following how the modes evolve over time. Experiments with artificial data of varying degrees of complexity, as well as on real-world datasets, prove the effectiveness of the proposed method in accurately discovering the modes and in identifying which one best explains the current observations from multiple data streams. [ABSTRACT FROM AUTHOR]

Many industries today are struggling with early the identification of quality issues, given the shortening of product design cycles and the desire to decrease production costs, coupled with the customer requirement for high uptime. The vehicle industry is no exception, as breakdowns often lead to on-road stops and delays in delivery missions. In this paper we consider quality issues to be an unexpected increase in failure rates of a particular component; those are particularly problematic for the original equipment manufacturers (OEMs) since they lead to unplanned costs and can significantly affect brand value. We propose a new approach towards the early detection of quality issues using machine learning (ML) to forecast the failures of a given component across the large population of units. In this study, we combine the usage information of vehicles with the records of their failures. The former is continuously collected, as the usage statistics are transmitted over telematics connections. The latter is based on invoice and warranty information collected in the workshops. We compare two different ML approaches: the first is an auto-regression model of the failure ratios for vehicles based on past information, while the second is the aggregation of individual vehicle failure predictions based on their individual usage. We present experimental evaluations on the real data captured from heavy-duty trucks demonstrating how these two formulations have complementary strengths and weaknesses; in particular, they can outperform each other given different volumes of the data. The classification approach surpasses the regressor model whenever enough data is available, i.e., once the vehicles are in-service for a longer time. On the other hand, the regression shows better predictive performance with a smaller amount of data, i.e., for vehicles that have been deployed recently. [ABSTRACT FROM AUTHOR]

The maximum current that an overhead transmission line can continuously carry depends on external weather conditions, most commonly obtained from real-time streaming weather sensors. The accuracy of the sensor data is very important in order to avoid problems such as overheating. Furthermore, faulty sensor readings may cause operators to limit or even stop the energy production from renewable sources in radial networks. This paper presents a method for detecting and replacing sequences of consecutive faulty data originating from streaming weather sensors. The method is based on a combination of (a) a set of constraints obtained from derivatives in consecutive data, and (b) association rules that are automatically generated from historical data. In smart grids, a large amount of historical data from different weather stations are available but rarely used. In this work, we show that mining and analyzing this historical data provides valuable information that can be used for detecting and replacing faulty sensor readings. We compare the result of the proposed method against the exponentially weighted moving average and vector autoregression models. Experiments on data sets with real and synthetic errors demonstrate the good performance of the proposed method for monitoring weather sensors. [ABSTRACT FROM AUTHOR]