Search

The overwhelming increase of parcel transports has prompted the need for effective and scalable intelligent logistics systems. In parallel, with the advent of Industry 4.0, a tight integration of Internet of Things technologies and Big Data analytics solution has become necessary to effectively manage industrial processes and to early predict product faults or service disruptions. In the context of good transports, the development of smart monitoring tools is particularly useful for couriers to ensure effective and efficient parcel deliveries. However, the existing predictive maintenance frameworks are not tailored to parcel delivery services. We present REDTag Service, an integrated framework to track and monitor the shipped packages. It relies on a network of IoT-enabled devices, called REDTags, allowing courier employees to easily collect the status of the package at each delivery step. The framework provides back-end functionalities for smart data transmission, management, storage, and analytics. A machine-learning process is included to promptly analyze the features describing event-related data to predict potential breaks of the goods in the packages. The framework provides also a dynamic view on the integrated data tailored to the different stakeholders, as well as on the prediction outcomes, enabling immediate feedback and model improvements. We analyze a real-world dataset including event-related data about parcel transports. To validate the hypothesis that the acquired data contains information relevant to predict the package status (i.e., broken or safe), we empirically analyze the performance of different, scalable classifiers. The experimental results confirm, in good approximation, the predictive power of the models extracted from the event-related features. To the best of the authors' knowledge, this work is the first attempt to address predictive maintenance in smart good transport logistics to predict package breaks from real-world data.

In the last few years, manufacturing systems are getting gradually transformed into smart factories. In this context, an increasing number of information and communication technologies is incorporated towards facilitating management, production, and control processes. The introduction of advanced embedded systems with enhanced connectivity produces a vast amount of data, posing a challenge in terms of data analytics. However, the in-time collection and analysis of acquired data can create insight into the manufacturing process as well as its assets. One aspect of major importance for every production system is preserving its equipment in operational condition, and within those limits that could minimize unplanned breakdowns and production stoppages. This paper details the predictive analytics methodology integrated into the SERENA platform able to: (i) streamline the prognostics of the industrial components, (ii) characterize the health status of the monitored equipment, (iii) generate an early warning related to the condition of the equipment, and (iv) forecast the future evolution of the monitored equipment’s degradation. To demonstrate the effectiveness of the proposed methodology, different use cases are discussed with results obtained on real-data collected in real-time from the industrial environments.

The increasing capability to collect data gives us the possibility to collect a massive amount of heterogeneous data. Among the heterogeneous data available, time-series represents a mother lode of information yet to be fully explored. Current data mining techniques have several shortcomings while analyzing time-series, especially when more than one time-series, i.e., multi-dimensional timeseries, should be analyzed together to extract knowledge from the data. In this context, we present K-MDTSC (K-Multi-Dimensional Time-Series Clustering), a novel clustering algorithm specifically designed to deal with multi-dimensional time-series. Firstly, we demonstrate K-MDTSC capability to group multi-dimensional time-series using synthetic datasets. We compare K-MDTSC results with k-Shape, a state-of-art time-series clustering algorithm based on K-means. Our results show both K-MDTSC and k-Shape create good clustering results. However, K-MDTSC outperforms k-Shape when complicating the synthetic dataset. Secondly, we apply K-MDTSC in a real case scenario where we are asked to replace a scheduled maintenance with a predictive approach. To this end, we create a generalized pipeline to process data from a real industrial plant welding process. We apply K-MDTSC to create clusters of weldings based on their welding shape. Our results show that K-MDTSC identifies different welding profiles, but that the aging of the electrode does not negatively impact the welding process.

Nowadays, cars are instrumented with thousands of sensors continuously collecting data about its components. Thanks to the concept of connected cars, this data can be now transferred to the cloud for advanced analytics functionalities, such as prognostic or predictive maintenance. In this paper, we dissect a data-driven prognostic pipeline and apply it in the automotive scenario. Our pipeline is composed of three main steps: (i) selection of most important signals and features describing the scenario for the target problem, (ii) creation of machine learning models based on different classification algorithms, and (iii) selection of the model that works better for a deployment scenario. For the development of the pipeline, we exploit an extensive experimental campaign where an actual engine runs in a controlled test bench under different working conditions. We aim to predict failures of the High-Pressure Fuel System, a key part of the diesel engine responsible for delivering high-pressure fuel to the cylinders for combustion. Our results show the advantage of data-driven solutions to automatically discover the most important signals to predict failures of the High-Pressure Fuel System. We also highlight how an accurate model selection step is fundamental to identify a robust model suitable for deployment.

In the last few years, cities have become engines of wealth creation thanks to the advent of the new information and communication [...]

In this paper, we present CAROL (unConventionAl suRvey methOdoLogy) to collect users’ opinions, to analyze them effectively, and to graphically provide interesting summaries in near-real-time. CAROL has been designed to collect citizens’ opinions, needs, and a perception that can be very useful to redesign cities and drive the innovation process. To this aim, particular importance is given to the architecture implemented to collect the data, which involves sending voice messages instead of text, allowing users greater freedom of expression. Moreover, to make the survey more friendly, an initial analysis is done in real-time thanks to an intuitive dashboard, which shows the first results in the form of graphs and charts, updated after each message. Finally, a more in-depth analysis can be carried out at the end of the survey, thanks to text-mining techniques adapted to the context. As use case, the proposed methodology has been tested in a public event organized by Politecnico di Torino and addressed to female students, with the aim to collect their ideas about new services/opportunities that can be offered to support women in STEM.

International audience

Recent trends in intelligent manufacturing are transforming shop floor environments into digital factories, thanks to a pervasive integration of information and communication technologies in production lines. Industrial processes become the source of high-volume heterogeneous data, paving the way to create manufacturing intelligence by means of machine learning and data-driven methodologies. In such settings, predictive diagnostics play a crucial role, as they promise to predict future critical conditions in the production process. Unfortunately, the diffusion of data-driven predictive maintenance methodologies is limited by (i) the absence of timely ground-truth knowledge (i.e., class labels), required in the learning phase of data-driven supervised approaches, and (ii) the limited availability of data-mining expertise among application-domain experts, required to harness the power of machine learning techniques. Innovative data-driven services are needed to support domain experts in (i) applying powerful self-learning intelligent techniques with limited technical expertise and (ii) easily understanding results and choices operated by such intelligent techniques, to increase trust by means of transparency. To this aim, this paper presents UDaMP, an integrated platform to support manufacturing intelligence by providing a transparent, self-tuning, unsupervised discovery and assisted data labelling service for predictive maintenance, specifically targeted at cyclic industrial processes. UDaMP includes (i) production-cycle-aware feature engineering, (ii) unsupervised discovery of production-cycle categories, (iii) self-tuning of the optimal number of categories, (iv) human-readable characterisation of production-cycle categories, and (v) assisted data labelling for domain experts. Scalable clustering algorithms automatically discover groups of production cycles sharing common time-independent properties. A self-tuning strategy is integrated to automatically configure the specific input parameter and select the best approach for the data under analysis. Each cluster is then locally characterised through the data distribution of the top 10 most relevant features to support domain experts in uncovering its meaning. Experimental evaluation of UDaMP has been performed on real-world data collected in two different industrial settings.

In today’s world, social networks and online communities continuously generate tons of data that reflect users’ habits, personal interests, opinions and emotions. However, little profit can be gained from such huge raw data collections unless we are able to translate them into useful knowledge. Microblogs like Twitter have recently attracted a great body of research works to mine useful insights about users interests and preferences in different geographical areas and time periods. Indeed, the rather heterogeneous dimensions characterizing Twitter data, such as space, time and text content, impose innovative methods in the data mining discovery process. This paper presents TCharM , a data analytics methodology based on cluster analysis and association rule discovery to gain interesting knowledge from large collections of Twitter data. TCharM explores tweet collections along the three dimensions characterizing tweets (i.e., text content, posting time and place) to support context-aware topic trend analysis. To discover groups of tweets with a good cohesion on the three tweet features, TCharM exploits a novel distance measure ( TASTE ) which allows driving the clustering task by considering in one step the three tweet features. Association rule analysis is then exploited to concisely describe the cluster content with a set of understandable and significant patterns which reveal underlying correlations among frequent topics, tweeting times and places. TCharM can provide useful information to understand the evolution of people’s involvement in different topics, across geographical areas and over time. TCharM find applications in various domains by providing a valuable support in decision making to domain experts. The experimental evaluation performed on real datasets demonstrates the effectiveness of the proposed approach in discovering cohesive clusters and actionable knowledge from Twitter data.

In the era of “connected vehicles,” i.e., vehicles that generate long data streams during their usage through the telematics on-board device, data-driven methodologies assume a crucial role in creating valuable insights to support the decision-making process effectively. Predictive analytics allows anticipation of vehicle issues and optimized maintenance, reducing the resulting costs. In this paper, we focus on analyzing data collected from heavy trucks during their use, a relevant task for companies due to the high commercial value of the monitored vehicle. The proposed methodology, named TETRAPAC, offers a generalizable approach to estimate vehicle health conditions based on monitored features enriched by innovative key performance indicators. We discussed performance of TETRAPAC in two real-life settings related to trucks. The obtained results in both tasks are promising and able to support the company’s decision-making process in the planning of maintenance interventions.

Intraday traders buy and sell financial instruments in the short term, typically within the same trading day. Stocks are notable examples of financial instruments. However, since hundreds of stocks are listed on the stock exchange selecting on each trading day the most tradeable stocks is a challenging task, which is commonly addressed through manual inspection of historical stock prices and technical indicators. This paper aims at discovering tradeable stocks on a given trading day by analyzing the historical prices assumed by the same stocks or by other ones on the preceding days by means of regression and weighted sequence mining techniques. The use of regression and weighted sequence mining techniques allows traders to automatically consider a potentially large number of candidate stocks and to effectively analyze their price variations across consecutive days. The experimental results, which were achieved on data acquired from different markets and under different market conditions, show that sequence mining algorithms yield profits higher than both regression techniques and naive strategies.

Itemset mining is a well-known exploratory data mining technique used to discover interesting correlations hidden in a data collection. Since it supports different targeted analyses, it is profitably exploited in a wide range of different domains, ranging from network traffic data to medical records. With the increasing amount of generated data, different scalable algorithms have been developed, exploiting the advantages of distributed computing frameworks, such as Apache Hadoop and Spark. This paper reviews Hadoop- and Spark-based scalable algorithms addressing the frequent itemset mining problem in the Big Data domain through both theoretical and experimental comparative analyses. Since the itemset mining task is computationally expensive, its distribution and parallelization strategies heavily affect memory usage, load balancing, and communication costs. A detailed discussion of the algorithmic choices of the distributed methods for frequent itemset mining is followed by an experimental analysis comparing the performance of state-of-the-art distributed implementations on both synthetic and real datasets. The strengths and weaknesses of the algorithms are thoroughly discussed with respect to the dataset features (e.g., data distribution, average transaction length, number of records), and specific parameter settings. Finally, based on theoretical and experimental analyses, open research directions for the parallelization of the itemset mining problem are presented.

Automating predictive machine learning entails the capability of properly triggering the update of the trained models. To this aim, the degradation of predictive models has to be continuously evaluated over time to detect data distribution drifts between the original training set and the new data. Traditionally, prediction performance is used as a degradation metric. However, prediction quality indices require ground-truth class labels to be known for the newly classified data, making them unsuitable for real-time applications, as ground-truth labels might be totally absent or be available only later.In this paper, we propose a novel unsupervised methodology to automatically detect prediction-quality degradation of machine learning models. Thanks to the unsupervised approach and a novel scalable estimation technique, we provide an effective and efficient solution to the above-mentioned problem with soft real-time constraints. Specifically, our approach is able to detect class-based concept drift, i.e., when new data contain samples that do not fit the set of class labels known by the currently-trained predictive model. Experiments on synthetic and real-world public datasets show the effectiveness of the proposed methodology in automatically detecting and describing concept drift caused by changes in the class-label data distributions. Thanks to its scalability performance, the proposed approach is suitable for soft real-time applications such as predictive maintenance, Industry 4.0, and text mining.

In recent years, the number of industry-4.0-enabled manufacturing sites has been continuously growing, and both the quantity and variety of signals and data collected in plants are increasing at an unprecedented rate. At the same time, the demand of Big Data processing platforms and analytical tools tailored to manufacturing environments has become more and more prominent. Manufacturing companies are collecting huge amounts of information during the production process through a plethora of sensors and networks. To extract value and actionable knowledge from such precious repositories, suitable data-driven approaches are required. They are expected to improve the production processes by reducing maintenance costs, reliably predicting equipment failures, and avoiding quality degradation. To this aim, Machine Learning techniques tailored for predictive maintenance analysis have been adopted in PREMISES (PREdictive Maintenance service for Industrial procesSES), an innovative framework providing a scalable Big Data service able to predict alarming conditions in slowly-degrading processes characterized by cyclic procedures. PREMISES has been experimentally tested and validated on a real industrial use case, resulting efficient and effective in predicting alarms. The framework has been designed to address the main Big Data and industrial requirements, by being developed on a solid and scalable processing framework, Apache Spark, and supporting the deployment on modularized containers, specifically upon the Docker technology stack.

The pervasive and increasing deployment of smart meters allows collecting a huge amount of fine-grained energy data in different urban scenarios. The analysis of such data is challenging and opening up a variety of interesting and new research issues across energy and computer science research areas. The key role of computer scientists is providing energy researchers and practitioners with cutting-edge and scalable analytics engines to effectively support their daily research activities, hence fostering and leveraging data-driven approaches. This paper presents SPEC, a scalable and distributed engine to predict building-specific power consumption. SPEC addresses the full analytic stack and exploits a data stream approach over sliding time windows to train a prediction model tailored to each building. The model allows us to predict the upcoming power consumption at a time instant in the near future. SPEC integrates different machine learning approaches, specifically ridge regression, artificial neural networks, and random forest regression, to predict fine-grained values of power consumption, and a classification model, the random forest classifier, to forecast a coarse consumption level. SPEC exploits state-of-the-art distributed computing frameworks to address the big data challenges in harvesting energy data: the current implementation runs on Apache Spark, the most widespread high-performance data-processing platform, and can natively scale to huge datasets. As a case study, SPEC has been tested on real data of an heating distribution network and power consumption data collected in a major Italian city. Experimental results demonstrate the effectiveness of SPEC to forecast both fine-grained values and coarse levels of power consumption of buildings.

Energy performance certification is an important tool for the assessment and improvement of energy efficiency in buildings. In this context, estimating building energy demand also in a quick and reliable way, for different combinations of building features, is a key issue for architects and engineers who wish, for example, to benchmark the performance of a stock of buildings or optimise a refurbishment strategy. This paper proposes a methodology for (i) the automatic estimation of the building Primary Energy Demand for space heating ( P E D h ) and (ii) the characterization of the relationship between the P E D h value and the main building features reported by Energy Performance Certificates (EPCs). The proposed methodology relies on a two-layer approach and was developed on a database of almost 90,000 EPCs of flats in the Piedmont region of Italy. First, the classification layer estimates the segment of energy demand for a flat. Then, the regression layer estimates the P E D h value for the same flat. A different regression model is built for each segment of energy demand. Four different machine learning algorithms (Decision Tree, Support Vector Machine, Random Forest, Artificial Neural Network) are used and compared in both layers. Compared to the current state-of-the-art, this paper brings a contribution in the use of data mining techniques for the asset rating of building performance, introducing a novel approach based on the use of independent data-driven models. Such configuration makes the methodology flexible and adaptable to different EPCs datasets. Experimental results demonstrate that the proposed methodology can estimate the energy demand with reasonable errors, using a small set of building features. Moreover, the use of Decision Tree algorithm enables a concise interpretation of the quantitative rules used for the estimation of the energy demand. The methodology can be useful during both designing and refurbishment of buildings, to quickly estimate the expected building energy demand and set credible targets for improving performance.

Nowadays, an exponential growth in biological data has been recorded, including both structured and unstructured data. One of the main computational and scientific challenges in the modern age is to extract useful information from unstructured textual corpora to effectively support the decision making process. Since the emergence of topic modelling, new and interesting approaches to compactly represent the content of a document collection have been proposed. However, the effective exploitation of the proposed strategies requires a lot of expertise.

Technological advances in areas such as communications, computer processing, connectivity, data management are gradually introducing the internet of things (IoT) paradigm across companies of different domain. In this context and as systems are making a shift into cyber-physical system of systems, connected devices provide massive data, that are usually streamed to a central node for further processing. In particular and related to the manufacturing domain, Data processing can provide insight in the operational condition of the organization or process monitored. However, there are near real time constraints for such insights to be generated and data-driven decision making to be enabled. In the context of internet of things for smart manufacturing and empowered by the aforementioned, this study discusses a fog computing paradigm for enabling maintenance related predictive analytic in a manufacturing environment through a two step approach: (1) Model training on the cloud, (2) Model execution on the edge. The proposed approach has been applied to a use case coming from the robotic industry.

The energy performance certificate (EPC) is a document that certifies the average annual energy consumption of a building in standard conditions and allows it to be classified within a so-called energy class. In a period such as this, when greenhouse gas emissions are of considerable importance and where the objective is to improve energy security and reduce energy costs in our cities, energy certification has a key role to play. The proposed work aims to model and characterize residential buildings&rsquo, energy efficiency by exploring heterogeneous, geo-referenced data with different spatial and temporal granularity. The paper presents TUCANA (TUrin Certificates ANAlysis), an innovative data mining engine able to cover the whole analytics workflow for the analysis of the energy performance certificates, including cluster analysis and a model generalization step based on a novel spatial constrained K-NN, able to automatically characterize a broad set of buildings distributed across a major city and predict different energy-related features for new unseen buildings. The energy certificates analyzed in this work have been issued by the Piedmont Region (a northwest region of Italy) through open data. The results obtained on a large dataset are displayed in novel, dynamic, and interactive geospatial maps that can be consulted on a web application integrated into the system. The visualization tool provides transparent and human-readable knowledge to various stakeholders, thus supporting the decision-making process.

Several cities have built on-the-ground air quality monitoring stations to measure daily concentration of air pollutants, like \(\textit{PM}_{10}\) and \(\textit{NO}_{2}\). The identification of the causalities for air pollution will help governments’ decision-making on mitigating air pollution and on prioritizing recommendations. This paper presents a two-level methodology based on unsupervised analytics methods, named PANDA, to discover interesting insights from air quality-related data. First, PANDA discovers groups of pollutants that have occurred with similar concentrations. Then, each cluster is locally characterized through three forms of human-readable knowledge to provide interesting correlations between air pollution and meteorological conditions at different abstraction level. As a case study, PANDA has been validated on real pollutant measurements collected in a major Italian city. Preliminary experimental results show that PANDA is effective in discovering cohesive and well-separated groups of similar concentrations of pollutants along with different forms of interpretable correlations among air pollution and weather data.

Computer Aided Diagnosis (CAD) systems are designed to aid the radiologist in interpreting medical images. They are usually based on lesion detection and segmentation algorithms whose performance depends on a large number of parameters. While time consuming and sub-optimal, parameter values are most often selected through manual search strategies. Genetic or evolutionary algorithms (GA) are effective optimization methods that mimic biological evolution. Genetic algorithms have been shown to efficiently manage complex search spaces, and can be applied to all kinds of objective functions, including discontinuous, nondifferentiable, or highly nonlinear ones. In this study, we have adopted an evolutionary approach to the problem of parameter optimization. We show that the genetic algorithm is able to effectively converge to a better solution than manual optimization on a case study for digital breast tomosynthesis CAD. Parameter optimization was framed as a constrained optimization problem, where the function to be maximized was defined as the weighted sum of sensitivity, false positive rate and segmentation accuracy. A modified Dice coefficient was defined to assess the segmentation quality of individual lesions. Finally, all viable solutions evaluated by the GA were studied by means of exploratory data analysis techniques, such as association rules, to gain useful insights on the strength of the influence of each parameter on overall algorithm performance. We showed that this combination was able to identify multiple ranges of viable solutions with good segmentation accuracy.

TToPIC (Tuning of Parameters for Inference of Concepts) is a distributed self-tuning engine whose aim is to cluster collections of textual data into correlated groups of documents through a topic modeling methodology (i.e., LDA). ToPIC includes automatic strategies to relieve the end-user of the burden of selecting proper values for the overall analytics process. ToPIC's current implementation runs on Apache Spark, a state-of-the-art distributed computing framework. As a case study, ToPIC has been validated on three real collections of textual documents characterized by different distributions. The experimental results show the effectiveness and efficiency of the proposed solution in analyzing collections of documents without tuning algorithm parameters and in discovering cohesive and well-separated groups of documents with a similar topic.

The recent expansion of IoT-enabled (Internet of Things) devices in manufacturing contexts and their subsequent data-driven exploitation paved the way to the advent of the Industry 4.0, promoting a full integration of IT services, smart devices, and control systems with physical objects, their electronics and sensors. The real-time transmission and analysis of collected data from factories has the potential to create manufacturing intelligence, of which predictive maintenance is an expression. Hence the need to design new approaches able to manage not only the data volume, but also the variety and velocity, extracting actual value from the humongous amounts of collected data. To this aim, we present iSTEP, an integrated Self-Tuning Engine for Predictive maintenance, based on Big Data technologies and designed for Industry 4.0 applications. The proposed approach targets some of the most common needs of manufacturing enterprises: compatibility with both the on-premises and the in-the-cloud environments, exploitation of reliable and largely supported Big Data platforms, easy deployment through containerized software modules, virtually unlimited horizontal scalability, fault-tolerant self-reconfiguration, flexible yet friendly streaming-KPI computations, and above all, the integrated provisioning of self-tuning machine learning techniques for predictive maintenance. The current implementation of iSTEP exploits a distributed architecture based on Apache Kafka, Spark Streaming, MLlib, and Cassandra; iSTEP provides (i) a specific feature engineering block aimed at automatically extracting metrics from the production monitoring time series, which improves the predictive performance by 77% on average, and (ii) a self-tuning approach that dynamically selects the best prediction algorithm, which improves the predictive performance up to 60%. The iSTEP engine provides transparent predictive models, able to provide end users with insights into the knowledge learned, and it has been experimentally evaluated on a public unbalanced failure dataset, whose extensive results are discussed in the paper.

Algorithms are powerful and necessary tools behind a large part of the information we use every day. However, they may introduce new sources of bias, discrimination and other unfair practices that affect people who are unaware of it. Greater algorithm transparency is indispensable to provide more credible and reliable services. Moreover, requiring developers to design transparent algorithm-driven applications allows them to keep the model accessible and human understandable, increasing the trust of end users. In this paper we present EBAnO, a new engine able to produce prediction-local explanations for a black-box model exploiting interpretable feature perturbations. EBAnO exploits the hypercolumns representation together with the cluster analysis to identify a set of interpretable features of images. Furthermore two indices have been proposed to measure the influence of input features on the final prediction made by a CNN model. EBAnO has been preliminary tested on a set of heterogeneous images. The results highlight the effectiveness of EBAnO in explaining the CNN classification through the evaluation of interpretable features influence.

Predictive maintenance is an ever-growing area of interest, spanning different fields and approaches. In the automotive industry faulty behaviors of the oxygen sensor are a key challenge to address. This paper presents OxyClog, a data-driven framework that, given a large number of time series collected from a vehicle's ECU (engine control unit), builds a model to predict if the oxygen sensor is currently unclogged, almost clogged (since the clogging of the sensor happens gradually), or clogged. OxyClog is characterized by a tailored preprocessing, which includes a custom and interpretable feature selection algorithm, along with a summarization strategy to transform a time-dependent problem into a time-independent one. Furthermore, a semi-supervised labeling methodology has been devised to use different data sources with different characteristics to define meaningful clogging labels. OxyClog integrates state-of-the-art classification algorithms – both interpretable and non-interpretable – to process real ECU data with good prediction performance.

Wireless Sensor Network (WSN) monitoring takes a primary role in many industrial and research processes. Huge amounts of WSN sensor readings are nowadays available and can be analyzed to discover fruitful knowledge. This paper focuses on analyzing historical WSN sensor readings to identify the combinations of sensors whose readings show an unexpected trend. Although significant variations of single sensor readings may be easily detected, discovering correlations between multiple sensor readings is challenging without using advanced data analytics tools. To tackle this issue, we present an itemset-based data mining approach to analyzing WSN data. It identifies the combinations of sensors (of arbitrary size) whose readings are unexpectedly low in a given time period. Since the readings acquired by multiple sensors may decrease in an alternate fashion, the discovered patterns provide new information compared to single sensor analysis. To make the mined patterns manageable by domain experts for manual inspection, the mining algorithm is driven by spatial constraints defined on the WSN topology. The experimental results, achieved on real WSN data, demonstrate the effectiveness of the proposed approach in detecting heating system malfunctioning.

In this paper we propose a new self-learning engine to streamline the analytics process, as it enables analysts to mine massive data repositories with minimal user intervention. In the context of cluster analysis on a collection of documents this new system, named SELF-DATA (SELF-learning DAta TrAnsformation), suggests to the analyst how to configure the whole mining process for a given dataset. SELF-DATA relies on an engine exploring different data weighting schemas (e.g., normalized term frequencies) and data transformation methods (e.g., PCA) before applying the cluster analysis, evaluating and comparing solutions through different quality indices (e.g., weighted Silhouette), and presenting the k-top solutions to the analyst. SELF-DATA will also include a knowledge base storing results of experiments on previously processed datasets, and a classification algorithm trained on the knowledge base content to forecast the best configuration for the whole mining process for an unexplored dataset. The first development of SELF-DATA running on Apache Spark has been validated on 5 collections of documents. Experimental results highlight that TF-IDF and logarithmic entropy are effective to measure item relevance with sparse datasets, and the LSI method outperforms PCA with a large dictionary.

Energy Certificates of Buildings (ECB) provide interesting information on the standard energy performance, thermo-physical and geometrical related properties of existing buildings. The analysis of such data collection is challenging due to data volume and heterogeneity of attributes. This paper presents EPICA a data mining framework to automatically explore a collection of ECB to extract interesting knowledge items. To this aim, EPICA first reduces the data dimensionality through the Principal Component Analysis, then a clustering algorithm is exploited to discover groups of ECB with similar features. Each group is then locally characterized by a set of relevant generalized association rules able to summarize interesting relations among variables influencing energy performance of buildings at different coarse granularities. Experimental results, obtained on real data collected from an energy certification dataset related to Piedmont Region, in North Western of Italy, shows the effectiveness of EPICA in extracting a manageable set of human-readable knowledge items characterizing the groups of buildings with different energy performance levels.

This paper proposes PASTA (PArameter-free Solutions for Textual Analysis), a large scale engine providing strategies to automatically tune the algorithm parameters for the whole text clustering process. A data weighting strategy (e.g., TF-IDF) and a transformation method of input data (e.g., LSI) is explored before performing the cluster analysis to reduce sparseness, and make the overall analysis problem more effectively tractable. PASTA includes auto-selection strategies to off-load the end-user from parameter tuning and achieve a good quality of the clustering results. PASTA's current implementation runs on Apache Spark, a state-of-the-art distributed computing framework. As a case study, PASTA has been validated on three collections of Wikipedia documents. The experimental results show the effectiveness and the efficiency of the proposed solution in analyzing collections of documents without tuning algorithm parameters and in discovering cohesive and well-separated groups of documents.

Today a large volume of energy-related data have been continuously collected. Extracting actionable knowledge from such data is a multi-step process that opens up a variety of interesting and novel research issues across two domains: energy and computer science. The computer science aim is to provide energy scientists with cutting-edge and scalable engines to effectively support them in their daily research activities. This paper presents SPEC, a scalable and distributed predictor of fine grain energy consumption in buildings. SPEC exploits a data stream methodology analysis over a sliding time window to train a prediction model tailored to each building. The building model is then exploited to predict the upcoming energy consumption at a time instant in the near future. SPEC currently integrates the artificial neural networks technique and the random forest regression algorithm. The SPEC methodology exploits the computational advantages of distributed computing frameworks as the current implementation runs on Spark. As a case study, real data of thermal energy consumption collected in a major city have been exploited to preliminarily assess the SPEC accuracy. The initial results are promising and represent a first step towards predicting fine grain energy consumption over a sliding time window.

In today's world, large volumes of data are being continuously generated by many scientific applications, such as bioinformatics or networking. Since each monitored event is usually characterized by a variety of features, high-dimensional datasets have been continuously generated. To extract value from these complex collections of data, different exploratory data mining algorithms can be used to discover hidden and non-trivial correlations among data. Frequent closed itemset mining is an effective but computational expensive technique that is usually used to support data exploration. Thanks to the spread of distributed and parallel frameworks, the development of scalable approaches able to deal with the so called Big Data has been extended to frequent itemset mining. Unfortunately, most of the current algorithms are designed to cope with low-dimensional datasets, delivering poor performances in those use cases characterized by high-dimensional data. This work introduces PaMPa-HD, a MapReduce-based frequent closed itemset mining algorithm for high dimensional datasets. An efficient solution has been proposed to parallelize and speed up the mining process. Furthermore, different strategies have been proposed to easily configure the algorithm parameter. The experimental results, performed on real-life high-dimensional use cases, show the efficiency of the proposed approach in terms of execution time, load balancing and robustness to memory issues.

Bicycle sharing systems are eco-friendly transportation systems that have found wide application in Smart urban environments. Monitoring and analyzing the occupancy levels of the system's stations is crucial for guaranteeing the quality of the offered service. Advanced data mining solutions tailored to bicycle sharing data analysis are needed to support system managers to react to critical situations (e.g., lack of parked bicycles at a station) that can yield service disruption. This paper presents STation Occupancy Predictor (STOP), a data mining framework to predicting the occupancy levels (i.e., critical or non-critical) of the stations in the near future through Bayesian and associative classifiers. The prediction is made based on the current and past station occupancy values and on the temporal information associated with the predicted time instant. A classification model per station is generated on the collected data to support domain experts in understanding the underlying events. The model allows predicting short-term station occupancy levels and characterizing system usage to support planning maintenance activities in the medium term as well. As a case study, STOP has been thoroughly evaluated on real data acquired from the bicycle sharing system of New York City. The results demonstrate the effectiveness of the STOP system.

Natural disasters have become more frequent during the past 20 years due to significant climate changes. These natural events are hotly debated on social networks like Twitter and a huge amount of short text messages are continuously and promptly exchanged with personal opinions, descriptions of the natural events and their corresponding consequences. The analysis of these large and complex data could help policy-makers to better understand the event as well as to set priorities. However, the correct configuration of the tweet mining process is still challenging due to variable data distribution and the availability of a large number of algorithms with different specific parameters. The analyst need to perform a large number of experiments to identify the best configuration for the overall knowledge discovery process. Innovative, scalable, and parameter-free solutions need to be explored to streamline the analytics process. This paper presents an enhanced version of PASTA (a distributed selftuning engine) applied to a crisis tweet collection to group a corpus of tweets into cohesive and well-separated clusters with minimal analyst intervention. Experimental results performed on real data collected during natural disasters show the effectiveness of PASTA in discovering interesting groups of correlated tweets without selecting neither the algorithms nor their parameters., © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Bicycle sharing systems are becoming increasingly popular in cities around the world as they are an inexpensive and sustainable means of transportation. Promoting the use of these systems substantially improves the quality of life in cities by reducing pollutant emissions and traffic congestion. In these systems, bikes are made available for shared use to individuals on a short-term basis. They allow people to borrow a bike in one dock and return it to any other station with free docks belonging to the same system. The occupancy level of the stations can be constantly monitored. However, to achieve a satisfactory user experience, all the stations in the system must be neither overloaded nor empty when the user needs to access the station. The aim of this paper is to analyze occupancy level data acquired from real systems to determine situations of dock overload in multiple stations which could lead to service disruption. The proposed methodology relies on a pattern mining approach. A new pattern type called Occupancy Monitoring Pattern is proposed here to detect situations of dock overload in multiple stations. Since stations are geo-referenced and their occupancy levels are periodically monitored, occupancy patterns can be filtered and evaluated by taking into consideration both the spatial and temporal correlation of the acquired measurements. The results achieved on real data highlight the potential of the proposed methodology in supporting domain experts in their maintenance activities, such as periodic re-balancing of the occupancy levels of the stations, as well as in improving user experience by suggesting alternative stations in the nearby area.

Monitoring air quality is currently a critical issue in smart cities. Air pollution-related data are commonly acquired through sensors deployed throughout the city area. To analyze these data collections, data analytics algorithms should be combined with reporting tools for discovering critical conditions and informing citizens and municipality actors. This paper proposes a new data mining engine to discover air quality patterns from air pollution-related data. This class of patterns includes many established patterns proposed in the data mining literature. In this study, we focused on a specific type of patterns, namely the frequent weighted itemsets, to identify combinations of pollutants that are, on average, in a critical condition. To show the usefulness of the proposed approach, the proposed engine was tested on real data acquired in a major Italian city.

Air pollution is one of the most important factor that can affect the quality of citizen life in the urban environment. Consequently, monitoring air pollution is currently a critical issue that needs to be addressed for enhancing the well being of citizens. This paper proposes a data analysis engine, based on business intelligence methodologies and open technologies, to support different targeted analysis on air pollution data. To analyse the problem from different facets, air pollution measurements are enriched with additional information such as meteorological and traffic data, which are collected through sensor networks available in the smart city context. This integrated dataset is periodically analyzed to generate informative dashboards based on a selection of Key Performance Indicators (KPIs). The informative dashboards can provide useful insights about pollutant concentration at different time granularity levels in the urban areas and support a joint evaluation of pollutant concentrations with climate conditions and traffic flow. As a reference use case, open data on air pollution in the urban area of a major Italian city is analyzed to demonstrate the effectiveness of the proposed approach in a real smart city context.

In today's world, large volumes of medical data are being continuously generated, but their value is severely undermined by our inability to translate them into knowledge and, ultimately, actions. Data mining techniques allow the extraction of previously unknown interesting patterns from large datasets, but their complexity limits their practical diffusion. Data-driven analysis is a multi-step process, in which health care professionals define analysis goals and assess extracted knowledge, while computer scientists tackle the non trivial task of driving the miner system analysis activity. This paper addresses the mining activity from a different perspective. We believe that mining systems should be able to devise which knowledge could be most interesting to users and extract actionable knowledge from large medical datasets, with minimal user intervention. More specifically, mining systems should be capable of (i) devising viable end-goals for a specific dataset (i.e., that yield actionable knowledge to the user and are feasible given the dataset characteristics and size), based on the expertise acquired during the analysis of previous datasets, and (ii) extracting a manageable set of knowledge. Automated data analysis should fuel the next generation of medical mining systems, thus enabling users to automatically mine massive data repositories.

In today's world, plenty of textual news on stock markets written in different languages are available for traders, financial promoters, and private investors. However, their potential in supporting trading in multiple foreign markets is limited by the large volume of the textual corpora, which is practically unmanageable for manual inspection. Although, text mining and information retrieval techniques allow the automatic generation of interesting summaries from document collections, the study and application of multilingual summarization algorithms to financial news is still an open research problem. This paper addresses the summarization of collections of financial documents written in different languages to enhance the financial actor's awareness of foreign markets. Specifically, the proposed mining system (i) is able to cope with news written in multiple languages, (ii) generates multiple-level summaries covering specific and high-level concepts in separate sections, on behalf of users with different skill levels, and (iii) ranks the summary content based on both objective and subjective quality indices. These features are taking an increasingly important role in financial data summarization. As a case study, a preliminary implementation of the proposed system has been presented and validated on real multilingual news ranging over stocks of different markets. The preliminary results show the effectiveness and usability of the proposed approach.

A two-phase methodology to analyze real-world data collections.Cluster content characterized through sequential patterns.A mobile application to allow ubiquitous data classification.Evaluation in the health care scenario using a collection of diabetic patients. Clustering real-world data is a challenging task, since many real-data collections are characterized by an inherent sparseness and variable distribution. An appealing domain that generates such data collections is the medical care scenario where collected data include a large cardinality of patient records and a variety of medical treatments usually adopted for a given disease pathology.This paper proposes a two-phase data mining methodology (MLC) to iteratively analyze different dataset portions and locally identify groups of objects with common properties. Discovered cohesive clusters are then analyzed using sequential patterns to characterize temporal relationships among data features. To support an automatic classification of new data objects within one of the discovered groups, a classification model is created starting from the computed cluster set. A mobile application has been also designed and developed to visualize and update data under analysis as well as categorizing new unlabeled data objects.The experimental evaluation conducted on real datasets in the medical care scenario showed the effectiveness of MLC to discover interesting knowledge items and to easily exploit them through a mobile application. Results have been also discussed from a medical perspective.

In the last few years, the capability to both generate and collect data of public interest within the urban area has increased at an unprecedented rate, to such an extent that data rapidly scale towards big urban data. The abundance of information collected through ad-hoc sensors networks in the smart city context provides a remarkable opportunity to tackle interesting urban challenges and to add intelligences in the urban environment. However, for each data source and type, different spatial and temporal references are potentially used. Hence, the complexity of dealing with such an heterogeneity of data has significantly increased. This paper proposes a distributed business intelligence engine, named BI2CITY, able to efficiently manage the process of integrating and analysing a large volume of heterogeneous data generated by various sources in the smart city context. BI2CITY exploits a Big Data approach to support (i) data storage, (ii) spatio-temporal data aggregation, and (iii) different targeted data analyses, such as correlating urban data and forecasting the expected values of some interesting data (e.g., air pollution). Spatio-temporal data aggregation and analyses are performed on the fly using MapReduce based algorithms. Experimental results on real data collected in a major Italian city demonstrate the effectiveness of the proposed distributed system to perform interesting and efficient analysis.

Today's citizens and city administrations have an increasing interest in monitoring the air quality in urban areas. Studying the causes of air pollution entails analyzing the correlations between heterogeneous data, among which pollutant concentrations, traffic flow measurements, and meteorological data. To this end, innovative data analytics solutions able to acquire, integrate, and analyze very large amounts of data are needed. This paper presents a new data mining system, named GEneralized Correlation analyzer of pOllution data (GECKO), to discover interesting and multiple-level correlations among a large variety of open air pollution-related data. Specifically, correlations among pollutant levels and traffic and climate conditions are discovered and analyzed at different abstraction levels. The knowledge extraction process is driven by a taxonomy to generalize low-level measurement values as the corresponding categories. To ease the manual inspection of the result, the extracted correlations are classified into few classes based on the semantics of underlying data. The experiments, performed on real data acquired in a major Italian Smart City, demonstrate the effectiveness of the proposed analytics engine in discovering correlations among pollutant data that are potentially useful for supporting city administrators in decision-making.

A new methodology to analyze Internet access behavior using frequency histograms.A two-level clustering approach to analyze real network measurements with noisy data.A new distance measure to identify user histogram outliers.Data mining support for distributed Internet monitoring applications. Users typically subscribe to an Internet access service on the basis of a specific download speed, but the actual service may differ. Several projects are active collecting internet access performance measurements on a large scale at the end user location. However, less attention has been devoted to analyzing such data and to inform users on the received services. This paper presents MiND, a cluster-based methodology to analyze the characteristics of periodic Internet measurements collected at the end user location. MiND allows to discover (i) groups of users with a similar Internet access behavior and (ii) the (few) users with somehow anomalous service. User measurements over time have been modeled through histograms and then analyzed through a new two-level clustering strategy. MiNDhas been evaluated on real data collected by Neubot, an open source tool, voluntary installed by users, that periodically collects Internet measurements. Experimental results show that the majority of users can be grouped into homogeneous and cohesive clusters according to the Internet access service that they receive in practice, while a few users receiving anomalous services are correctly identified as outliers. Both users and ISPs can benefit from such information: users can constantly monitor the ISP offered service, whereas ISPs can quickly identify anomalous behaviors in their offered services and act accordingly.

Understanding the behavior of a network from a large scale traffic dataset is a challenging problem. Big data frameworks offer scalable algorithms to extract information from raw data, but often require a sophisticated fine-tuning and a detailed knowledge of machine learning algorithms. To streamline this process, we propose self-learning insightful network analyzer (SeLINA), a generic, self-tuning, simple tool to extract knowledge from network traffic measurements. SeLINA includes different data analytics techniques providing self-learning capabilities to state-of-the-art scalable approaches, jointly with parameter auto-selection to off-load the network expert from parameter tuning. We combine both unsupervised and supervised approaches to mine data with a scalable approach. SeLINA embeds mechanisms to check if the new data fits the model, to detect possible changes in the traffic, and to, possibly automatically, trigger model rebuilding. The result is a system that offers human-readable models of the data with minimal user intervention, supporting domain experts in extracting actionable knowledge and highlighting possibly meaningful interpretations. SeLINA’s current implementation runs on Apache Spark. We tested it on large collections of real-world passive network measurements from a nationwide ISP, investigating YouTube, and P2P traffic. The experimental results confirmed the ability of SeLINA to provide insights and detect changes in the data that suggest further analyses.

In the last few years, a large number of smart meters have been deployed in buildings to continuously monitor fine-grained energy consumption. Meteorological data deeply impact energy consumption, and an in-depth analysis of collected and correlated data can uncover interesting and actionable insights to improve the overall energy balance of our communities and to enhance people’s awareness of energy wasting. To effectively extract meaningful and interpretable insights from large collections of energy measurements and multi-dimensional meteorological data, innovative data science methodologies should be devised. Research frontiers are addressing self-learning approaches, which allow non-experts to exploit machine learning techniques more easily, and algorithmic transparency of models, hence providing actionable, explicit, declarative knowledge representation. This paper presents METeorological Data Analysis for Thermal Energy CHaracterization (METATECH), a data mining engine based on both exploratory and unsupervised data analytics algorithms, devised to build transparent models correlating weather conditions and energy consumption in buildings. METATECH exploits a joint approach coupling cluster analysis and generalized association rules to allow a deeper yet human-readable understanding of how meteorological data impact heating consumption. First, a partitional clustering algorithm is applied to weather conditions. Then, resulting clusters are characterized by means of generalized association rules, which provide a self-learning explainable model of the most interesting correlations between energy consumption and weather conditions at different granularity levels. The experimental evaluation performed on real datasets demonstrates the effectiveness of the proposed approach in automatically extracting interesting knowledge from data, and provide it transparently to domain experts.

Large volumes of data are being produced by various modern applications at an ever increasing rate. These applications range from wireless sensors networks to social networks. The automatic analysis of such huge data volume is a challenging task since a large amount of interesting knowledge can be extracted. Association rule mining is an exploratory data analysis method able to discover interesting and hidden correlations among data. Since this data mining process is characterized by computationally intensive tasks, efficient distributed approaches are needed to increase its scalability. This paper proposes a novel cloud-based service, named SEARUM, to efficiently mine association rules on a distributed computing model. SEARUM consists of a series of distributed MapReduce jobs run in the cloud. Each job performs a different step in the association rule mining process. As a case study, the proposed approach has been applied to the network data scenario. The experimental validation, performed on two real network datasets, shows the effectiveness and the efficiency of SEARUM in mining association rules on a distributed computing model.

Internet measured data collected via passive measurement are analyzed to obtain localization information on nodes by clustering (i.e., grouping together) nodes that exhibit similar network path properties. Since traditional clustering algorithms fail to correctly identify clusters of homogeneous nodes, we propose the NetCluster novel framework, suited to analyze Internet measurement datasets. We show that the proposed framework correctly analyzes synthetically generated traces. Finally, we apply it to real traces collected at the access link of Politecnico di Torino campus LAN and discuss the network characteristics as seen at the vantage point.

Evaluating the degradation of predictive models over time has always been a difficult task, also considering that new unseen data might not fit the training distribution. This is a well-known problem in real-world use cases, where collecting the historical training set for all possible prediction labels may be very hard, too expensive or completely unfeasible. To solve this issue, we present a new unsupervised approach to detect and evaluate the degradation of classification and prediction models, based on a scalable variant of the Silhouette index, named Descriptor Silhouette, specifically designed to advance current Big Data state-of-the-art solutions. The newly proposed strategy has been tested and validated over both synthetic and real-world industrial use cases. To this aim, it has been included in a framework named SCALE and resulted to be efficient and more effective in assessing the degradation of prediction performance than current state-of-the-art best solutions.