Search

In many cases, a machine learning model must learn to correctly predict a few data points with particular values of interest in a broader range of data where many target values are zero. Zero-inflated data can be found in diverse scenarios, such as lumpy and intermittent demands, power consumption for home appliances being turned on and off, impurities measurement in distillation processes, and even airport shuttle demand prediction. The presence of zeroes affects the models' learning and may result in poor performance. Furthermore, zeroes also distort the metrics used to compute the model's prediction quality. This paper showcases two real-world use cases (home appliances classification and airport shuttle demand prediction) where a hierarchical model applied in the context of zero-inflated data leads to excellent results. In particular, for home appliances classification, the weighted average of Precision, Recall, F1, and AUC ROC was increased by 27%, 34%, 49%, and 27%, respectively. Furthermore, it is estimated that the proposed approach is also four times more energy efficient than the SOTA approach against which it was compared to. Two-fold models performed best in all cases when predicting airport shuttle demand, and the difference against other models has been proven to be statistically significant.

In recent years, the traditional feature engineering process for training machine learning models is being automated by the feature extraction layers integrated in deep learning architectures. In wireless networks, many studies were conducted in automatic learning of feature representations for domain-related challenges. However, most of the existing works assume some supervision along the learning process by using labels to optimize the model. In this paper, we investigate an approach to learning feature representations for wireless transmission clustering in a completely unsupervised manner, i.e. requiring no labels in the process. We propose a model based on convolutional neural networks that automatically learns a reduced dimensionality representation of the input data with 99.3% less components compared to a baseline principal component analysis (PCA). We show that the automatic representation learning is able to extract fine-grained clusters containing the shapes of the wireless transmission bursts, while the baseline enables only general separability of the data based on the background noise.

Non-intrusive load monitoring (NILM) is the process of obtaining appliance-level data from a single metering point, measuring total electricity consumption of a household or a business. Appliance-level data can be directly used for demand response applications and energy management systems as well as for awareness raising and motivation for improvements in energy efficiency. Recently, classical machine learning and deep learning (DL) techniques became very popular and proved as highly effective for NILM classification, but with the growing complexity these methods are faced with significant computational and energy demands during both their training and operation. In this paper, we introduce a novel DL model aimed at enhanced multi-label classification of NILM with improved computation and energy efficiency. We also propose an evaluation methodology for comparison of different models using data synthesized from the measurement datasets so as to better represent real-world scenarios. Compared to the state-of-the-art, the proposed model has its energy consumption reduced by more than 23% while providing on average approximately 8 percentage points in performance improvement when evaluating on data derived from REFIT and UK-DALE datasets. We also show a 12 percentage point performance advantage of the proposed DL based model over a random forest model and observe performance degradation with the increase of the number of devices in the household, namely with each additional 5 devices, the average performance degrades by approximately 7 percentage points.

The so-called black-box deep learning (DL) models are increasingly used in classification tasks across many scientific disciplines, including wireless communications domain. In this trend, supervised DL models appear as most commonly proposed solutions to domain-related classification problems. Although they are proven to have unmatched performance, the necessity for large labeled training data and their intractable reasoning, as two major drawbacks, are constraining their usage. The self-supervised architectures emerged as a promising solution that reduces the size of the needed labeled data, but the explainability problem remains. In this paper, we propose a methodology for explaining deep clustering, self-supervised learning architectures comprised of a representation learning part based on a Convolutional Neural Network (CNN) and a clustering part. For the state of the art representation learning part, our methodology employs Guided Backpropagation to interpret the regions of interest of the input data. For the clustering part, the methodology relies on Shallow Trees to explain the clustering result using optimized depth decision tree. Finally, a data-specific visualizations part enables connection for each of the clusters to the input data trough the relevant features. We explain on a use case of wireless spectrum activity clustering how the CNN-based, deep clustering architecture reasons.

Location based services, already popular with end users, are now inevitably becoming part of new wireless infrastructures and emerging business processes. The increasingly popular Deep Learning (DL) artificial intelligence methods perform very well in wireless fingerprinting localization based on extensive indoor radio measurement data. However, with the increasing complexity these methods become computationally very intensive and energy hungry, both for their training and subsequent operation. Considering only mobile users, estimated to exceed 7.4 billion by the end of 2025, and assuming that the networks serving these users will need to perform only one localization per user per hour on average, the machine learning models used for the calculation would need to perform $65 \times 10^{12}$ predictions per year. Add to this equation tens of billions of other connected devices and applications that rely heavily on more frequent location updates, and it becomes apparent that localization will contribute significantly to carbon emissions unless more energy-efficient models are developed and used. In this Chapter, we discuss the latest results and trends in wireless localization and look at paths towards achieving more sustainable AI. We then elaborate on a methodology for computing DL model complexity, energy consumption and carbon footprint and show on a concrete example how to develop a more resource-aware model for fingerprinting. We finally compare relevant works in terms of complexity and training CO$_2$ footprint., Comment: 20 pages, 7 figures, book chapter

Artificial Intelligence (AI) technologies are moving from customized deployments in specific domains towards generic solutions horizontally permeating vertical domains and industries. For instance, decisions on when to perform maintenance of roads or bridges or how to optimize public lighting in view of costs and safety in smart cities are increasingly informed by AI models. While various commercial solutions offer user friendly and easy to use AI as a Service (AIaaS), functionality-wise enabling the democratization of such ecosystems, open-source equivalent ecosystems are lagging behind. In this chapter, we discuss AIaaS functionality and corresponding technology stack and analyze possible realizations using open source user friendly technologies that are suitable for on-premise set-ups of small and medium sized users allowing full control over the data and technological platform without any third-party dependence or vendor lock-in., Comment: 11 Pages, 8 Figures, book chapter

In recent years, much work has been done on processing of wireless spectrum data involving machine learning techniques in domain-related problems for cognitive radio networks, such as anomaly detection, modulation classification, technology classification and device fingerprinting. Most of the solutions are based on labeled data, created in a controlled manner and processed with supervised learning approaches. However, spectrum data measured in real-world environment is highly nondeterministic, making its labeling a laborious and expensive process, requiring domain expertise, thus being one of the main drawbacks of using supervised learning approaches in this domain. In this paper, we investigate the use of self-supervised learning (SSL) for exploring spectrum activities in a real-world unlabeled data. In particular, we compare the performance of two SSL models, one based on a reference DeepCluster architecture and one adapted for spectrum activity identification and clustering, and a baseline model based on K-means clustering algorithm. We show that SSL models achieve superior performance regarding the quality of extracted features and clustering performance. With SSL models we achieve reduction of the feature vectors size by two orders of magnitude, while improving the performance by a factor of 2 to 2.5 across the evaluation metrics, supported by visual assessment. Additionally we show that adaptation of the reference SSL architecture to the domain data provides reduction of model complexity by one order of magnitude, while preserving or even improving the clustering performance.

The digital transformation of the energy infrastructure enables new, data driven, applications often supported by machine learning models. However, domain specific data transformations, pre-processing and management in modern data driven pipelines is yet to be addressed. In this paper we perform a first time study on generic data models that are able to support designing feature management solutions that are the most important component in developing ML-based energy applications. We first propose a taxonomy for designing data models suitable for energy applications, explain how this model can support the design of features and their subsequent management by specialized feature stores. Using a short-term forecasting dataset, we show the benefits of designing richer data models and engineering the features on the performance of the resulting models. Finally, we benchmark three complementary feature management solutions, including an open-source feature store suitable for time series., Comment: 16 pages, 5 figures, 3 tables

Location based services, already popular with end users, are now inevitably becoming part of new wireless infrastructures and emerging business processes. The increasingly popular Deep Learning (DL) artificial intelligence methods perform very well in wireless fingerprinting localization based on extensive indoor radio measurement data. However, with the increasing complexity these methods become computationally very intensive and energy hungry, both for their training and subsequent operation. Considering only mobile users, estimated to exceed 7.4billion by the end of 2025, and assuming that the networks serving these users will need to perform only one localization per user per hour on average, the machine learning models used for the calculation would need to perform 65*10^12 predictions per year. Add to this equation tens of billions of other connected devices and applications that rely heavily on more frequent location updates, and it becomes apparent that localization will contribute significantly to carbon emissions unless more energy-efficient models are developed and used. This motivated our work on a new DL-based architecture for indoor localization that is more energy efficient compared to related state-of-the-art approaches while showing only marginal performance degradation. A detailed performance evaluation shows that the proposed model producesonly 58 % of the carbon footprint while maintaining 98.7 % of the overall performance compared to state of the art model external to our group. Additionally, we elaborate on a methodology to calculate the complexity of the DL model and thus the CO2 footprint during its training and operation., Comment: 6 pages, 6 sections, accepted to ICC 2022

Machine learning (ML) has been used to develop increasingly accurate link quality estimators for wireless networks. However, more in-depth questions regarding the most suitable class of models, most suitable metrics and model performance on imbalanced datasets remain open. In this paper, we propose a new tree-based link quality classifier that meets high performance and fairly classifies the minority class and, at the same time, incurs low training cost. We compare the tree-based model, to a multilayer perceptron (MLP) non-linear model and two linear models, namely logistic regression (LR) and SVM, on a selected imbalanced dataset and evaluate their results using five different performance metrics. Our study shows that 1) non-linear models perform slightly better than linear models in general, 2) the proposed non-linear tree-based model yields the best performance trade-off considering F1, training time and fairness, 3) single metric aggregated evaluations based only on accuracy can hide poor, unfair performance especially on minority classes, and 4) it is possible to improve the performance on minority classes, by over 40% through feature selection and by over 20% through resampling, therefore leading to fairer classification results., Comment: 8 pages, 7 pictures, 4 tables

Hypothesis: By combining the experimental small- and wide-angle x-ray scattering (SWAXS) method with molecular dynamics simulations and the theoretical 'complemented-system approach' it is possible to obtain detailed information about the intra- and inter-molecular structure and dynamics of the solvation and hydration of the surfactant in organic and mixed solvents, e.g., of the nonionic surfactant Brij 35 (C12E23) in alcohols and aqueous alcohol-rich ternary systems. This first application of the complemented-system approach to the surfactant system will promote the use of this powerful methodology that is based on experimental and calculated SWAXS data in studies of colloidal systems. By applying high-performance computing systems, such an approach is readily available for studies in the colloidal domain. Experiments: SWAXS experiments and MD simulations were performed for binary Brij 35/alcohol and ternary Brij 35/water/alcohol systems with ethanol, n-butanol and n-hexanol as the organic solvent component at 25 {\deg}C. Findings: We confirmed the presence of solvated Brij 35 monomers in the studied organic media, revealed their preferential hydration and discussed their structural and dynamic features at the intra- and inter-molecular levels. Anisotropic effective surfactant molecular conformations were found. The influence of the hydrophobicity of the organic solvent on the hydration phenomena of surfactant molecules was explained., Comment: 31 pages, 6 figures, 1 table

Multiple-input multiple-output (MIMO) is an enabling technology to meet the growing demand for faster and more reliable communications in wireless networks with a large number of terminals, but it can also be applied for position estimation of a terminal exploiting multipath propagation from multiple antennas. In this paper, we investigate new convolutional neural network (CNN) structures for exploiting MIMO-based channel state information (CSI) to improve indoor positioning. We evaluate and compare the performance of three variants of the proposed CNN structure to five NN structures proposed in the scientific literature using the same sets of training-evaluation data. The results demonstrate that the proposed residual convolutional NN structure improves the accuracy of position estimation and keeps the total number of weights lower than the published NN structures. The proposed CNN structure yields from 2cm to 10cm better position accuracy than known NN structures used as a reference., Comment: 6 pages, 4 figures, 19 subfigures

After decades of research, the Internet of Things (IoT) is finally permeating real-life and helps improve the efficiency of infrastructures and processes as well as our health. As a massive number of IoT devices are deployed, they naturally incur great operational costs to ensure intended operations. To effectively handle such intended operations in massive IoT networks, automatic detection of malfunctioning, namely anomaly detection, becomes a critical but challenging task. In this paper, motivated by a real-world experimental IoT deployment, we introduce four types of wireless network anomalies that are identified at the link layer. We study the performance of threshold- and machine learning (ML)-based classifiers to automatically detect these anomalies. We examine the relative performance of three supervised and three unsupervised ML techniques on both non-encoded and encoded (autoencoder) feature representations. Our results demonstrate that; i) selected supervised approaches are able to detect anomalies with F1 scores of above 0.98, while unsupervised ones are also capable of detecting the said anomalies with F1 scores of, on average, 0.90, and ii) OC-SVM outperforms all the other unsupervised ML approaches reaching at F1 scores of 0.99 for SuddenD, 0.95 for SuddenR, 0.93 for InstaD and 0.95 for SlowD.

Ensuring a reliable communication in wireless networks strictly depends on the effective estimation of the link quality, which is particularly challenging when propagation environment for radio signals significantly varies. In such environments, intelligent algorithms that can provide robust, resilient and adaptive links are being investigated to complement traditional algorithms in maintaining a reliable communication. In this respect, the data-driven link quality estimation (LQE) using machine learning (ML) algorithms is one of the most promising approaches. In this paper, we provide a quantitative evaluation of design decisions taken at each step involved in developing a ML based wireless LQE on a selected, publicly available dataset. Our study shows that, re-sampling to achieve training class balance and feature engineering have a larger impact on the final performance of the LQE than the selection of the ML method on the selected data., Comment: accepted in PIMRC 2020. arXiv admin note: text overlap with arXiv:1812.08856

Hypothesis: The structural details, viscosity trends and dynamic phenomena in t-butanol/water solutions are closely related on the molecular scales across the entire composition range. Utilizing the experimental small- and wide-angle x-ray scattering (SWAXS) method, molecular dynamics (MD) simulations and the complemented-system approach method developed in our group it is possible to comprehensively describe the structure-viscosity-dynamics relationship in such structurally versatile hydrogen-bonded molecular liquids, as well as in similar, self-assembling systems with pronounced molecular and supramolecular structures at the intra-, inter-, and supra-molecular scales. Experiments: The SWAXS and x-ray diffraction experiments and MD simulations were performed for aqueous t-butanol solutions at 25 {\deg}C. Literature viscosity and self-diffusion data were also used. Findings: The interpretive power of the proposed scheme was demonstrated by the extensive and diverse results obtained for aqueous t-butanol solutions across the whole concentration range. Four composition ranges with qualitatively different structures and viscosity trends were revealed. The experimental and calculated zero-shear viscosities and molecular self-diffusion coefficients were successfully related to the corresponding structural details. The hydrogen bonds, that were, along with hydrophobic effects, recognized as the most important driving force for the formation of t-butanol aggregates, show intriguing lifetime trends and thermodynamic properties of their formation., Comment: Main manuscript file AND Supplemental Information, united in one file

Since the emergence of wireless communication networks, a plethora of research papers focus their attention on the quality aspects of wireless links. The analysis of the rich body of existing literature on link quality estimation using models developed from data traces indicates that the techniques used for modeling link quality estimation are becoming increasingly sophisticated. A number of recent estimators leverage machine learning (ML) techniques that require a sophisticated design and development process, each of which has a great potential to significantly affect the overall model performance. In this paper, we provide a comprehensive survey on link quality estimators developed from empirical data and then focus on the subset that use ML algorithms. We analyze ML-based link quality estimation (LQE) models from two perspectives using performance data. Firstly, we focus on how they address quality requirements that are important from the perspective of the applications they serve. Secondly, we analyze how they approach the standard design steps commonly used in the ML community. Having analyzed the scientific body of the survey, we review existing open source datasets suitable for LQE research. Finally, we round up our survey with the lessons learned and design guidelines for ML-based LQE development and dataset collection., Comment: Accepted in IEEE Communications Surveys and Tutorials (COMST)