Search

Electricity losses are a frequently appearing problem in power grids. Non-technical losses (NTL) appear during distribution and include, but are not limited to, the following causes: Meter tampering in order to record lower consumptions, bypassing meters by rigging lines from the power source, arranged false meter readings by bribing meter readers, faulty or broken meters, un-metered supply, technical and human errors in meter readings, data processing and billing. NTLs are also reported to range up to 40% of the total electricity distributed in countries such as India, Pakistan, Malaysia, Brazil or Lebanon. This is an introductory level course to discuss how to predict if a customer causes a NTL. In the last years, employing data analytics methods such as machine learning and data mining have evolved as the primary direction to solve this problem. This course will present and compare different approaches reported in the literature. Practical case studies on real data sets will be included. As an additional outcome, attendees will understand the open challenges of NTL detection and learn how these challenges could be solved in the coming years.

The field of Machine Learning grew out of the quest for artificial intelligence. It gives computers the ability to learn statistical patterns from data without being explicitly programmed. These patterns can then be applied to new data in order to make predictions. Machine Learning also allows to automatically adapt to changes in the data without amending the underlying model. We deal every day dozens of times with Machine Learning applications such as when doing a Google search, using spam filters, face detection, speaking to voice recognition software or when sitting in a self-driving car. In recent years, machine learning methods have evolved in the smart grid community. This change towards analyzing data rather than modeling specific problems has lead to adaptable, more generic methods, that require less expert knowledge and that are easier to deploy in a number of use cases. This is an introductory level course to discuss what machine learning is and how to apply it to data-driven smart grid applications. Practical case studies on real data sets, such as load forecasting, detection of irregular power usage and visualization of customer data, will be included. Therefore, attendees will not only understand, but rather experience, how to apply machine learning methods to smart grid data.

Electricity losses are a frequently appearing problem in power grids. Non-technical losses (NTL) appear during distribution and include, but are not limited to, the following causes: Meter tampering in order to record lower consumptions, bypassing meters by rigging lines from the power source, arranged false meter readings by bribing meter readers, faulty or broken meters, un-metered supply, technical and human errors in meter readings, data processing and billing. NTLs are also reported to range up to 40% of the total electricity distributed in countries such as Brazil, India, Malaysia or Lebanon. This is an introductory level course to discuss how to predict if a customer causes a NTL. In the last years, employing data analytics methods such as data mining and machine learning have evolved as the primary direction to solve this problem. This course will compare and contrast different approaches reported in the literature. Practical case studies on real data sets will be included. Therefore, attendees will not only understand, but rather experience the challenges of NTL detection and learn how these challenges could be solved in the coming years.

Electricity theft is a major problem around the world in both developed and developing countries and may range up to 40% of the total electricity distributed. More generally, electricity theft belongs to non-technical losses (NTL), which are losses that occur during the distribution of electricity in power grids. In this paper, we build features from the neighborhood of customers. We first split the area in which the customers are located into grids of different sizes. For each grid cell we then compute the proportion of inspected customers and the proportion of NTL found among the inspected customers. We then analyze the distributions of features generated and show why they are useful to predict NTL. In addition, we compute features from the consumption time series of customers. We also use master data features of customers, such as their customer class and voltage of their connection. We compute these features for a Big Data base of 31M meter readings, 700K customers and 400K inspection results. We then use these features to train four machine learning algorithms that are particularly suitable for Big Data sets because of their parallelizable structure: logistic regression, k-nearest neighbors, linear support vector machine and random forest. Using the neighborhood features instead of only analyzing the time series has resulted in appreciable results for Big Data sets for varying NTL proportions of 1%-90%. This work can therefore be deployed to a wide range of different regions around the world., Proceedings of the 3rd IEEE/ACM International Conference on Big Data Computing Applications and Technologies (BDCAT 2016)

Orientador : Prof. Dr. Eduardo Cunha de Almeida Co-orientador : Prof. Dr. Yves Le Traon Tese (doutorado) - Universidade Federal do Paraná, Setor de Tecnologia, Programa de Pós-Graduação em Ciências da Computação. Defesa: Curitiba, 17/12/2014 Inclui referências Abstract: Database Management Systems (DBMS) have been successful at processing transaction workloads over decades. But contemporary systems, including Cloud computing, Internet-based systems, and sensors (i.e., Internet of Things (IoT)), are challenging the architecture of the DBMS with burgeoning transaction workloads. The direct consequence is that the development agenda of the DBMS is now heavily concerned with meeting non-functional requirements, such as performance, robustness and scalability [85]. Otherwise, any stressing workload will make the DBMS lose control of simple functional requirements, such as responding to a transaction request [62]. While traditional DBMS, including DB2, Oracle, and PostgreSQL, require embedding new features to meet non-functional requirements, the contemporary DBMS called as NewSQL [56, 98, 65] present a completely new architecture. What is still lacking in the development agenda is a proper testing approach coupled with burgeoning transaction workloads for validating the DBMS with nonfunctional requirements in mind. The typical non-functional validation is carried out by performance benchmarks. However, they focus on metrics comparison instead of finding defects. In this thesis, we address this lack by presenting different contributions for the domain of DBMS stress testing. These contributions fit different testing objectives to challenge each specific architecture of traditional and contemporary DBMS. For instance, testing the earlier DBMS (e.g., DB2, Oracle) requires incremental performance tuning (i.e., from simple setup to complex one), while testing the latter DBMS (e.g., VoltDB, NuoDB) requires driving it into different performance states due to its self-tuning capabilities [85]. Overall, this thesis makes the following contributions: 1) Stress TEsting Methodology (STEM): A methodology to capture performance degradation and expose system defects in the internal code due to the combination of a stress workload and mistuning; 2) Model-based approach for Database Stress Testing (MoDaST): An approach to test NewSQL database systems. Supported by a Database State Machine (DSM), MoDaST infers internal states of the database based on performance observations under different workload levels; 3) Under Pressure Benchmark (UPB): A benchmark to assess the impact of availability mechanisms in NewSQL database systems. We validate our contributions with several popular DBMS. Among the outcomes, we highlight that our methodologies succeed in driving the DBMS up to stress state conditions and expose several related defects, including a new major defect in a popular NewSQL. Resumo: Sistemas de Gerenciamento de Bando de Dados (SGBD) têm sido bem sucedidos no processamento de cargas de trabalho transacionais ao longo de décadas. No entanto, sistemas atuais, incluindo Cloud computing, sistemas baseados na Internet, e os sensores (ou seja, Internet of Things (IoT)), estão desafiando a arquitetura dos SGBD com crescentes cargas de trabalho. A conseqüência direta é que a agenda de desenvolvimento de SGBD está agora fortemente preocupada em atender requisitos não funcionais, tais como desempenho, robustez e escalabilidade [85]. Caso contrário, uma simples carga de trabalho de estresse pode fazer com que os SGBD não atendam requisitos funcionais simples, como responder a um pedido de transação [62]. Enquanto SGBD tradicionais exigem a incorporação de novos recursos para atender tais requisitos não-funcionais, os SGBD contemporâneos conhecidos como NewSQL [56, 98, 65] apresentam uma arquitetura completamente nova. O que ainda falta na agenda do desenvolvimento é uma abordagem de teste adequada que leve em conta requisitos não-funcionais. A validação não-funcional típica para SGBD é realizada por benchmarks. No entanto, eles se concentram na comparação baseada em métricas em vez de encontrar defeitos. Nesta tese, abordamos essa deficiência na agenda de desenvolvimento, apresentando contribuições diferentes para o domínio de testes de estresse para SGBD. Estas contribuições atendem diferentes objetivos de teste que desafiam arquiteturas específica de SGBD tradicionais e contemporâneos. No geral, esta tese faz as seguintes contribuições: 1) Stress TEstingMethodology (STEM): Uma metodologia para capturar a degradação do desempenho e expor os defeitos do sistema no código interno devido a combinação de uma carga de trabalho de estresse e problemas de configuração; 2) Model-based Database Stress Testing (MoDaST): Uma abordagem para testar sistemas de banco de dados NewSQL. Apoiado por uma máquina de estado de banco de dados (DSM), MoDaST infere estados internos do banco de dados com base em observações de desempenho sob diferentes níveis de carga de trabalho; 3) Under Pressure Benchmark (UPB): Um benchmark para avaliar o impacto dos mecanismos de disponibilidade em sistemas de banco de dados NewSQL. Nós validamos nossas contribuições com vários SGBD populares. Entre os resultados, destaca-se em nossas metodologias o sucesso em conduzir o SGBD para condições de estresse e expor defeitos relacionados, incluindo um novo major bug em um SGBD NewSQL popular.