1. Validation of wastewater data using artificial intelligence tools and the evaluation of their performance regarding annotator agreement
- Author
-
Zidaoui, Imane, Wemmert, Cédric, Dufresne, Matthieu, Joannis, Claude, Isel, Sandra, Wertel, Jonathan, Vazquez, José, Laboratoire des sciences de l'ingénieur, de l'informatique et de l'imagerie (ICube), École Nationale du Génie de l'Eau et de l'Environnement de Strasbourg (ENGEES)-Université de Strasbourg (UNISTRA)-Institut National des Sciences Appliquées - Strasbourg (INSA Strasbourg), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Institut National de Recherche en Informatique et en Automatique (Inria)-Les Hôpitaux Universitaires de Strasbourg (HUS)-Centre National de la Recherche Scientifique (CNRS)-Matériaux et Nanosciences Grand-Est (MNGE), Université de Strasbourg (UNISTRA)-Université de Haute-Alsace (UHA) Mulhouse - Colmar (Université de Haute-Alsace (UHA))-Institut National de la Santé et de la Recherche Médicale (INSERM)-Institut de Chimie du CNRS (INC)-Centre National de la Recherche Scientifique (CNRS)-Université de Strasbourg (UNISTRA)-Université de Haute-Alsace (UHA) Mulhouse - Colmar (Université de Haute-Alsace (UHA))-Institut National de la Santé et de la Recherche Médicale (INSERM)-Institut de Chimie du CNRS (INC)-Centre National de la Recherche Scientifique (CNRS)-Réseau nanophotonique et optique, Université de Strasbourg (UNISTRA)-Université de Haute-Alsace (UHA) Mulhouse - Colmar (Université de Haute-Alsace (UHA))-Centre National de la Recherche Scientifique (CNRS)-Université de Strasbourg (UNISTRA)-Centre National de la Recherche Scientifique (CNRS), 3D EAU, and parent
- Subjects
[INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] ,[SDE.IE]Environmental Sciences/Environmental Engineering ,matrix profile ,one-class SVM ,annotator agreement ,artificial intelligence ,data validation ,wastewater ,Informatique [cs]/Apprentissage [cs.LG] - Abstract
To prevent the pollution of water resources, the measurement and the limitation of wastewater discharges are required. Despite the progress in the field of data acquisition systems, sensors are subject to malfunctions that can bias the evaluation of the pollution flow. It is therefore essential to identify potential anomalies in the data before any use. The objective of this work is to deploy artificial intelligence tools to automate the data validation and to assess the added value of this approach in assisting the validation performed by an operator. To do so, we compare two state-of-the-art anomaly detection algorithms on turbidity data in a sewer network. On the one hand, we conclude that the One-class SVM model is not adapted to the nature of the studied data which is heterogeneous and noisy. The Matrix Profile model, on the other hand, provides promising results with a majority of anomalies detected and a relatively limited number of false positives. By comparing these results to the expert validation, it turns out that the use of the Matrix Profile model objectifies and accelerates the validation task while maintaining the same level of performance compared to the annotator agreement rate between two experts.
- Published
- 2023
- Full Text
- View/download PDF