1. SentemQC - A novel and cost-efficient method for quality assurance and quality control of high-resolution frequency sensor data in fresh waters [version 1; peer review: 1 approved, 2 approved with reservations]
- Author
-
Sofie Gyritia Madsen van't Veen, Joachim Audet, Eti Ester Levi, Brian Kronvang, Søren Erik Larsen, Erik Jeppesen, Esben Astrup Kristensen, Anders Nielsen, Thomas Alexander Davidson, Jane Rosenstand Laugesen, and Peter Mejlhede Andersen
- Subjects
SentemQC ,quality assurance ,quality control ,sensor data ,high-frequency data ,python tool ,eng ,Science ,Social Sciences - Abstract
The growing use of sensors in fresh waters for water quality measurements generates an increasingly large amount of data that requires quality assurance (QA)/quality control (QC) before the results can be exploited. Such a process is often resource-intensive and may not be consistent across users and sensors. SentemQC (QA-QC of high temporal resolution sensor data) is a cost-efficient, and open-source Python approach developed to ensure the quality of sensor data by performing data QA and QC on large volumes of high-frequency (HF) sensor data. The SentemQC method is computationally efficient and features a six-step user-friendly setup for anomaly detection. The method marks anomalies in data using five moving windows. These windows connect each data point to neighboring points, including those further away in the moving window. As a result, the method can mark not only individual outliers but also clusters of anomalies. Our analysis shows that the method is robust for detecting anomalies in HF sensor data from multiple water quality sensors measuring nitrate, turbidity, oxygen, and pH. The sensors were installed in three different freshwater ecosystems (two streams and one lake) and experimental lake mesocosms. Sensor data from the stream stations yielded anomaly percentages of 0.1%, 0.1%, and 0.2%, which were lower than the anomaly percentages of 0.5%, 0.6%, and 0.8% for the sensors in Lake and mesocosms, respectively. While the sensors in this study contained relatively few anomalies (
- Published
- 2024
- Full Text
- View/download PDF