Back to Search
Start Over
Articulating Heterogeneous Data Streams with the Attribute-Relation File Format.
- Source :
- AIP Conference Proceedings; 2019, Vol. 2173 Issue 1, p020021-1-020021-10, 10p
- Publication Year :
- 2019
-
Abstract
- The processing strategy based on measurement metadata is a data stream engine running on Apache Storm, who is able to process measures in real-time. In the data stream context, the data have no an associated limit, they are al-ways arriving. The Attribute-Relation File Format (ARFF) is used by popular software like Weka, allowing offline analysis in the machine learning and data mining area. However, the ARFF file has a finite size. The CincamimisConversor library allows exporting from the data streams organized under a measurement interchange schema to a columnar-data organization in real-time. Here, an extension to the library is introduced for supporting the real-time translating and storing from the heterogeneous data streams to the ARFF file format. This is very useful, because through the library now is possible to collect data from heterogeneous data sources (e.g. Internet-of-Thing -IoT- devices) and export them in real-time for offline analysis in Weka. Even, this could foster a lot of educational applications among IoT, the measurement process with heterogeneous sources, data stream processing strategy, and Weka. A discrete simulation was carried out, obtaining promising results. It is just required at most 0.2387 ms for translating 5000 measures, while the storing operation for them consumed less than 0.2028 ms on a Solid-State disk. [ABSTRACT FROM AUTHOR]
- Subjects :
- RIVERS
DATA mining
INFORMATION storage & retrieval systems
MACHINE learning
METADATA
Subjects
Details
- Language :
- English
- ISSN :
- 0094243X
- Volume :
- 2173
- Issue :
- 1
- Database :
- Complementary Index
- Journal :
- AIP Conference Proceedings
- Publication Type :
- Conference
- Accession number :
- 139607360
- Full Text :
- https://doi.org/10.1063/1.5133936