Back to Search
Start Over
HB-File: An efficient and effective high-dimensional big data storage structure based on US-ELM.
- Source :
-
Neurocomputing . Oct2017, Vol. 261, p184-192. 9p. - Publication Year :
- 2017
-
Abstract
- With the rapid development of computer and the Internet techniques, the amount of data in all walks of life increases sharply, especially accumulating numerous high-dimensional big data such as the network transactions data, the user reviews data and the multimedia data. High-dimensional big data mixes the typical features of both high-dimensional data and big data, which has also brought new problems and great challenges for processing and optimizing the high-dimensional big data. In this case, the storage structure of high-dimensional big data is a critical factor that can affect the processing performance in a fundamental way. However, due to the huge dimensionality feature of high-dimensional data, the existing data storage techniques, such as row-store and column-store, are not very suitable for high-dimensional and large scale data. Therefore, in this paper, we present an efficient high-dimensional big data storage structure based on US-ELM, H igh-dimensional B ig Data File , named HB-File . Then, we propose a fuzzy cluster algorithm to differentiate the key dimension and non-key dimension of high-dimensional big data based on US-ELM, which can also gain the clusters of key dimension . After that, we propose the execution and API of HB-File based on the open source implementation of MapReduce, Hadoop system. With the intensive experiments, we show the effectiveness of HB-File in satisfying the storage of high-dimensional big data. [ABSTRACT FROM AUTHOR]
Details
- Language :
- English
- ISSN :
- 09252312
- Volume :
- 261
- Database :
- Academic Search Index
- Journal :
- Neurocomputing
- Publication Type :
- Academic Journal
- Accession number :
- 124075755
- Full Text :
- https://doi.org/10.1016/j.neucom.2016.06.080