1. 基于HDFS 的分布式文件系统.
- Author
-
刘 军, 冷芳玲, 李世奇, and 鲍玉斌
- Subjects
- *
DATA warehousing , *UPLOADING of data , *STORAGE , *BIG data , *OPEN source software , *SPACE , *HEALTH information technology - Abstract
This paper establishes an intelligent big data storage system IHDFS, based on the existing open source distributed file storage system HDFS. The system proposes and implements big data de-duplication module, big data placement module, big data intelligent migration module, and big data encoding module, which improves the efficiency of user visits and saves the storage space of the cluster. Experimental results show that the data de-duplication module can save the storage space. The data placement module provides a reasonable distribution of file upload storage layer, which twice the uploading speed; the data intelligent migration module improves the hit rate of files on the upper storage layer, which improves the efficiency of obtaining data; the data encoding module saves the storage space of the cluster about one third of the original. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF