1. Research of Hadoop Parameters Tuning Based on Function Monitoring
- Author
-
Yue Gao Tang, Feng Ping Chen, and Li Miao
- Subjects
Software framework ,Key factors ,Computer science ,media_common.quotation_subject ,Subroutine ,General Medicine ,Data mining ,Function (engineering) ,computer.software_genre ,computer ,Task (project management) ,media_common - Abstract
Hadoop is a popular software framework supports distributed processing of large data sets. However, with Hadoop being a relatively new technology, practitioners and administers often lack the expertise to tune it to get better performance. Hadoop parameters configuration is one of the key factors which influence the performance. In this article, we present a novel Hadoop parameters tuning method based on function monitoring. This method monitors the function call information during task run to analyze why the performance of Hadoop changes when tuning parameters, which will be helpful for practitioners and administer to tune parameters to get better performance.
- Published
- 2014