Search

The majority of large-scale data intensive applications designed by MapReduce model are deployed and executed on a large-scale distributed Hadoop system. Running such application on a large cluster requires a large amount of energy. Therefore improving energy efficiency and minimizing energy consumption when executing each MapReduce job is a critical concern for data centers. We propose a control model based on model prediction control (MPC) for improving energy efficiency of Hadoop cluster while satisfying performance goal. The controller can dynamically scale CPU frequency in facing the workloads changes. The wavelet neural network is utilized to build the prediction model, which can accurately estimate energy efficiency ratio based on the features extracting from the variation of Hadoop workloads. With the help of a wavelet neural network, the controller can guarantee the energy efficiency ratio converges to an ideal set point within a finite settling time. We perform the extensive experiments on a Hadoop cluster to compare our controller with the previous MPC controller. The results demonstrate that our controller outperforms the baseline on energy efficiency control and adapts to workloads of Hadoop cluster.