Search

Developing and executing large-scale data analysis applications in parallel and distributed environments can be a complex and time-consuming task. Developers often find themselves diverted from their application logic to handle technical details about the underlying runtime and related issues. To simplify this process, ParSoDA, a Java library, has been proposed to facilitate the development of parallel data mining applications executed on HPC systems. It simplifies the process by providing built-in scalability mechanisms relying on the Hadoop and Spark frameworks. This paper presents ParSoDA-Py, the Python version of the ParSoDA library, which allows for further support of commonly used runtimes and libraries for big data analysis. After a complete library redesign, ParSoDA can be now easily integrated with other Python-based distributed runtimes for HPC systems, such as COMPSs and Apache Spark, and with the large ecosystem of Python-based data processing libraries. The paper discusses the adaptation process, which takes into consideration the new technical requirements, and evaluates both usability and scalability through some case study applications. [ABSTRACT FROM AUTHOR]

Data centers have become essential to modern society by catering to increasing number of Internet users and technologies. This results in significant challenges in terms of escalating energy consumption. Research on green initiatives that reduce energy consumption while maintaining performance levels is exigent for data centers. However, energy efficiency and resource utilization are conflicting in general. Thus, it is imperative to develop an application assignment strategy that maintains a trade-off between energy and quality of service. To address this problem, a profile-based dynamic energy management framework is presented in this paper for dynamic application assignment to virtual machines (VMs). It estimates application finishing times and addresses real-time issues in application resource provisioning. The framework implements a dynamic assignment strategy by a repairing genetic algorithm (RGA), which employs realistic profiles of applications, virtual machines and physical servers. The RGA is integrated into a three-layer energy management system incorporating VM placement to derive actual energy savings. Experiments are conducted to demonstrate the effectiveness of the dynamic approach to application management. The dynamic approach produces up to 48% better energy savings than existing application assignment approaches under investigated scenarios. It also performs better than the static application management approach with 10% higher resource utilization efficiency and lower degree of imbalance. [ABSTRACT FROM AUTHOR]

Internet technology has developed rapidly, especially in the field of cloud computing. With the gradual growth of cloud computing capabilities, power consumption in data centres has become a very important issue. The development of cloud computing has made data centres the cornerstone of today's global economic development, so data centres have also developed rapidly both in terms of construction scale and growth speed. However, large numbers of data centres consume huge amounts of power while also increasing the economic cost of cloud computing. They have led to soaring carbon dioxide emissions, which will have an unimaginable impact on the global climate. Therefore, the energy-consumption problem has become an important topic in current cloud computing research. How to save energy and reduce power consumption is a key issue, and this paper proposes an energy-saving job-scheduling method, which considers task dependency in a cloud computing environment. The proposed method considers the heterogeneous characteristics of data centres, models energy consumption based on the frequency and kernel number of the virtual machine CPU and provides new solutions to the problem of energy-consumption monitoring of cloud computing data centres. The main task is to divide each job into several tasks and then assign the tasks to virtual machines. Comparison of the simulation results, i.e. total execution time with job cutting and without job cutting, using the virtual machine (VM) (with the number of jobs set to 1000 and 2000), indicated that the total execution time and total energy consumption are better with job cutting than when the job is not cut, and this was not affected by the dependency of tasks. Moreover, job cutting also effectively reduces energy consumption and job discard rate. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of devising a systematic approach and software tools to support development of interactive supercomputer applications on the basis of low level codes that are typically used on supercomputers for numerical simulation and data processing. An interactive application should help a user to systematically organize all the activities associated with solution of some class of problems on remote high performance computing systems. Activities include input data preparation, chaining of remotely run computing jobs, visualization, search and comparison of results, performance optimization and others. A platform for development of interactive supercomputer applications is proposed. The core of the platform is a visual language that allows a developer to formally describe activities (operations) and their relations to immutable data objects ("inputs" and "outputs"). Such a representation of a problem domain contains information about meaningful combinations of operations and becomes a basis for automated derivation of necessary user scenarios. A developer collects a library of UI components to represent data objects and a library of program modules that implement operations. These libraries are used in generation of a web-application that provides end users with appropriate interface to support derived scenarios. [ABSTRACT FROM AUTHOR]

Scientific workflow applications have a large amount of tasks and data sets to be processed in a systematic manner. These applications benefit from cloud computing platform that offer access to virtually limitless resources provisioned elastically and on demand. Running data-intensive scientific workflow on geographically distributed data centres faces massive amount of data transfer. That affects the whole execution time and monitory cost of scientific workflows. The existing efforts on scheduling workflow concentrate on decreasing make span and budget; little concern has been paid to contemplate tasks and data sets dependency. In this paper, we introduced workflow scheduling technique to overcome data transfer and execute workflow tasks within deadline and budget constraints. The proposed techniques consist of initial data placement stage, which clusters and distributes datasets based on their dependence and replication-based partial critical path (R-PCP) technique which schedules tasks with data locality and dynamically maintains dependency matrix for the placement of generated data sets. To reduce run time datasets movement, we use interdata centre tasks replication and data sets replication to make sure data sets availability. Simulation results with four workflow applications illustrate that our strategy efficiently reduces data movement and executes all chosen workflows within user specified budget and deadline. Results reveal that R-PCP has 44.93% and 31.37% less data movement compared to random and adaptive data-aware scheduling (ADAS) techniques, respectively. R-PCP has 26.48% less energy consumption compared with ADAS technique. [ABSTRACT FROM AUTHOR]

Reliability analysis is one of the crucial issues for any scalable optical interconnection network. Torus is a highly scalable optical interconnect for data centre networks. The traditional torus network has XY routing algorithm. We have proposed a novel optimised routing algorithm. This paper focuses on the time-dependent and time-independent analysis for both terminal and broadcast reliabilities of the torus network using XY and optimised routing algorithm under various network sizes ( N × N where N = 8 , 16 , 32 , 64 ). The results are evaluated and compared considering nodes failures in MATLAB. [ABSTRACT FROM AUTHOR]

Improving the energy efficiency while guaranteeing quality of services (QoS) is one of the main challenges of efficient resource management of large-scale data centers. Dynamic virtual machine (VM) consolidation is a promising approach that aims to reduce the energy consumption by reallocating VMs to hosts dynamically. Previous works mostly have considered only the current utilization of resources in the dynamic VM consolidation procedure, which imposes unnecessary migrations and host power mode transitions. Moreover, they select the destinations of VM migrations with conservative approaches to keep the service-level agreements , which is not in line with packing VMs on fewer physical hosts. In this paper, we propose a regression-based approach that predicts the resource utilization of the VMs and hosts based on their historical data and uses the predictions in different problems of the whole process. Predicting future utilization provides the opportunity of selecting the host with higher utilization for the destination of a VM migration, which leads to a better VMs placement from the viewpoint of VM consolidation. Results show that our proposed approach reduces the energy consumption of the modeled data center by up to 38% compared to other works in the area, guaranteeing the same QoS. Moreover, the results show a better scalability than all other approaches. Our proposed approach improves the energy efficiency even for the largest simulated benchmarks and takes less than 5% time overhead to execute for a data center with 7600 physical hosts. [ABSTRACT FROM AUTHOR]

Cloud computing systems are splitting compute- and data-intensive jobs into smaller tasks to execute them in a parallel manner using clusters to improve execution time. However, such systems at increasing scale are exposed to stragglers, whereby abnormally slow running tasks executing within a job substantially affect job performance completion. Such stragglers are a direct threat towards attaining fast execution of data-intensive jobs within cloud computing. Researchers have proposed an assortment of different mechanisms, frameworks, and management techniques to detect and mitigate stragglers both proactively and reactively. In this paper, we present a comprehensive review of straggler management techniques within large-scale cloud data centres. We provide a detailed taxonomy of straggler causes, as well as proposed management and mitigation techniques based on straggler characteristics and properties. From this systematic review, we outline several outstanding challenges and potential directions of possible future work for straggler research. [ABSTRACT FROM AUTHOR]

The optimal selection of a datacenter is one of the most important challenges in the structure of a network for the wide distribution of resources in the environment of a geographically distributed cloud. This is due to the variety of datacenters with different quality-of-service (QoS) attributes. The user's requests and the conditions of the service-level agreements (SLAs) should be considered in the selection of datacenters. In terms of the frequency of datacenters and the range of QoS attributes, the selection of the optimal datacenter is an NP-hard problem. A method is therefore required that can suggest the best datacenter, based on the user's request and SLAs. Various attributes are considered in the SLA; in the current research, the focus is on the four important attributes of cost, response time, availability, and reliability. In a geo-distributed cloud environment, the nearest datacenter should be suggested after receiving the user's request, and according to its conditions, SLA violations can be minimized. In the approach proposed here, datacenters are clustered according to these four important attributes, so that the user can access these quickly based on specific need. In addition, in this method, cost and response time are taken as negative criteria, while accessibility and reliability are taken as positive, and the multi-objective NSGA-II algorithm is used for the selection of the optimal datacenter according to these positive and negative attributes. In this paper, the proposed method, known as NSGAII_Cluster, is implemented with the Random, Greedy and MOPSO algorithms; the extent of SLA violation of each of the above-mentioned attributes are compared using four methods. The simulation results indicate that compared to the Random, Greedy and MOPSO methods, the proposed approach has fewer SLA violations in terms of the cost, response time, availability, and reliability of the selected datacenters. [ABSTRACT FROM AUTHOR]

Data centre infrastructure management (DCIM) is the integration of information technology and facility management disciplines to centralise monitoring and management in data centres. One of the most important problems of DCIM tools is the analysis of the huge amount of data obtained from the real-time monitoring of thousands of resources. In this paper, an adaptation of the knowledge discovery process for dealing with the data analysis in DCIM tools is proposed. A case of study based on monitoring and labelling of nodes of a high performance computing data centre in real time is presented. This shows that characterising the state of the nodes according to a reduced and relevant set of metrics is feasible and its outcome directly usable, simplifying consequently the decision-making process in these complex infrastructures. [ABSTRACT FROM AUTHOR]

Data centers now play an important role in modern IT infrastructures. Although much research effort has been made in the field of green data center computing, performance metrics for green data centers have been left ignored. This paper is devoted to categorization of green computing performance metrics in data centers, such as basic metrics like power metrics, thermal metrics and extended performance metrics i.e. multiple data center indicators. Based on a taxonomy of performance metrics, this paper summarizes features of currently available metrics and presents insights for the study on green data center computing. [ABSTRACT FROM AUTHOR]

Data centers now play an important role in modern IT infrastructures. Related research shows that the energy consumption for data center cooling systems has recently increased significantly. There is also strong evidence to show that high temperatures in a data center will lead to higher hardware failure rates, and thus an increase in maintenance costs. This paper devotes itself in the field of thermal aware workload placement for data centers. In this paper, we propose an analytical model, which describes data center resources with heat transfer properties and workloads with thermal features. Then two thermal aware task scheduling algorithms, TASA and TASA-B, are presented which aim to reduce temperatures and cooling system power consumption in a data center. A simulation study is carried out to evaluate the performance of the proposed algorithms. Simulation results show that our algorithms can significantly reduce temperatures in data centers by introducing endurable decline in system performance. [ABSTRACT FROM AUTHOR]

Increasing demand for acquiring diverse range of services has led to the establishment of huge energy hungry cloud data centers all around the world. Cloud providers face with major concerns to reduce their energy consumption while ensuring high quality of service based on the Service Level Agreement (SLA). Consolidation is proposed as one of the most effective techniques for online energy saving in cloud environments with dynamic workloads. This paper proposes novel proactive online resource management policies to optimize energy, SLA, and number of migrations in cloud data centers. More precisely, this paper proposes new prediction algorithm for determination of overloaded hosts as well as novel multi-criteria decision making techniques to select virtual machines. The results of simulations using CloudSim simulator shows up to 98.11 % reduction in the output metric which is representative of energy consumption, SLA violation, and number of migrations, in comparison with state of the art. [ABSTRACT FROM AUTHOR]

Various topologies have been proposed for high-performance computing (HPC), i.e., fat-tree, Torus topology. Compared with conventional fat-tree topology, Torus performs much better when applied in HPC. Unfortunately, due to its wraparound links, Torus topology naturally has the tendency to trigger deadlock incidents inside the network. Researchers solve this problem by means of virtual channel, but this approach will also restrict the routing of message. In this paper, we propose a deadlock-free topology for HPC, called Mesh-of-Torus, which incarnates the good characteristics of Mesh and Torus topology. Comparing with mesh, Mesh-of-Torus has shorter network diameter. Furthermore, we have proposed a corresponding port assignment rules in consideration of complicated internal arbitration or scheduling mechanism incurred by the employment of virtual channel. Deadlock avoidance can be achieved when dimension-order routing algorithm and our port assignment rules are applied to Mesh-of-Torus. Finally, simulations and mathematical analysis have shown that Mesh-of-Torus outperforms Mesh in terms of average end-to-end latency and network load distribution. [ABSTRACT FROM AUTHOR]

In this paper, we develop the optimal minimum-energy scheduler for the adaptive joint allocation of the task sizes, computing rates, communication rates and communication powers in virtualized networked data centers (VNetDCs) that operate under hard per-job delay-constraints. The considered VNetDC platform works at the Middleware layer of the underlying protocol stack. It aims at supporting real-time stream service (such as, for example, the emerging big data stream computing (BDSC) services) by adopting the software-as-a-service (SaaS) computing model. Our objective is the minimization of the overall computing-plus-communication energy consumption. The main new contributions of the paper are the following ones: (i) the computing-plus-communication resources are jointly allotted in an adaptive fashion by accounting in real-time for both the (possibly, unpredictable) time fluctuations of the offered workload and the reconfiguration costs of the considered VNetDC platform; (ii) hard per-job delay-constraints on the overall allowed computing-plus-communication latencies are enforced; and, (iii) to deal with the inherently nonconvex nature of the resulting resource optimization problem, a novel solving approach is developed, that leads to the lossless decomposition of the afforded problem into the cascade of two simpler sub-problems. The sensitivity of the energy consumption of the proposed scheduler on the allowed processing latency, as well as the peak-to-mean ratio (PMR) and the correlation coefficient (i.e., the smoothness) of the offered workload is numerically tested under both synthetically generated and real-world workload traces. Finally, as an index of the attained energy efficiency, we compare the energy consumption of the proposed scheduler with the corresponding ones of some benchmark static, hybrid and sequential schedulers and numerically evaluate the resulting percent energy gaps. [ABSTRACT FROM AUTHOR]

With the development of the Internet, data centers have become vital infrastructures which provide computing, storage and other services for the networks. According to statistics, data centers consume large amount of electricity all around the world. In most cases, the majority of network devices in data centers are relatively idle, resulting in a waste of energy. Software defined network (SDN) was proposed by UC Berkeley and Stanford University around 2008, which allows the administrators to manage the network and set configurations through abstraction of lower level functionality. It also separates the control plane and the data plane, so administrators can control the network traffic through centralized controller instead of access to physical devices. This paper discusses the energy-saving model in data center networks based on SDN. We propose two different energy-saving algorithms, which can be applied to different data centers. Through centralized management and preprocessing traffic by SDN, we get better energy efficiency and reduce the energy cost by 30-40 %. To the best of our knowledge, this is the first work on energy saving in SDN architecture which provides two different algorithms that can be applied in different scenarios. [ABSTRACT FROM AUTHOR]

The rapid growth in demand for computational power has led to a shift to the cloud computing model established by large-scale virtualized data centers. Such data centers consume enormous amounts of electrical energy. Cloud providers must ensure that their service delivery is flexible to meet various consumer requirements. However, to support green computing, cloud providers also need to minimize the cloud infrastructure energy consumption while conducting the service delivery. In this paper, for cloud environments, a novel QoS-aware VMs consolidation approach is proposed that adopts a method based on resource utilization history of virtual machines. Proposed algorithms have been implemented and evaluated using CloudSim simulator. Simulation results show improvement in QoS metrics and energy consumption as well as demonstrate that there is a trade-off between energy consumption and quality of service in the cloud environment. [ABSTRACT FROM AUTHOR]

Computational clouds are increasingly becoming popular for the provisioning of computing resources and service on demand basis. As a backbone in computational clouds, a set of applications are configured over virtual machines running on a large number of server machines in data center networks (DCNs). Currently, DCNs use tree-based architecture which inherits the problems of limited bandwidth capacity and lower server utilization. This requires a new design of scalable and inexpensive DCN infrastructure which enables high-speed interconnection for exponentially increasing number of client devices and provides fault-tolerant and high network capacity. In this paper, we propose a novel architecture for DCN which uses Sierpinski triangle fractal to mitigate throughput bottleneck in aggregate layers as accumulated in tree-based structure. Sierpinski Triangle Based (STB) is a fault-tolerant architecture which provides at least two parallel paths for each pair of servers. The proposed architecture is evaluated in NS2 simulation. The performance of STB-based architecture is then validated by comparing the results with DCell and BCube DCN architecture. Theoretical analysis and simulation results verify that the proportion of switches to servers is 0.167 in STB, lower than BCube (3.67); the average shortest path length is limited between 5.0 and 6.7, whenever node failure proportion remains between 0.02 and 0.2, shorter than DCell and BCube in a four-level architecture. Network throughput is also increased in STB, which spends 87 s to transfer data than DCell and BCube in a given condition. The simulation results validate the significance of STB based DCN architecture for datacenter in computational clouds. [ABSTRACT FROM AUTHOR]

The vast majority of Web services and sites are hosted in various kinds of cloud services, and ordering some level of quality of service (QoS) in such systems requires effective load-balancing policies that choose among multiple clouds. Recently, software-defined networking (SDN) is one of the most promising solutions for load balancing in cloud data center. SDN is characterized by its two distinguished features, including decoupling the control plane from the data plane and providing programmability for network application development. By using these technologies, SDN and cloud computing can improve cloud reliability, manageability, scalability and controllability. SDN-based cloud is a new type cloud in which SDN technology is used to acquire control on network infrastructure and to provide networking-as-a-service (NaaS) in cloud computing environments. In this paper, we introduce an SDN-enhanced Inter cloud Manager (S-ICM) that allocates network flows in the cloud environment. S-ICM consists of two main parts, monitoring and decision making. For monitoring, S-ICM uses SDN control message that observes and collects data, and decision-making is based on the measured network delay of packets. Measurements are used to compare S-ICM with a round robin (RR) allocation of jobs between clouds which spreads the workload equitably, and with a honeybee foraging algorithm (HFA). We see that S-ICM is better at avoiding system saturation than HFA and RR under heavy load formula using RR job scheduler. Measurements are also used to evaluate whether a simple queueing formula can be used to predict system performance for several clouds being operated under an RR scheduling policy, and show the validity of the theoretical approximation. [ABSTRACT FROM AUTHOR]

Energy management for large-scale clusters has been the subject of significant research attention in recent years. The principle of energy proportionality states that we can save energy by activating only a subset of cluster nodes, in proportion to the current load. However, achieving the energy proportionality in shared-nothing clusters is challenging, because the arbitrary deactivation of nodes would make some data become unavailable. In this paper, we propose a new algorithm, named popularity-based covering sets (PCS), to achieve the energy proportionality in large-scale shared-nothing clusters. PCS determines the set of active nodes dynamically, in order to achieve the design goals of (a) guaranteeing the minimum level of availability for every data so that any job can execute promptly, and (b) providing more replicas for popular data to mitigate contention on the data. This differs from previous studies, where some data may become unavailable, or they provide the same number of replicas for every data. Furthermore, PCS is rack-aware and thus it can reduce the energy consumption of power-hungry rack components. Experiment results indicate that PCS improves the overall energy savings by up to 62% compared to previous algorithms without significant performance loss. [ABSTRACT FROM AUTHOR]

The rapid growth of todays IT demands reflects the increased use of cloud data centers. Reducing computational power consumption in cloud data center is one of the challenging research issues in the current era. Power consumption is directly proportional to a number of resources assigned to tasks. So, the power consumption can be reduced by a demotivating number of resources assigned to serve the task. In this paper, we have studied the energy consumption in cloud environment based on varieties of services and achieved the provisions to promote green cloud computing. This will help to preserve overall energy consumption of the system. Task allocation in the cloud computing environment is a well-known problem, and through this problem, we can facilitate green cloud computing. We have proposed an adaptive task allocation algorithm for the heterogeneous cloud environment. We applied the proposed technique to minimize the makespan of the cloud system and reduce the energy consumption. We have evaluated the proposed algorithm in CloudSim simulation environment, and simulation results show that our proposed algorithm is energy efficient in cloud environment compared to other existing techniques. [ABSTRACT FROM AUTHOR]

Nowadays, most systems and applications produce log records that are useful for security and monitoring purposes such as debugging programming errors, checking system status, and detecting configuration problems or even attacks. To this end, a log repository becomes necessary whereby logs can be accessed and visualized in a timely manner. This paper presents Loginson, a high-performance log centralization system for large-scale log collection and processing in large IT infrastructures. Besides log collection, Loginson provides high-level analytics through a visual interface for the purpose of troubleshooting critical incidents. We note that Loginson outperforms all of the other log centralization solutions by taking full advantage of the vertical scalability, and therefore decreasing Capital Expenditure (CAPEX) and Operating Expense (OPEX) costs for deployment scenarios with a huge volume of log data. [ABSTRACT FROM AUTHOR]

It is usual that the applications executed in data centers require the distribution of the same data from one node to others at various execution points and that some of them require to cope with multiple of these diffusions in parallel. Multicast-based communications are an alternative solution to sending data efficiently to multiple nodes. This paper proposes a novel technique which offers reliability and congestion control in the multi-multicast transfers in data center networks. The proposal is based on: (1) a new congestion control mechanism, which monitors the control information of the receivers, reducing the server injection rate, (2) taking advantage of the switch diffusion hardware, and (3) using IGMP snooping, which allows a network switch to multicast a packet just to the output links with host receivers joined to a multicast group. The implementation is made at user level and uses the UDP interface. Evaluation tests are performed in a CentOS-based cluster composed of 12 servers in the presence of multiple diffusions at the same time. Test results show improvements in the global bandwidth, avoid network saturation, and reduce overhead included by unicast communications in data transmission. [ABSTRACT FROM AUTHOR]

Energy management has become a significant concern in data centers for reducing operational costs. Using virtualization allows server consolidation, which increases server utilization and reduces energy consumption by turning off idle servers. This needs to consider the power state change overhead. In this paper, we investigate proactive resource provisioning in short-term planning for performance and energy management. To implement short-term planning based on workload prediction, this requires dealing with high fluctuations that are inaccurately predictable by using single value prediction. Unlike long-term planning, short-term planning can not depend on periodical patterns. Thus, we propose an adaptive range-based prediction algorithm instead of a single value. We implement and extensively evaluate the proposed range-based prediction algorithm with different days of real workload. Then, we exploit the range prediction for implementing proactive provisioning using robust optimization taking into consideration uncertainty of the demand. We formulate proactive VM provisioning as a multiperiod robust optimization problem. To evaluate the proposed approach, we use several experimental setups and different days of real workload. We use two metrics: energy savings and robustness for ranking the efficiency of different scenarios. Our approach mitigates undesirable changes in the power state of servers. This enhances servers' availability for accommodating new VMs, its robustness against uncertainty in workload change, and its reliability against a system failure due to frequent power state changes. [ABSTRACT FROM AUTHOR]

Differential matrix Riccati equations (DMREs) enable to model many physical systems appearing in different branches of science, in some cases, involving very large problem sizes. In this paper, we propose an adaptive algorithm for time-invariant DMREs that uses a piecewise-linearized approach based on the Padé approximation of the matrix exponential. The algorithm designed is based upon intensive use of matrix products and linear system solutions so we can seize the large computational capability that modern graphics processing units (GPUs) have on these types of operations using CUBLAS and CULATOOLS libraries (general purpose GPU), which are efficient implementations of BLAS and LAPACK libraries, respectively, for NVIDIA $$\copyright $$ GPUs. A thorough analysis showed that some parts of the algorithm proposed can be carried out in parallel, thus allowing to leverage the two GPUs available in many current compute nodes. Besides, our algorithm can be used by any interested researcher through a friendly MATLAB $$\copyright $$ interface. [ABSTRACT FROM AUTHOR]

Size and number of high-performance data centers are rapidly growing all around the world in recent years. The growth in the leakage power consumption of servers along with its exponential dependence on the ever increasing process variation in nanometer technologies has made it inevitable to move toward variation-aware power reduction strategies in data centers. In this paper, we address the problem of joint server placement and chassis consolidation to minimize power consumption of high-performance computing data centers under process variation. To this end, we introduce two variation-aware server placement heuristics as well as an integer linear programming (ILP)-based server placement method to find the best location of each server in the data center based on its power consumption and the data center heat recirculation model. We then incorporate a novel ILP-based variation-aware chassis consolidation technique to find the optimum task assignment solution under the obtained server placement approach to minimize total power consumption. Experimental results show that by applying the proposed joint variation-aware server placement and chassis consolidation techniques, up to 14.6 % improvement can be obtained at common data center utilization rates compared to state-of-the-art variation-unaware approaches. [ABSTRACT FROM AUTHOR]

The power management issue has always been a critical concern in cloud computing for supporting rapid growth of data centers. In this paper, our strategy is to implement working vacation (WV) to lower and eliminate unnecessary power consumed by idle servers. Two green systems are first proposed where one implements a single WV and the other implements multiple WVs in an operational cycle. The effect of various service rates and WV lengths on system delay and operating state probabilities is compared and studied. A cost function is developed by taking response time, system holding cost and power consumption cost into consideration. Control procedures in both green systems are mapped into Petri net-based models which contribute to designing a multiple decision process and describing system behaviors. The issue of determining the optimal service rate and WV length to obtain the cost optimality within response time guarantee is studied. The proposed Green control ( $$\mu $$ , $$\Theta )$$ policy combined with a heuristic algorithm allows cloud providers to solve constrained optimization problems. Simulation results show that significant cost savings and response time improvement can be validated as compared to a typical system. [ABSTRACT FROM AUTHOR]

In this paper, we characterize and model Google applications and jobs, based on a 1-month Google trace from a large-scale Google data center. We address four contributions: (1) we compute the valuable statistics about task events and resource utilization for Google applications, based on various types of resources and execution types; (2) we analyze the classification of applications via a K-means clustering algorithm with optimized number of sets, based on task events and resource usage; (3) we study the correlation of Google application properties and running features (e.g., job priority and scheduling class); (4) we finally build a model that can simulate Google jobs/tasks and dynamic events, in accordance with Google trace. Experiments show that the tasks simulated based on our model exhibit fairly analogous features with those in Google trace. 95+ % of tasks' simulation errors are $$<$$ 20 %, confirming a high accuracy of our simulation model. [ABSTRACT FROM AUTHOR]

This work investigates power and performance trade-offs for Web servers on a state-of-the-art, high-density, power-efficient SeaMicro SM15k cluster by AMD. We relied on the concept of virtual power states (VPSs), a combination of CPU utilization rate to the P/C power states available in modern processors, and on our global optimization algorithm called Slack Recovery, to deploy an adaptive global power management system in a production environment. The main contributions of this paper are twofold. First, it presents the Slack Recovery algorithm deployed on a real cluster, composed of 25 SeaMicro nodes. The algorithm finds a P-state and a utilization rate for each CPU node to minimize power under a minimum performance requirement. Second, it proposes a novel mechanism to control utilization rates in each server, a key aspect on our power/performance optimization system which enables the implementation of the VPS concept in practice. Experimental results show that our Slack Recovery-based system can reduce up to 6.7 % of the power consumption when compared to policies usually deployed in SeaMicro production systems. [ABSTRACT FROM AUTHOR]

With energy costs now accounting for nearly 30 % of a datacenter's operating expenses, energy consumption has become an important issue when designing and executing a parallel algorithm. This paper analyzes the energy consumption of MPI applications following the master-slave paradigm. The analytical model is derived for this paradigm and is validated over a master-slave matrix-multiplication. This analytical model is parameterized through architectural and algorithmic parameters, and it is capable of predicting the energy consumption for a given instance of the problem over a given architecture. We use an external, metered, power distribution unit that allows to easily measure the power consumption of computing nodes without the needing of dedicated hardware. [ABSTRACT FROM AUTHOR]

Although cloud computing has rapidly emerged as a widely accepted computing paradigm, the research on cloud computing is still at an early stage. Cloud computing suffers from different challenging issues related to security, software frameworks, quality of service, standardization, and power consumption. Efficient energy management is one of the most challenging research issues. The core services in cloud computing system are the SaaS (Software as a Service), PaaS (Platform as a Service), and IaaS (Infrastructure as a Service). In this paper, we study state-of-the-art techniques and research related to power saving in the IaaS of a cloud computing system, which consumes a huge part of total energy in a cloud computing system. At the end, some feasible solutions for building green cloud computing are proposed. Our aim is to provide a better understanding of the design challenges of energy management in the IaaS of a cloud computing system. [ABSTRACT FROM AUTHOR]

Cloud computing data centers are becoming increasingly popular for the provisioning of computing resources. The cost and operating expenses of data centers have skyrocketed with the increase in computing capacity. Several governmental, industrial, and academic surveys indicate that the energy utilized by computing and communication units within a data center contributes to a considerable slice of the data center operational costs. In this paper, we present a simulation environment for energy-aware cloud computing data centers. Along with the workload distribution, the simulator is designed to capture details of the energy consumed by data center components (servers, switches, and links) as well as packet-level communication patterns in realistic setups. The simulation results obtained for two-tier, three-tier, and three-tier high-speed data center architectures demonstrate the effectiveness of the simulator in utilizing power management schema, such as voltage scaling, frequency scaling, and dynamic shutdown that are applied to the computing and networking components. [ABSTRACT FROM AUTHOR]

Due to economical reasons, the traditional philosophy in data centers was to scale out, rather than scaling up. However, the advances in CMP technology enabled chip multiprocessors to become more prevalent and they are expected to become more affordable and power-efficient in the coming years. Current trend towards more densely packaged systems and increasing demand for higher performance push the market towards placing datacenters on highly powerful chips that have many cores on a single platform. However, increasing the number of cores on a single chip brings along very important problems to be addressed at the chip level regarding the use of shared resources and QoS satisfaction. After briefly exploring current datacenter perspective, this paper captures the current state of the art in the field of chip multiprocessors through a detailed discussion of different studies that pave the way to the datacenters on-chip. Finally, a number of open research issues are highlighted with the intention of inspiring new contributions and developments in the field of datacenters on-chip. [ABSTRACT FROM AUTHOR]

Energy saving for computing systems has recently become an important and worrying need. Energy demand has been increasing in many systems, especially in data centers and supercomputers. This article considers the problem of saving energy on storage systems taking advantage of SSD drives. SSD and magnetic disk devices offer different power characteristics, being SSD drives much less power consuming than conventional magnetic disk drives. This paper presents the design and evaluation of a novel power consumption-aware prefetching mechanism for hybrid storage systems. The prefetching mechanism aims to reduce the power consumption of high performance storage subsystems. Every disk access request is absorbed by an associated SSD device, and only when the SSD device is full, requests are forwarded to the disk in background. We have evaluated the proposed approach with the help of both synthetic and realistic workloads. The experimental results demonstrate that our solution achieves significant reduction in energy consumption. Additionally, the performance evaluation shows that our solution may bring a substantial I/O performance benefit. [ABSTRACT FROM AUTHOR]

Efficient detection and classification of brain tumors using magnetic resonance images provide significant support to the neurologists. However, many approaches developed for this purpose exhibit limited accuracy due to irregular boundary pixels and intensity non-uniformity in MR images. Therefore, to minimize these issues and attain better performance, a new methodology is proposed based on the fuzzy thresholding and local texture feature descriptor. The proposed model includes four fundamental steps: noise reduction, tumor extraction, feature extraction, and classification. Anisotropic diffusion filtering is implemented to reduce the noise without losing information that is essential in the interpretation of the brain tumor images. Then, spatial fuzzy C-means thresholding and morphological operations-based image segmentation are applied to extract the tumor area of the brain. In the very next step, obtains texture features using a complete local binary pattern - based feature descriptor. These features capture inherent information from brain MR images. In the later stage, these features are concatenated using a serial-based fusion approach before classification using supervised learning approaches (decision tree, naive bayes, random forest, and LogitBoost). The above investigations are evaluated with simulations on harvard medical school and Kaggle repository data sets. The experimental outcomes support the significance of the proposed methodology which exhibited better performance compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Recent years have witnessed an increasing growth in the usage of GPUs in cloud data centers. It is known that conventional virtualization techniques are not directly applicable to GPUs, making it a challenge to effectively take advantage of virtualization benefits. API remoting, full, para and hardware-assisted virtualization methods are adopted to empower VMs with GPU capabilities. With such a diversity in approaches, there is a need for a simulation environment to study the effectiveness of GPU virtualization techniques and evaluate GPU provisioning and scheduling policies in cloud data centers. In order to model and simulate GPU-enabled VMs in cloud data centers, this work proposes and describes a simulator architecture implemented as an extension of CloudSim. The extension eases up conducting experimental studies that otherwise need to be carried out in real cloud infrastructures. It includes models to simulate interference among co-running applications, the overhead of virtualization and power consumption of GPUs. To demonstrate the usefulness of our extension, we study NVIDIA GRID, a hardware-assisted GPU virtualization solution. We show that for situations where the number of VMs outperforms the number of hosts, the first-fit VM placement of VMware Horizon may not be effective. Instead, we suggest a first-fit increasing VM placement algorithm which increases the acceptance rate by 59%, shortens makespan by 25% and saves energy by 21%. [ABSTRACT FROM AUTHOR]

A proactive multicriteria mechanism for virtual data center optimization through server consolidation is proposed. In contrast with previous works where heuristic mechanisms were designed using expert knowledge, the new proactive approach uses multiobjective evolutionary algorithms to learn fuzzy rule-based systems that determine optimal reallocation decisions according to the preferences of the data center operator and a prediction of the load. Experimental evaluations based on an actual IT service provider show that the proactive mechanism is capable of improving energy savings compared to commercial hypervisors while complying with service provider's preferences and constraints. [ABSTRACT FROM AUTHOR]

Energy consumption is a growing concern in cloud data centers because underutilization of servers results in significant wasted power. Thus, improving server utilization for optimal energy use is now an urgent issue. We propose an energy-saving strategy based on multi-server vacation queuing theory that switches servers between on and sleep in groups. The strategy incorporates both synchronous and asynchronous strategies. When the number of idle servers reaches to a given threshold, idle servers enter sleep mode synchronously as a group. Varying workloads cause groups of servers to sleep asynchronously. We model the data center with our strategy as an M/M/H vacation queuing system and construct a two-dimensional continuous-time Markov chain to formulate the queuing system. Using a powerful matrix-geometric method, we obtain the stationary probability distribution for the system states. We use results from theoretical and simulated experiments to estimate the performance of our approach. The results are valuable for studying the power-performance trade-off in cloud data centers. [ABSTRACT FROM AUTHOR]

Computational scientific applications tend to be very data I/O intensive, producing a large amount of data as the execution result. In this research, we propose a new storage system using next-generation non-volatile memory that is suitable for exa-scale computing systems. This storage system is called the Cloud Computing Burst System (CCBS) and is composed of a unified table management module, data scoring module, and CCBS storage. In particular, CCBS operates as a workload enlightened storage system using its own data scoring module. The CCBS storage architecture consists of PCM/NAND Flash arrays and a data migration engine. CCBS storage cannot only provide a scaling out feature, but also improve the overall performance of the storage system. In addition, by using new non-volatile memory array, many benefits, such as low energy consumption, density scaling, and high performance, can be achieved. We demonstrate the effectiveness of our proposed system by simulating the storage system using scientific benchmarking tool. Our data scoring algorithm can provide 7% more hit rate than other methods for CCBS. In addition, our proposed system has improved storage system speed by 1.64 times, compared with only NAND Flash conventional model. [ABSTRACT FROM AUTHOR]

Datacenter servers are stepping into an era marked by powerful multi-/many-core processors. Severe problems such as I/O contentions in those large-scale platforms pose an unprecedented challenge. Prior studies primarily considered I/O bandwidth as a major performance bottleneck. However, our work reveals that in many cases the fundamental cause of I/O contentions is the inefficiency of OS schedulers. Particularly, the modern system is not aware of this fact and thus suffers from poor I/O performance, especially for datacenter servers. Based on our findings, we propose a new software-based scheduling approach, WiseThrottling, to reduce I/O contention. WiseThrottling performs asynchronous and self-adjustment scheduling for concurrent tasks. We evaluate our approach across a wide range of C/OpenMP/MapReduce workloads on a 64-core server in Dawning Cluster datacenter. The experimental results exhibit that WiseThrottling is effective for reducing the I/O bottleneck and it can improve the overall system performance by up to 207 %. [ABSTRACT FROM AUTHOR]

To cope with increasing demands of computation and storage, data centers should follow the pace of the rapid growth of data size. It is necessary for a data center with a scalability property of which each expansion of a data center network is done with a few modifications. Besides the scalability property, we also need a data center to have good performance, such as high throughput. For these purposes, we propose Decluster, a complex network model-based data center network topology. The complex network model of Decluster is derived from a random network. Such a model just satisfies the requirement of scalability. Decluster employs a complex network model to achieve high throughput via reducing the variance of local clustering coefficients. We have carried out extensive simulations to demonstrate that Decluster enjoys good performance while keeping scalability. [ABSTRACT FROM AUTHOR]

Modern data centers are playing an important role in a world full of information and communication technologies (ICTs). Many efforts have been paid to build a more efficient, cleaner data center for economic, social, and environmental benefits. This objective is being enabled by emerging technologies such as cloud computing and software-defined networking (SDN). However, a data center is inherently heterogeneous, consisting of servers, networking devices, cooling devices, power supply devices, etc., resulting in daunting challenges in its management and control. Previous approaches typically focus on only a single domain, for example, traditional cloud computing for server resource (e.g., computing resource and storage resource) management and SDN for network management. In a similar context of networking device heterogeneity, network function virtualization has been proposed to offer a standard abstract interface to manage all networking devices. In this research, we take the challenge of building a suit of unified middleware to monitor and control the three intrinsic subsystems in a data centre, including ICT, power, and cooling. Specifically, we present $$\upmu \mathrm{DC}^2$$ , a unified scalable IP-based data collection system for data center management with elevated extensibility, as an initial step to offer a unified platform for data center operations. Our system consists of three main parts, i.e., data-source adapters for information collection over various subsystems in a data center, a unified message bus for data transferring, and a high-performance database for persistent data storage. We have conducted performance benchmark for the key building components, namely messaging server and database, confirming that our system is scalable for a data center with high device density and real-time management requirements. Key features, such as configuration files, dynamical module loading, and data compression, enhance our implementation with high extensibility and performance. The effectiveness of our proposed data collection system is verified by sample applications, such as, traffic flow migration for load balancing, VM migration for resource reservation, and server power management for hardware safety. This research lays out a foundation for a unified data centre management in future. [ABSTRACT FROM AUTHOR]

Smart grid is an advanced electrical grid equipped with communication capability, which are utilized to improve the efficiency, reliability, and sustainability of electricity services. It often integrates itself with cloud computing and data centers, which help smart grid to provide high robustness and load balancing. Countries within Europe, North America, and East Asia are undergoing a transformation from an antiquated infrastructure to a smart grid. However, some of the problems arise due to the security and privacy issues of the smart grid. In this manuscript, we propose a novel privacy preserving billing protocol based on the priced oblivious transfer, which guarantees the grid operator to get the correct amount of money without knowing the current energy consumption of each customer. Additionally, we also implement the proposed protocol and provide a performance analysis of it. [ABSTRACT FROM AUTHOR]

Energy-efficient resource allocation within clusters and data centers is important because of the growing cost of energy. We study the problem of energy-constrained dynamic allocation of tasks to a heterogeneous cluster computing environment. Our goal is to complete as many tasks by their individual deadlines and within the system energy constraint as possible given that task execution times are uncertain and the system is oversubscribed at times. We use Dynamic Voltage and Frequency Scaling ( DVFS) to balance the energy consumption and execution time of each task. We design and evaluate (via simulation) a set of heuristics and filtering mechanisms for making allocations in our system. We show that the appropriate choice of filtering mechanisms improves performance more than the choice of heuristic (among the heuristics we tested). [ABSTRACT FROM AUTHOR]

We study the multi-objective problem of mapping independent tasks onto a set of data center machines that simultaneously minimizes the energy consumption and response time (makespan) subject to the constraints of deadlines and architectural requirements. We propose an algorithm based on goal programming that effectively converges to the compromised Pareto optimal solution. Compared to other traditional multi-objective optimization techniques that require identification of the Pareto frontier, goal programming directly converges to the compromised solution. Such a property makes goal programming a very efficient multi-objective optimization technique. Moreover, simulation results show that the proposed technique achieves superior performance compared to the greedy and linear relaxation heuristics, and competitive performance relative to the optimal solution implemented in Linear Interactive and Discrete Optimizer (LINDO) for small-scale problems. [ABSTRACT FROM AUTHOR]

The increasing demand for faster computing and high storage capacity has resulted in an increase in energy consumption and heat generation in datacenters. Because of the increase in heat generation, cooling requirements have become a critical concern, both in terms of growing operating costs as well as their environmental and societal impacts. Presently, thermal management techniques make an effort to thermally profile and control datacenters' cooling equipment to increase their efficiency. In conventional thermal management techniques, cooling systems are triggered by the temperature crossing predefined thresholds. Such reactive approaches result in delayed response as the temperature may already be too high, which can result in performance degradation of hardware. In this work, a proactive control approach is proposed that jointly optimizes the air conditioner compressor duty cycle and fan speed to prevent heat imbalance-the difference between the heat generated and extracted from a machine-thus minimizing the cost of cooling. The proposed proactive optimization framework has two objectives: (i) minimize the energy consumption of the cooling system, and (ii) minimize the risk of equipment damage due to overheating. Through thorough simulations comparing the proposed proactive heat-imbalance estimation-based approach against conventional reactive temperature-based schemes, the superiority of the proposed approach is highlighted in terms of cooling energy, response time, and equipment failure risk. [ABSTRACT FROM AUTHOR]

An introduction is presented in which the editors discuss various reports within the issue on topics including energy-efficient resource allocation in clusters and data centers, view-oriented transactional memory, and supercomputing centers that deploy large-scale compute system.

An introduction is presented in which the editor discusses various reports published within the issue on topics including performance metrics for energy-efficient data centers, the development of communication runtime systems, and the issues with the cloud computing for database query processing.