Search

FPGA accelerators offer performance and efficiency gains by narrowing the scope of acceleration to one algorithmic domain. However, real-life applications are often not limited to a single domain, which naturally makes Cross-Domain Multi-Acceleration a crucial next step. The challenge is, existing FPGA accelerators are built upon their specific vertically-specialized stacks, which prevents utilizing multiple accelerators from different domains. To that end, we propose a pair of dual abstractions, called Yin-Yang, which work in tandem and enable programmers to develop cross-domain applications using multiple accelerators on a FPGA. The Yin abstraction enables cross-domain algorithmic specification, while the Yang abstraction captures the accelerator capabilities. We also develop a dataflow virtual machine, dubbed XLVM, that transparently maps domain functions (Yin) to best-fit accelerator capabilities (Yang). With six real-world cross-domain applications, our evaluations show that Yin-Yang unlocks 29.4× speedup, while the best single-domain acceleration achieves 12.0×.

Field-programmable gate array (FPGA) accelerators offer performance and efficiency gains by narrowing the scope of acceleration to one algorithmic domain. However, real-life applications are often not limited to a single domain, which naturally makes Cross-Domain Multi-Acceleration a crucial next step. The challenge is, existing FPGA accelerators are built upon their specific vertically specialized stacks, which prevents utilizing multiple accelerators from different domains. To that end, we propose a pair of dual abstractions, called Yin-Yang, which work in tandem and enable programmers to develop cross-domain applications using multiple accelerators on a FPGA. The Yin abstraction enables cross-domain algorithmic specification, while the Yang abstraction captures the accelerator capabilities. We also developed a dataflow virtual machine, dubbed Accelerator-Level Virtual Machine (XLVM), which transparently maps domain functions (Yin) to best-fit accelerator capabilities (Yang). With six real-world cross-domain applications, our evaluations show that Yin-Yang unlocks 29.4× speedup, while the best single-domain acceleration achieves 12.0×. [ABSTRACT FROM AUTHOR]

With the end of CPU core scaling due to dark silicon limitations, customized accelerators on FPGAs have gained increased attention in modern datacenters due to their lower power, high performance and energy efficiency. Evidenced by Microsoft's FPGA deployment in its Bing search engine and Intel's 16.7 billion acquisition of Altera, integrating FPGAs into datacenters is considered one of the most promising approaches to sustain future datacenter growth. However, it is quite challenging for existing big data computing systems-like Apache Spark and Hadoop-to access the performance and energy benefits of FPGA accelerators. In this paper we design and implement Blaze to provide programming and runtime support for enabling easy and efficient deployments of FPGA accelerators in datacenters. In particular, Blaze abstracts FPGA accelerators as a service (FaaS) and provides a set of clean programming APIs for big data processing applications to easily utilize those accelerators. Our Blaze runtime implements an FaaS framework to efficiently share FPGA accelerators among multiple heterogeneous threads on a single node, and extends Hadoop YARN with accelerator-centric scheduling to efficiently share them among multiple computing tasks in the cluster. Experimental results using four representative big data applications demonstrate that Blaze greatly reduces the programming efforts to access FPGA accelerators in systems like Apache Spark and YARN, and improves the system throughput by 1.7 × to 3× (and energy efficiency by 1.5× to 2.7×) compared to a conventional CPU-only cluster.

The relation between a structure and the function it runs is of interest in many fields, including computer science, biology (organ vs. function) and psychology (body vs. mind). Our paper addresses this question with reference to computer science recent hardware and software advances, particularly in areas as Robotics, Self-Adaptive Systems, IoT, CPS, AI-Hardware, etc. At the modelling, conceptual level our main contribution is the introduction of the concept of "virtual organism" (VO), to populate the intermediary level between reconfigurable hardware agents and intelligent, adaptive software agents. A virtual organism has a structure, resembling the hardware capabilities, and it runs low-level functions, implementing the software requirements. The model is compositional in space (allowing the virtual organisms to aggregate into larger organisms) and in time (allowing the virtual organisms to get composed functionalities). The virtual organisms studied here are in 2D (two dimensions) and their structures are described by 2D patterns (adding time, we get a 3D model). By reconfiguration an organism may change its structure to another structure in the same 2D pattern. We illustrate the VO concept with a few increasingly more complex VO's dealing with flow management or a publisher-subscriber mechanism for handling services. We implemented a simulator for a VO, collecting flow over a tree-structure (TC-VO), and the quantitative results show reconfigurable structures are better suited than fixed structures in dynamically changing environments. Finally, we briefly show how Agapia - a structured parallel, interactive programming language where dataflow and control flow structures can be freely mixed - may be used for getting quick implementations for VO's simulation. [ABSTRACT FROM AUTHOR]

The relation between a structure and the function it runs is of interest in many fields, including computer science, biology (organ vs. function) and psychology (body vs. mind). Our paper addresses this question with reference to computer science recent hardware and software advances, particularly in areas as Robotics, Self-Adaptive Systems, IoT, CPS, AI-Hardware, etc. At the modelling, conceptual level our main contribution is the introduction of the concept of "virtual organism" (VO), to populate the intermediary level between reconfigurable hardware agents and intelligent, adaptive software agents. A virtual organism has a structure, resembling the hardware capabilities, and it runs low-level functions, implementing the software requirements. The model is compositional in space (allowing the virtual organisms to aggregate into larger organisms) and in time (allowing the virtual organisms to get composed functionalities). The virtual organisms studied here are in 2D (two dimensions) and their structures are described by 2D patterns (adding time, we get a 3D model). By reconfiguration an organism may change its structure to another structure in the same 2D pattern. We illustrate the VO concept with a few increasingly more complex VO's dealing with flow management or a publisher-subscriber mechanism for handling services. We implemented a simulator for a VO, collecting flow over a tree-structure (TC-VO), and the quantitative results show reconfigurable structures are better suited than fixed structures in dynamically changing environments. Finally, we briefly show how Agapia - a structured parallel, interactive programming language where dataflow and control flow structures can be freely mixed - may be used for getting quick implementations for VO's simulation. [ABSTRACT FROM AUTHOR]

We present the design and implementation of an indirect messaging extension for the existing NFComms framework that provides communication between a network flow processor and host CPU. This extension addresses the bulk throughput limitations of the framework and is intended to work in conjunction with existing communication mediums. Testing of the framework extensions shows an increase in throughput performance of up to 268x that of the current direct message passing framework at the cost of increased single message latency of up to 2x. This trade-off is considered acceptable as the proposed extensions are intended for bulk data transfer only while the existing message passing functionality of the framework is preserved and can be used in situations where low latency is required for small messages. [ABSTRACT FROM AUTHOR]

As the hardware found within data centers becomes more heterogeneous, it is important to allow for efficient execution of algorithms across architectures. We present RAPID, a high-level programming language and combined imperative and declarative model for functionally- and performance-portable execution of sequential pattern-matching applications across CPUs, GPUs, Field-Programmable Gate Arrays (FPGAs), and Micron’s D480 AP. RAPID is clear, maintainable, concise, and efficient both at compile and run time. Language features, such as code abstraction and parallel control structures, map well to pattern-matching problems, providing clarity and maintainability. For generation of efficient runtime code, we present algorithms to convert RAPID programs into finite automata. Our empirical evaluation of applications in the ANMLZoo benchmark suite demonstrates that the automata processing paradigm provides an abstraction that is portable across architectures. We evaluate RAPID programs against custom, baseline implementations previously demonstrated to be significantly accelerated. We also find that RAPID programs are much shorter in length, are expressible at a higher level of abstraction than their handcrafted counterparts, and yield generated code that is often more compact. [ABSTRACT FROM AUTHOR]

Designers making deep learning computing more efficient cannot rely solely on hardware. Incorporating software-optimization techniques such as model compression leads to significant power savings and performance improvement. This article provides an overview of DeePhi's technology flow, including compression, compilation, and hardware acceleration. Two accelerators, named Aristotle and Descartes, are designed to achieve extremely high energy efficiency for both client and datacenter applications with convolutional neural network and recurrent neural network, respectively. [ABSTRACT FROM PUBLISHER]

The feasibility of running MPI-parallel, GPU-accelerated, multiphase flow simulations on Amazon's Elastic Compute Cloud (EC2) service is evaluated as an alternative computational resource. A cloud cluster on Amazon EC2 is compared to a conventional local high-performance computing cluster in terms of performance and cost. The steps necessary to set up a cloud cluster and acquire the appropriate hardware and software stacks are outlined. The incompressible multiphase flow solver is benchmarked on both cloud and local clusters by performing strong and weak scaling analyses. Amazon's EC2 service is competitive with the local cluster in a certain range of simulations, but there are some performance limitations, particularly in its GPU card performance and cluster network connection, which negatively impact the parallel simulations presented herein. Finally, Amazon's EC2 service is studied from an economic perspective and compared with a conventional local cluster. [ABSTRACT FROM PUBLISHER]

Data movement in high-performance computing systems accelerated by graphics processing units (GPUs) remains a challenging problem. Data communication in popular parallel programming models, such as the Message Passing Interface (MPI), is currently limited to the data stored in the CPU memory space. Auxiliary memory systems, such as GPU memory, are not integrated into such data movement standards, thus providing applications with no direct mechanism to perform end-to-end data movement. We introduce MPI-ACC, an integrated and extensible framework that allows end-to-end data movement in accelerator-based systems. MPI-ACC provides productivity and performance benefits by integrating support for auxiliary memory spaces into MPI. MPI-ACC supports data transfer among CUDA, OpenCL and CPU memory spaces and is extensible to other offload models as well. MPI-ACC's runtime system enables several key optimizations, including pipelining of data transfers, scalable memory management techniques, and balancing of communication based on accelerator and node architecture. MPI-ACC is designed to work concurrently with other GPU workloads with minimum contention. We describe how MPI-ACC can be used to design new communication-computation patterns in scientific applications from domains such as epidemiology simulation and seismology modeling, and we discuss the lessons learned. We present experimental results on a state-of-the-art cluster with hundreds of GPUs; and we compare the performance and productivity of MPI-ACC with MVAPICH, a popular CUDA-aware MPI solution. MPI-ACC encourages programmers to explore novel application-specific optimizations for improved overall cluster utilization. [ABSTRACT FROM PUBLISHER]

The relentless push in technology scaling driven by Moore's law has witnessed fantastic gains in the quantities of transistors available on chips. Computer architects have exploited the extra transistors by incorporating several computing cores within a single processor. Heterogeneous processing in particular has become a useful technique for dealing with ever-present power and memory restrictions. Yet, the scope and diversity of current heterogeneous designs remain bounded by the level of functional abstraction specified by conventional instruction-set architectures (ISAs). In this article, the authors demonstrate how the functional abstraction level determines the capability and variety of a processor's functional units and accelerators, thereby restricting its degree of heterogeneity. Combining current heterogeneous techniques with software abstraction concepts, the authors propose a new functional ISA (F-ISA), which raises the functional abstraction level of machine instructions. Using this model to complement existing architectures makes available a wider scope and diversity of functional units and accelerators in order to exploit the ever-increasing transistor densities. Greater heterogeneity can offer advances in terms of object data mapping and execution, resulting in potentially substantial latency, memory footprint, and power/performance gains. [ABSTRACT FROM PUBLISHER]

Accelerator-based heterogeneous computing is gaining momentum in the high-performance computing arena. However, the increased complexity of heterogeneous architectures demands more generic, high-level programming models. OpenACC is one such attempt to tackle this problem. Although the abstraction provided by OpenACC offers productivity, it raises questions concerning both functional and performance portability. In this article, the authors propose HeteroIR, a high-level, architecture-independent intermediate representation, to map high-level programming models, such as OpenACC, to heterogeneous architectures. They present a compiler approach that translates OpenACC programs into HeteroIR and accelerator kernels to obtain OpenACC functional portability. They then evaluate the performance portability obtained by OpenACC with their approach on 12 OpenACC programs on Nvidia CUDA, AMD GCN, and Intel Xeon Phi architectures. They study the effects of various compiler optimizations and OpenACC program settings on these architectures to provide insights into the achieved performance portability. [ABSTRACT FROM PUBLISHER]

The paradigm of pipelined MPSoC (processors connected in a pipeline) is well suited to data flow nature of multimedia applications. Often design space exploration is performed to optimize execution time, latency or throughput of a pipelined MPSoC where the variants in the system are processor configurations due to customizable options in each of the processors. Since there can be billions of combinations of processor configurations (design points), the challenge is to quickly provide estimates of performance metrics of those design points. Hence, in this article, we propose analytical models to estimate execution time, latency and throughput of a pipelined MPSoC's design points, avoiding slow full-system cycle accurate simulations of all the design points. For effective use of these analytical models, latencies of individual processor configurations should be available. We propose two estimation methods (PS and PSP) to quickly gather latencies of processor configurations with reduced number of simulations. The PS method simulates all the processor configurations once, while the PSP method simulates only a subset of processor configurations and then uses a processor analytical model to estimate the latencies of the remaining processor configurations. We experimented with several pipelined MPSoCs executing typical multimedia applications (JPEG encoder/decoder, MP3 encoder and H.264 encoder). Our results show that the analytical models with PS and PSP methods had maximum absolute error of 12.95 percent and 18.67 percent respectively, and minimum fidelity of 0.93 and 0.88 respectively. The design spaces of the pipelined MPSoCs ranged from 10^12 to 10^18 design points, and hence simulation of all design points will take years and is infeasible. Compared to PS method, the PSP method reduced simulation time from days to several hours. [ABSTRACT FROM PUBLISHER]

Developing and maintaining context-aware efficient systems in heterogeneous environments is a challenging task. In our research work, we enable context awareness in users, devices, and applications to enable context-based systems in Ambient Networks. We achieve this by proposing a context dissemination system which propagates the fast evolving context information from its sources (e.g., context sensors) to various interested information sinks (e.g., context sensitive clients). The proposed system implements a context aware overlay architecture composed of multi-level overlay networks. This overlay architecture acts as a base (middleware) for the development and maintenance of the application-layer context-specific dissemination protocol (the CSON-D protocol). The protocol’s multi-level overlay structure and its intelligence, personalization, and fault tolerance features exhibit adaptive behavior with minimized casting functionality. Our conceptual model is reinforced by means of experimental evaluations. In general, the cost-based overhead for multi-level overlay formation and maintenance is minimal. [ABSTRACT FROM PUBLISHER]

Given the recent progress in the evolution of high-performance computing (HPC) technologies, the research in computational intelligence has entered a new era. In this paper, we present an HPC-based context-aware intelligent text recognition system (ITRS) that serves as the physical layer of machine reading. A parallel computing architecture is adopted that incorporates the HPC technologies with advances in neuromorphic computing models. The algorithm learns from what has been read and, based on the obtained knowledge, it forms anticipations of the word and sentence level context. The information processing flow of the ITRS imitates the function of the neocortex system. It incorporates large number of simple pattern detection modules with advanced information association layer to achieve perception and recognition. Such architecture provides robust performance to images with large noise. The implemented ITRS software is able to process about 16 to 20 scanned pages per second on the 500 trillion floating point operations per second (TFLOPS) Air Force Research Laboratory (AFRL)/Information Directorate (RI) Condor HPC after performance optimization. [ABSTRACT FROM AUTHOR]

Abstract: Cluster virtualization is a promising approach to construct customized execution environments for Grid users. However, Virtual-Machine Cluster (VCluster) comes with the cost of the overhead caused by virtual machines, which therefore degrades system performance. In this paper, we propose a novel hybrid cluster model, namely TopCluster. By exploiting a Hybrid Batch System (HyBS) mechanism, TopCluster can flexibly allocate both physical or virtualized resources via a unified front-end. To reduce the overhead caused by the dynamic deployment of virtual machines and applications, we also propose a value-based deployment strategy along with TopCluster. An XML-based Application Deployment Description Language (ADDL) is also newly proposed to describe properties, installation procedure, and configuration operations of applications. A prototype system has been implemented and a series of experiments was conducted to evaluate TopCluster by replaying a real world trace. Results show that TopCluster can averagely generate 24.6% and 46.8% higher system throughput, 26.3% and 110.6% higher resource utility, and 35.1% and 56.1% lower waiting time of jobs than two commonly applied cluster models in Grid: Traditional Physical Cluster (PCluster) and VCluster. [Copyright &y& Elsevier]

The term “Dark Silicon” has been coined to describe the threat to microprocessor performance caused by increasing transistor power density. Improving energy efficiency is now the primary design goal for all market segments of microprocessors from mobile to server. Specialized hardware accelerators, designed to run only a subset of workloads with orders of magnitude energy efficiency improvement, are seen as a potential solution. Selecting an ensemble of accelerators to best cover the workloads run on a platform remains a challenge. We propose metrics for accelerator selection derived from a detailed communication-aware performance model and present an automated methodology to populate this model. Employing a combination of characterized RTL and our selection metrics, we evaluate a set of accelerators for a sample application and compare performance to selections based on execution time and Pollack’s rule. We find that the architecture selected by our communication-aware metric shows improved performance over architectures selected based on execution time and Pollack’s rule, as they do not account for speedup being limited by communication. [ABSTRACT FROM PUBLISHER]

The hundreds of thousands of servers in modern warehouse-scale systems make performance and efficiency optimizations pressing design challenges. These systems are traditionally considered homogeneous. However, that is not typically the case. Multiple server generations compose a heterogeneous environment, whose performance opportunities have not been fully explored since techniques that account for platform heterogeneity typically do not scale to the tens of thousands of applications hosted in large-scale cloud providers. We present ADSM, a scalable and efficient recommendation system for application-to-server mapping in large-scale datacenters (DCs) that is QoS-aware. ADSM overcomes the drawbacks of previous techniques, by leveraging robust and computationally efficient analytical methods to scale to tens of thousands of applications with minimal overheads. It is also QoS-aware, mapping applications to platforms while enforcing strict QoS guarantees. ADSM is derived from validated analytical models, has low and bounded prediction errors, is simple to implement and scales to thousands of applications without significant changes to the system. Over 390 real DC workloads, ADSM improves performance by 16% on average and up to 2.5x and efficiency by 22% in a DC with 10 different server configurations. [ABSTRACT FROM PUBLISHER]

RTL design complexity discouraged adoption of reconfigurable logic in general purpose systems, impeding opportunities for performance and energy improvements. Recent improvements to HLS compilers simplify RTL design and are easing this barrier. A new challenge will emerge: managing reconfigurable resources between multiple applications with custom hardware designs. In this paper, we propose a method to "shrink-fit" accelerators within widely varying fabric budgets. Shrink-fit automatically shrinks existing accelerator designs within small fabric budgets and grows designs to increase performance when larger budgets are available. Our method takes advantage of current accelerator design techniques and introduces a novel architectural approach based on fine-grained virtualization. We evaluate shrink-fit using a synthesized implementation of an IDCT for decoding JPEGs and show the IDCT accelerator can shrink by a factor of 16x with minimal performance and area overheads. Using shrink-fit, application designers can achieve the benefits of hardware acceleration with single RTL designs on FPGAs large and small. [ABSTRACT FROM PUBLISHER]

A grid differs from traditional high-performance computing systems in the heterogeneity of the computing nodes, as well as the communication links that connect the different nodes together. In grids, there exist users and service providers. The service providers provide the service for jobs that the users generate. Typically, the amount of jobs generated by all of the users is more than any single provider can handle alone with any acceptable quality of service (QoS). As such, the service providers need to cooperate and allocate jobs among them so that each is providing an acceptable QoS to their customers. QoS is of particular concern to service providers as it directly affects customers' satisfaction and loyalty. In this paper, we propose a game theoretic solution to the QoS sensitive grid job allocation problem. We model the QoS-based grid job allocation problem as a cooperative game and present the structure of the Nash Bargaining Solution. The proposed algorithm is fair to all users and represents a Pareto optimal solution to the QoS objective. One advantage of our scheme is the relatively low overhead and robust performance against inaccuracies in performance prediction information. [ABSTRACT FROM AUTHOR]

The paper presents a performance model that can be used to optimally distribute computations over heterogeneous computers. This model is application-centric representing the speed of each computer by a function of the problem size. This way it takes into account the processor heterogeneity, the heterogeneity of memory structure, and the memory limitations at each level of memory hierarchy. A problem of optimal partitioning of an n-element set over p heterogeneous processors using this performance model is formulated, and its efficient solution of the complexity O(p^{3}× log_{2} n) is given. [ABSTRACT FROM AUTHOR]

Field-programmable gate arrays (FPGAs) have become an extremely popular implementation technology for custom hardware because they offer a combination of low cost and very fast turnaround. Because of their in-system reconfigurability, FPGAs have also been suggested as an efficient replacement for application-specific integrated circuits (ASICs) and digital signal processors (DSPs) for applications that require a combination of high performance, low cost, and flexibility. Unfortunately, the use of FPGAs in mobile embedded systems platforms is hampered by the very large overhead of FPGA-based architectures. Coarse-grained configurable architectures can reduce this overhead substantially by taking advantage of the application domain to specialize the reconfigurable architecture via coarse-grained components and interconnects. This paper presents the design and implementation of an OFDM receiver in the RaPiD reconfigurable architecture as a case study for comparing the relative cost and performance of ASIC, DSP, FPGA, and coarse-grained reconfigurable architectures. RaPiD is a coarse-grained reconfigurable architecture specified to the domain of signal and image processing. The RaPiD architecture provides a reconfigurable pipelined datapath controlled by efficient reconfigurable control logic. We have implemented the computationally intensive parts of an OFDM receiver on the RaPiD architecture and have developed careful estimates of corresponding implementations in representative ASIC, DSP and FPGA technology. Our results show that, for this application, RaPiD fills the cost/performance gap between programmable DSP and ASIC architectures, achieving a factor of 6 better than a DSP implementation but a factor of 6 less than an ASIC implementation. [ABSTRACT FROM AUTHOR]

With the end of CPU core scaling due to dark silicon limitations, customized accelerators on FPGAs have gained increased attention in modern datacenters due to their lower power, high performance and energy efficiency. Evidenced by Microsoft's FPGA deployment in its Bing search engine and Intel's 16.7 billion acquisition of Altera, integrating FPGAs into datacenters is considered one of the most promising approaches to sustain future datacenter growth. However, it is quite challenging for existing big data computing systems-like Apache Spark and Hadoop-to access the performance and energy benefits of FPGA accelerators. In this paper we design and implement Blaze to provide programming and runtime support for enabling easy and efficient deployments of FPGA accelerators in datacenters. In particular, Blaze abstracts FPGA accelerators as a service (FaaS) and provides a set of clean programming APIs for big data processing applications to easily utilize those accelerators. Our Blaze runtime implements an FaaS framework to efficiently share FPGA accelerators among multiple heterogeneous threads on a single node, and extends Hadoop YARN with accelerator-centric scheduling to efficiently share them among multiple computing tasks in the cluster. Experimental results using four representative big data applications demonstrate that Blaze greatly reduces the programming efforts to access FPGA accelerators in systems like Apache Spark and YARN, and improves the system throughput by 1.7 × to 3× (and energy efficiency by 1.5× to 2.7×) compared to a conventional CPU-only cluster.

Heterogeneous parallel computing applications often process large data sets that require multiple GPUs to jointly meet their needs for physical memory capacity and compute throughput. However, the lack of high-level abstractions in previous heterogeneous parallel programming models force programmers to resort to multiple code versions, complex data copy steps and synchronization schemes when exchanging data between multiple GPU devices, which results in high software development cost, poor maintainability, and even poor performance. This paper describes the HPE runtime system, and the associated architecture support, which enables a simple, efficient programming interface for exchanging data between multiple GPUs through either interconnects or cross-node network interfaces. The runtime and architecture support presented in this paper can also be used to support other types of accelerators. We show that the simplified programming interface reduces programming complexity. The research presented in this paper started in 2009. It has been implemented and tested extensively in several generations of HPE runtime systems as well as adopted into the NVIDIA GPU hardware and drivers for CUDA 4.0 and beyond since 2011. The availability of real hardware that support key HPE features gives rise to a rare opportunity for studying the effectiveness of the hardware support by running important benchmarks on real runtime and hardware. Experimental results show that in a exemplar heterogeneous system, peer DMA and double-buffering, pinned buffers, and software techniques can improve the inter-accelerator data communication bandwidth by 2×. They can also improve the execution speed by 1.6× for a 3D finite difference, 2.5× for 1D FFT, and 1.6× for merge sort, all measured on real hardware. The proposed architecture support enables the HPE runtime to transparently deploy these optimizations under simple portable user code, allowing system designers to freely employ devices of different capabilities. We further argue that simple interfaces such as HPE are needed for most applications to benefit from advanced hardware features in practice.

In this work, we present a survey of the different parallel programming models and tools available today with special consideration to their suitability for high-performance computing. Thus, we review the shared and distributed memory approaches, as well as the current heterogeneous parallel programming model. In addition, we analyze how the partitioned global address space (PGAS) and hybrid parallel programming models are used to combine the advantages of shared and distributed memory systems. The work is completed by considering languages with specific parallel support and the distributed programming paradigm. In all cases, we present characteristics, strengths, and weaknesses. The study shows that the availability of multi-core CPUs has given new impulse to the shared memory parallel programming approach. In addition, we find that hybrid parallel programming is the current way of harnessing the capabilities of computer clusters with multi-core nodes. On the other hand, heterogeneous programming is found to be an increasingly popular paradigm, as a consequence of the availability of multi-core CPUs+GPUs systems. The use of open industry standards like OpenMP, MPI, or OpenCL, as opposed to proprietary solutions, seems to be the way to uniformize and extend the use of parallel programming models. [ABSTRACT FROM AUTHOR]

High Performance Linpack can maximize requirements throughout a computer system. An efficient multi-GPU double-precision general matrix multiply (DGEMM), together with adjustments to the HPL, is required to utilize a heterogeneous computer to its full extent. The authors present the resulting energy-efficiency measurements and suggest a cluster design that can utilize multiple GPUs. [ABSTRACT FROM PUBLISHER]