Search

Siguiendo la invitacion realizada por la ONU para este ano 2012 en su declaracion del Ano Internacional de la ENERGIA SOSTENIBLE PARA TODOS, las Universidades Libre, Santo Tomas y la Escuela Colombiana de Carreras Industriales (ECCI) con el apoyo logistico y academico de la ONG Akuaippa se organizo el “I Congreso de Energia Sostenible”. Siendo este un punto de encuentro entre organismos gubernamentales, academicos y empresariales del sector energetico, con el fin de presentar su vision y posicion sobre temas tan relevantes como lo son los desarrollos tecnologicos en el sector energetico y la sostenibilidad energetica a largo plazo. Asi pues, como lo declara la ONU, “La «energia sostenible» es aquella energia que se produce y se usa de forma que apoye a largo plazo el desarrollo humano en el ambito social, economico y ecologico”, de tal manera que es de importancia vital que en el contexto regional y mundial los investigadores colombianos se vinculen al diseno, desarrollo y analisis de factibilidad tecnico economica de sistemas de generacion de energia que permitan la sostenibilidad de la misma en la region, y que ello sea tambien en pro del desarrollo de las mas zonas y comunidades mas necesitadas. Entendiendo la importancia del tema, es tarea de la Universidad Colombiana, como centro de desarrollo tecnologico, proponer y promover un espacio para la discusion y la relexion desde diversos angulos acerca de su papel en el desarrollo de fuentes de energia y su impacto en el medio ambiente. Dicha vision, no ha sido esquiva para el enfoque del Congreso, en donde este se convierte en espacio de discusion y divulgacion de desarrollo en el tema energetico pero con responsabilidad ambiental y social en la generacion y uso racional de la energia.

A software stack relies primarily on graph-based methods to implement scalable resource description framework databases on top of commodity clusters, providing an inexpensive way to extract meaning from volumes of heterogeneous data.

An extended description of the RDESC use case with example metadata.An updated description of the GEMS software stack to reflect latest state.A more in-depth evaluation of GEMS' ability to answer science-based queries.A performance and scalability evaluation using the BSBM benchmark. Data-intensive science simultaneously derives from and creates the need for large quantities of data. As such, scientists increasingly need to discover and analyze new datasets from diverse sources. Beyond the sheer volume of data, issues posed by the resultant data heterogeneity are often overlooked. We postulate that heterogeneity challenges can be solved (at least in part) with the adoption of the Resource Description Framework (RDF), a graph-based data model. In turn, this requires scalable graph query systems for discovering and analyzing data. Consequently, we investigate GEMS, a graph engine for large-scale clusters. We describe the features of GEMS that make it suitable for answering graph queries and scaling to larger quantities of data. We evaluate GEMS' ability to answer real science-based queries over real-world, curated, science metadata. We also demonstrate GEMS' ability to scale to larger datasets using a benchmark.

This article presents SGEM, a full software system for accelerating large-scale graph databases on commodity clusters. Unlike current approaches, GEMS addresses graph databases by primarily employing graph-based methods, which is reflected at all levels of the stack. On the one hand, this allows exploiting the space efficiency of graph data structures and the inherent parallelism of some graph algorithms. These features adapt well to the increasing system memory and core counts of modern commodity clusters. On the other hand, these systems are optimized for regular computation and batched data transfers, whereas graph-based methods usually are irregular and generate fine-grained data accesses with poor spatial and temporal locality. The framework comprises a SPARQL-to-C++ compiler; a library of distributed data structures; and a custom, multithreaded, runtime system. The authors introduce their stack, discuss its advantages with respect to other solutions, and show how they overcame the challenges posed by irregular behaviors. They evaluated their software stack on the Berlin SPARQL benchmark with datasets of up to 10 billion graph edges, demonstrating scaling in dataset size and in performance as they added nodes to the cluster.

Irregular applications present unpredictable memory-access patterns, data-dependent control flow, and fine-grained data transfers. Only a holistic view spanning all layers of the hardware and software stack can provide effective solutions to address these challenges.

Graph matching is a prototypical combinatorial problem with many applications in high-performance scientific computing. Optimal algorithms for computing matchings are challenging to parallelize. Approximation algorithms are amenable to parallelization and are therefore important to compute matchings for large-scale problems. Approximation algorithms also generate nearly optimal solutions that are sufficient for many applications. In this paper we present multithreaded algorithms for computing half-approximate weighted matching on state-of-the-art multicore (Intel Nehalem and AMD Magny-Cours), manycore (Nvidia Tesla and Nvidia Fermi), and massively multithreaded (Cray XMT) platforms. We provide two implementations: the first uses shared work queues and is suited for all platforms; and the second implementation, based on dataflow principles, exploits special features available on the Cray XMT. Using a carefully chosen dataset that exhibits characteristics from a wide range of applications, we show scalable performance across different platforms. In particular, for one instance of the input, an R-MAT graph (RMAT-G), we show speedups of about [Formula: see text] on [Formula: see text] cores of an AMD Magny-Cours, [Formula: see text] on [Formula: see text] cores of Intel Nehalem, [Formula: see text] on Nvidia Tesla and [Formula: see text] on Nvidia Fermi relative to one core of Intel Nehalem, and [Formula: see text] on [Formula: see text] processors of Cray XMT. We demonstrate strong as well as weak scaling for graphs with up to a billion edges using up to 12,800 threads. We avoid excessive fine-tuning for each platform and retain the basic structure of the algorithm uniformly across platforms. An exception is the dataflow algorithm designed specifically for the Cray XMT. To the best of the authors' knowledge, this is the first such large-scale study of the half-approximate weighted matching problem on multithreaded platforms. Driven by the critical enabling role of combinatorial algorithms such as matching in scientific computing and the emergence of informatics applications, there is a growing demand to support irregular computations on current and future computing platforms. In this context, we evaluate the capability of emerging multithreaded platforms to tolerate latency induced by irregular memory access patterns, and to support fine-grained parallelism via light-weight synchronization mechanisms. By contrasting the architectural features of these platforms against the Cray XMT, which is specifically designed to support irregular memory-intensive applications, we delineate the impact of these choices on performance.

It is our view that the state of the art in constructing a large collection of graph algorithms in terms of linear algebraic operations is mature enough to support the emergence of a standard set of primitive building blocks. This paper is a position paper defining the problem and announcing our intention to launch an open effort to define this standard., Comment: 2 pages, IEEE HPEC 2013

We propose an algebraic framework for the design and implementation of a large class of data-sorting procedures, including all index-digit permutations used in FFTs. We discuss both old and new algorithms in terms of this framework. We show that the algebraic formulation of the new algorithms can be easily encoded using a functional programming language, and that the resulting code introduces no inefficiencies. We present performance results for implementations of three new algorithms for mixed-radix digit-reversal on a Cray C-90 and on a Sun Sparc 5.

Acting on time-critical events by processing ever growing social media, news or cyber data streams is a major technical challenge. Many of these data sources can be modeled as multi-relational graphs. Mining and searching for subgraph patterns in a continuous setting requires an efficient approach to incremental graph search. The goal of our work is to enable real-time search capabilities for graph databases. This demonstration will present a dynamic graph query system that leverages the structural and semantic characteristics of the underlying multi-relational graph., Comment: SIGMOD 2013: International Conference on Management of Data

We introduce a tensor sum which is useful for the design and analysis of digit-index permutations (DIPs) algorithms. Using this operation we obtain a new high-performance algorithm for the family of DIPs. We discuss an implementation in the applicative language Sisal and show how different choices of parameters yield different DIPs. The efficiency of the special case of digit reversal is illustrated with performance results on a Cray C-90.

This workshop, this year in its second edition, aims at bringing together scientists with all these different backgrounds to discuss, define and design methods and technologies for efficiently supporting irregular applications on current and future machines. The call for papers attracted 10 submissions from Asia, Europe and the United States. The program committee accepted 6 papers, organized in three sessions: architectures for irregular algorithms, using GPUs for Irregular Applications, and Programming Models for Irregular Applications. The program included two keynote speeches: "Big Data: OPs vs FLOPs" by Steve Wallach and "Irregular Applications and Their Architectural Challenges" by Dr. Pradeep Dubey. Finally, an outstanding panel of speakers (Umit Catalyurek, Pradeep Dubey, David Gleich, Alex Ramirez, Vinod Tipparaju, Steve Wallach) discussing current challenges and possible solutions for the efficient execution of irregular applications concluded the program. We hope that these proceedings will serve as a valuable reference for researchers and developers in the field of irregular applications.

Triadic analysis encompasses a useful set of graph mining methods that are centered on the concept of a triad, which is a sub graph of three nodes. Such methods are often applied in the social sciences as well as many other diverse fields. Triadic methods commonly operate on a triad census that counts the number of triads of every possible edge configuration in a graph. Like other graph algorithms, triadic census algorithms do not scale well when graphs reach tens of millions to billions of nodes. To enable the triadic analysis of large-scale graphs, we developed and optimized a triad census algorithm to efficiently execute on shared memory architectures. We then conducted performance evaluations of the parallel triad census algorithm on three specific systems: CrayXMT, HP Superdome, and AMD multi-core NUMA machine. These three systems have shared memory architectures but with markedly different hardware capabilities to manage parallelism.

Graph pattern matching involves finding exact or approximate matches for a query subgraph in a larger graph. It has been studied extensively and has strong applications in domains such as computer vision, computational biology, social networks, security and finance. The problem of exact graph pattern matching is often described in terms of subgraph isomorphism which is NP-complete. The exponential growth in streaming data from online social networks, news and video streams and the continual need for situational awareness motivates a solution for finding patterns in streaming updates. This is also the prime driver for the real-time analytics market. Development of incremental algorithms for graph pattern matching on streaming inputs to a continually evolving graph is a nascent area of research. Some of the challenges associated with this problem are the same as found in continuous query (CQ) evaluation on streaming databases. This paper reviews some of the representative work from the exhaustively researched field of CQ systems and identifies important semantics, constraints and architectural features that are also appropriate for HPC systems performing real-time graph analytics. For each of these features we present a brief discussion of the challenge encountered in the database realm, the approach to the solution and state their relevance in a high-performance, streaming graph processing framework.graph of Gd and vertices of Gq such that all vertex adjacencies are preserved. Dynamic graphs refer to graphs that evolve over time through addition or deletion of vertices and edges. Therefore, the problem of graph pattern matching for dynamic graphs can be described as the continuous process of searching for patterns in the graph as it is updated. News [23], finance [7], cyber security and intelligence [10] are among the primary domains that drive the real-time analytics market [1, 19] and motivate development of HPC systems. These domains present data sources that lend themselves naturally to a graph based representation and additionally, provide semantic information in terms of types, labels and timestamps, which can be more generally described as attributes of the vertices and edges of the graph. The availability of the attributes influence the isomorphism computation because assigning a correspondence between a pair of vertices in the search and query graph requires them to satisfy equality constraints on type and possibly, other attributes as well. All these domains are also characterized by massive streaming data that are continuously providing updates from social networks, financial markets and malicious activities on the internet with a high emphasis on time-to-insight, the capability of learning about an event as soon as it happens. This motivates

Program partitioning and scheduling are essential steps in programming non-shared-memory computer systems. Partitioning is the separation of program operations into sequential tasks, and scheduling is the assignment of tasks to processors. To be effective, automatic methods require an accurate representation of the model of computation and the target architecture. Current partitioning methods assume today′s most prevalent models - macro dataflow and a homogeneous/two-level multicomputer system. Based on communication channels, neither model represents well the emerging class of NUMA multiprocessor computer systems consisting of hierarchical read/write memories. Consequently, the partitions generated by extant methods do not execute well on these systems. In this paper, we extend the conventional graph representation of the macro-dataflow model to enable mapping heuristics to consider the complex conununication options supported by NUMA architectures. We describe two such heuristics. Simulated execution times of program graphs show that our model and heuristics generate higher quality program mappings than current methods for NUMA architectures.

Two of the most commonly used hashing strategies—linear probing and hashing with chaining—are adapted for efficient execution on a Cray XMT. These strategies are designed to minimize memory contention. Datasets that follow a power law distribution cause significant performance challenges to shared memory parallel hashing implementations. Experimental results show good scalability up to 128 processors on two power law datasets with different data types: integer and string. These implementations can be used in a wide range of applications.

Sisal (Streams and Iterations in Single Assignment Language) is a general-purpose applicative language intended for use on both conventional and novel multiprocessor systems. In this report we discuss the project's objectives, philosophy, and accomplishments and state our future plans. Four significant results of the Sisal project are compilation techniques for high-performance parallel applicative computation, a microtasking environment that supports dataflow on conventional shared-memory architectures, execution times comparable to those of Fortran, and cost-effective speedup on shared-memory multiprocessors.

The problem of counting specified combinations of a given set of variables arises in many statistical and data mining applications. To solve this problem, we introduce the PDtree data structure, which avoids exponential time and space complexity associated with prior work by allowing user specification of the tree structure. A straightforward parallelization approach using a Cray MTA-2 provides a speedup that is linear in the number of processors, but introduces nondeterminism into probability estimates. We prove a general convergence result that bounds the non-deterministic deviation of probability estimates relative to a sequential implementation. Beyond PDtrees, this convergence result applies to any counting application that takes a multithreaded streaming approach.

We analyze the parallel performance of randomized interpolative decomposition by decomposing low rank complex-valued Gaussian random matrices of about 100 GB. We chose a Cray XMT supercomputer as it provides an almost ideal PRAM model permitting quick investigation of parallel algorithms without obfuscation from hardware idiosyncrasies. We obtain that on non-square matrices performance scales almost linearly with runtime about 100 times faster on 128 processors. We also verify that numerically discovered error bounds still hold on matrices two orders of magnitude larger than those previously tested.

Molecular mechanics applications model the interactions among large ensembles of discrete particles. They are used where probabilistic methods are inadequate, such as drug chemistry. This methodology is difficult to parallelize with good performance, due to its poor locality, uneven partitions, and dynamic behavior. Imperative programs have been written that attempt this on shared and distributed memory machines. Given such a program, the computational kernel can be rewritten in Sisal, a functional programming language, and integrated with the rest of the imperative program under the Sisal Foreign Language Interface. This allows minimal effort and maximal return from parallelization work, and leaves the work appropriate to imperative implementation in its original form. We describe such an effort, focusing on the parts of the application that are appropriate for Sisal implementation, the specifics of mixed-language programming, and the complex performance behavior of the resulting hybrid code.

A general overlapping-plate reduction program, REDUCE, has been written for the reduction of astrometric data from the Edwin P. Hubble Space Telescope. Pains have been taken to make this program powerful and flexible, yet easy to use. This paper describes the main features of the program.