Search

Forward flux sampling (FFS) is an established scientific method for sampling rare events in molecular simulations. However, as the difficulty of the scientific problem increases, the amount of data and the number of tasks required for FFS is challenging to manage with traditional scripting tools and languages for high performance computing. The SAFFIRE software framework has been developed to address these challenges. SAFFIRE utilizes Hadoop to manage a large number of tasks and data for large scale FFS simulations. The framework is shown to be highly scalable and able to support large scale FFS simulations. This enables studies of rare events in complex molecular systems on commodity cluster computing systems.

This study details the progress in transportation data analysis with a novel computing framework in keeping with the continuous evolution of the computing technology. The computing framework combines the Labeled Latent Dirichlet Allocation (L-LDA)-incorporated Support Vector Machine (SVM) classifier with the supporting computing strategy on publicly available Twitter data in determining transportation-related events to provide reliable information to travelers. The analytical approach includes analyzing tweets using text classification and geocoding locations based on string similarity. A case study conducted for the New York City and its surrounding areas demonstrates the feasibility of the analytical approach. Approximately 700,010 tweets are analyzed to extract relevant transportation-related information for one week. The SVM classifier achieves >85% accuracy in identifying transportation-related tweets from structured data. To further categorize the transportation-related tweets into sub-classes: incident, congestion, construction, special events, and other events, three supervised classifiers are used: L-LDA, SVM, and L-LDA incorporated SVM. Findings from this study demonstrate that the analytical framework, which uses the L-LDA incorporated SVM, can classify roadway transportation-related data from Twitter with over 98.3% accuracy, which is significantly higher than the accuracies achieved by standalone L-LDA and SVM.

Transportation is among the fields that can potentially benefit, and are under the pressure of, the data deluge the most. This comes from technological advances that improve both the ability to generate data from sensors and personal devices as well as the network capability that can make these data available for analytical purposes quickly. This chapter discusses techniques required for the first step in dealing with massive and complex data, which is to curate and integrate data, which can be under different formats and arrive from various sources. These techniques allow readers to explore transportation data under the most common format. In addition, this chapter will also introduce the R statistical programming framework, a popular open source framework with powerful support for statistical analysis.

The concept of Internet of Things (IoT) is rapidly moving from a vision to being pervasive in our everyday lives. This can be observed in the integration of connected sensors from a multitude of devices such as mobile phones, healthcare equipment, and vehicles. There is a need for the development of infrastructure support and analytical tools to handle IoT data, which are naturally big and complex. But, research on IoT data can be constrained by concerns about the release of privately owned data. In this paper, we present the design and implementation results of a synthetic IoT data generation framework. The framework enables research on synthetic data that exhibit the complex characteristics of original data without compromising proprietary information and personal privacy.

Academic data can be classified into multiple categories and come from a large number of sources. Many research areas require combining data from different sources into a unified set on which analytical techniques can be applied. In this research paper the authors introduce the High Performance Computing Cluster (HPCC) as a platform to streamline the process of ingesting, curating, integrating and transforming scholarly data from multiple sources and in varying formats, particularly when several of these datasets lack common attributes to support the integration process.

This paper presents the development of a Hadoop MapReduce module that has been taught in a course in distributed computing to upper undergraduate computer science students at Clemson University. The paper describes our teaching experiences and the feedback from the students over several semesters that have helped to shape the course. We provide suggested best practices for lecture materials, the computing platform, and the teaching methods. In addition, the computing platform and teaching methods can be extended to accommodate emerging technologies and modules for related courses.

In this paper, we present JUMMP, the Job Uninterrupted Maneuverable MapReduce Platform, an automated scheduling platform that provides a customized Hadoop environment within a batch-scheduled cluster environment. JUMMP enables an interactive pseudo-persistent MapReduce platform within the existing administrative structure of an academic high performance computing center by “jumping” between nodes with minimal administrative effort. Jumping is implemented by the synchronization of stopping and starting daemon processes on different nodes in the cluster. Our experimental evaluation shows that JUMMP can be as efficient as a persistent Hadoop cluster on dedicated computing resources, depending on the jump time. Additionally, we show that the cluster remains stable, with good performance, in the presence of jumps that occur as frequently as the average length of reduce tasks of the currently executing MapReduce job. JUMMP provides an attractive solution to academic institutions that desire to integrate Hadoop into their current computing environment within their financial, technical, and administrative constraints.

This research utilizes several well known forecasting techniques in combination with Empirical Mode Decomposition (EMD) to investigate the trade-offs of EMD's decomposition (sifting) step for forecasting the arrival workload of an enterprise cluster. The research is based on earlier work on the forecasting potential of EMD. Results show that EMD helps to improve forecasting results. Parallelization is used to perform extensive investigation across the full range of data. Future research is to increase the statistical confidence in the level of improvements possible when EMD is used as a decomposition method for forecasting.

Modeling has long been recognized as an invaluable tool for predicting the performance behavior of computer systems. Modeling software, both commercial and open source, is widely used as a guide for the development of new systems and the upgrading of exiting ones. Tools such as queuing network models, stochastic Petri nets, and event driven simulation are in common use for stand-alone computer systems and networks. Unfortunately, no set of comprehensive tools exists for modeling complex distributed computing environments such as the ones found in emerging grid deployments. With the rapid advance of grid computing, the need for improved modeling tools specific to the grid environment has become evident. This chapter addresses concepts, methodologies, and tools that are useful when designing, implementing, and tuning the performance in grid and cluster environments

Shibboleth is an architecture and protocol for allowing users to authenticate and be authorized to use a remote resource by logging into the identity management system that is maintained at their home institution. With Shibboleth, a federation of institutions can share resources among users and yet allow the administration of both the user access control to resources and the user identity and attribute information to be performed at the hosting or home institution. Subversion is a version control repository system that allows the creation of fine-grained permissions to files and directories. In this project an infrastructure, Shibbolized Subversion, has been created that consists of a Subversion repository with an Apache web interface that is protected by a Shibboleth authentication system. The infrastructure can allow authorized and authenticated data sharing between institutions yet retains simplicity and protects privacy for users. In addition, it also relieves local administrators from the task of having to perform extra account management for users from other institutions. This paper describes the Shibboleth and Subversion systems, the implementation of the file sharing infrastructure, and issues of attribute maintenance, privacy and security.

A version control repository structure has been created based on open source utility software. The structure consists of a subversion repository with an Apache Web interface that is protected by a Shibboleth authentication system. This structure can allow authorized and authenticated data sharing between institutions, yet retains simplicity and protects privacy for users. In addition, it also helps local administrators from having to perform extra account management for new users from other institutions

We are developing interactive web pages for distance learning that use Macromedia Flash animation in undergraduate electrical engineering courses in circuits and signals and systems. We have animated key concepts such as steady-state analysis, time scale, time reversal, and convolution to allow students to better visualize these ideas. We are also developing ways to allow student input and immediate feedback in problem solving. The interactive web pages will be formally evaluated this year.

An approximate model for the queuing analysis of the SR-ARQ scheme is proposed. The model is analysed using the matrix-analytic approach. Extensions of the model for the system with correlated block errors and for the system using a hybrid FEC/SR-ARQ scheme are also considered. Numerical examples compare the approximate analyses with exact analyses and simulated results.