Search

This article outlines a set of plug-in tools running on MATLAB to automatically import, preprocess, and evaluate the quality of electro-encephalography (EEG) data stored using the Brain Imaging Data Structure BIDS-EEG standard. As a proof of concept, we apply several possible data quality metrics to 30 EEG studies (sets of compatible datasets) currently available in BIDS-EEG format on the OpenNeuro.org platform. The bids-matlab-tools plug-in for EEGLAB (sccn.ucsd.edu/EEGLAB) checks for the presence of information needed for analysis, then applies preprocessing pipelines that compute data quality metrics and retain dataset portions (e.g., those channels and time points) that appear suitable for analysis. These and other data measures and visualizations will be made available to the EEG community through a new NEMAR.org web portal now under development to support the use of the OpenNeuro.org data archive by the human electrophysiology research community.

This work looks at analyzing I/O traffic of users’ jobs on a HPC machine for a period of time. Monitoring tools are collecting the data in a continuous basis on the HPC system. We looked at aggregate I/O data usage patterns of users’ jobs on the system both on the parallel shared Lustre file system and the node-local SSDs. Data mining tools are then applied to analyze the I/O usage pattern data in an attempt to tie the data to particular codes that produced those I/O behaviors from users’ jobs.

The Comet petascale supercomputer was put into production as an XSEDE resource in early 2015 with the goal of serving a much larger user community than HPC systems of similar size. The Comet project set an audacious goal of reaching over 10,000 users in its four years of planned operation. That goal was achieved in less than two years, due in large part to the adoption of policies that favor smaller allocations and science gateways. Here we describe our experiences in operating and supporting Comet, highlight some of the important science that it has enabled, and provide some practical lessons that we have learned by operating a system designed for the long tail of science.

Biomimetic simulation permits neuroscientists to better understand the complex neuronal dynamics of the brain. Embedding a biomimetic simulation in a closed-loop neuroprosthesis, which can read and write signals from the brain, will permit applications for amelioration of motor, psychiatric, and memory-related brain disorders. Biomimetic neuroprostheses require real-time adaptation to changes in the external environment, thus constituting an example of a dynamic data-driven application system. As model fidelity increases, so does the number of parameters and the complexity of finding appropriate parameter configurations. Instead of adapting synaptic weights via machine learning, we employed major biological learning methods: spike-timing dependent plasticity and reinforcement learning. We optimized the learning metaparameters using evolutionary algorithms, which were implemented in parallel and which used an island model approach to obtain sufficient speed. We employed these methods to train a cortical spiking model to utilize macaque brain activity, indicating a selected target, to drive a virtual musculoskeletal arm with realistic anatomical and biomechanical properties to reach to that target. The optimized system was able to reproduce macaque data from a comparable experimental motor task. These techniques can be used to efficiently tune the parameters of multiscale systems, linking realistic neuronal dynamics to behavior, and thus providing a useful tool for neuroscience and neuroprosthetics.

Big Data problems dealing with a variety of large data sets are now common in a wide range of domain science research areas such as bioinformatics, social science, astronomical image processing, weather and climate dynamics, and economics. In some cases, the data generation and computation is done on high performance computing (HPC) resources, thus presenting an incentive for developing/optimizing big data middleware and tools to take advantage of the existing HPC infrastructures. Data-intensive computing middleware (such as Hadoop, Spark) can potentially benefit greatly from the hardware already designed for high performance and scalability with advanced processor technology, large memory/core, and storage/filesystems. SDSC Comet represents such a resource with large numbers of compute nodes with fast node local SSD storage, and high performance Lustre filesystems. This paper discusses experiences and benefits of using optimized Remote Direct Memory Access (RDMA) Hadoop and Spark middleware on the XSEDE Comet HPC resource, including some performance results of Big Data benchmarks and applications. Comet is a general purpose HPC resource so some work is needed to integrate the middleware to run within the HPC scheduling framework. This aspect of the implementation is also discussed in detail.

Here we describe the CIPRES Workbench (CW), an open source software framework for creating new science gateways with minimal overhead. The CW is a web application that can be deployed on a modest server, and can be configured to submit command line instructions to any resource where the application has submission privileges. It is designed to be highly configurable/customizable, and supports GUI-based access to HPC resources through a web browser interface as well as programmatic access via a ReSTful API. Using browser access, the CW architecture creates an environment with secure user accounts where user input data, job results, and job provenance are stored. The ReSTful API allows users with a registered a client application to deliver command lines to analytical codes and retrieve results from remote compute resources. A development effort is underway to allow the CW to submit jobs via the Science Gateways as a Platform (SciGaP) services hosted at Indiana University.

NSF-funded computing centers have primarily focused on delivering high-performance computing resources to academic researchers with the most computationally demanding applications. But now that computational science is so pervasive, there is a need for infrastructure that can serve more researchers and disciplines than just those at the peak of the HPC pyramid. Here we describe SDSC's Comet system, which is scheduled for production in January 2015 and was designed to address the needs of a much larger and more expansive science community-- the "long tail of science". Comet will have a peak performance of 2 petaflop/s, mostly delivered using Intel's next generation Xeon processor. It will include some large-memory and GPU-accelerated nodes, node-local flash memory, 7 PB of Performance Storage, and 6 PB of Durable Storage. These features, together with the availability of high performance virtualization, will enable users to run complex, heterogeneous workloads on a single integrated resource.

We present the Neuroscience Gateway (NSG) Portal http://www.nsgportal.org/, which we are developing so that neuroscientists can easily use High Performance Computing (HPC) resources for computationally intensive modeling tasks such as simulating large-scale networks or exploring high-dimensional parameter spaces. Potential users of HPC resources typically face a high entry barrier because of numerous complex side-issues, such as applying for CPU time, installing and configuring simulation software, remote authentication, data transfer and retrieval, batch system management, and administrative policies--all of which may vary from one site to another. The NSG Portal reduces this entry barrier by streamlining administrative procedures and providing a convenient web browser-based interface that hides technical details, and thereby enables investigators to focus on scientific issues relevant to their research. Brian, NEST, NEURON, PGENESIS, and PyNN are now installed and available through the Portal; suggestions for other tools are welcome. NSG's entry point is a simple browser-based interface through which users can upload input files or model source code, specify code-specific input files, specify job submission parameters (such as number of cores and nodes, expected wall clock time for job completion), monitor the status of submitted jobs, and extract and download output files. Files may be uploaded as plain text, and collections of files in a pkzip archive. Job results are downloadable as a pkzip archive of that job's working directory, including input and output files. Users are notified of job completion by email. We have been obtaining time allocations for the NSG Portal on HPC resources from XSEDE, an NSF-supported partnership of academic institutions whose goal is to facilitate sharing of computing resources, data, and expertise. Through XSEDE's national-level peer review process, our application for HPC time was awarded 1.6 million "service units" (~ 1.6 million CPU hours) for the current year. From this, each NSG user is given an initial allotment of 5000 core hours per year, subject to adjustment in light of actual usage by all NSG users. Those who have independently obtained their own HPC allocations may manage that time via the NSG.

In this paper, we describe the neuroscience gateway (NSG), which facilitates access to high performance computing resources for computational neuroscientists. Through a simple web-based portal, the NSG provides a streamlined environment for uploading models, specifying HPC job parameters, querying running job status, receiving job completion notices, and storing and retrieving output data. The NSG architecture transparently distributes user jobs to appropriate HPC resources available through the XSEDE organization.

This paper is intended to explain how the Tera Grid would like to be able to measure "usage modalities." We would like to (and are beginning to) measure these modalities to understand what objectives our users are pursuing, how they go about achieving them, and why, so that we can make changes in the Tera Grid to better support them.

The emerging cyberinfrastructure holds the promise of providing on-demand access to high performance network, compute and data resources. Image guided neurosurgery is one of many applications that requires such on-demand access to resources. In this paper, we have studied the feasibility of accessing such resources on-demand. An experiment was designed and carried out across five TeraGrid clusters for this study. This paper provides an analysis of the results and draws some conclusion regarding feasibility of on-demand access to high performance resources.

Multithreading has received considerable attention in recent years as a promising way to hide memory latency in high performance computers, while providing access to a large and uniform shared memory. Tera Computer of Seattle has designed and built a state of the art multithreaded computer called the MTA. Its intended benefits are high processor utilization, scalable performance on applications that are difficult to parallelize, and reduced programming effort. The largest MTA and the only one outside of Seattle is at the San Diego Supercomputer Center (SDSC) on the campus of the University of California, San Diego (UCSD). Currently the MTA at SDSC has 8 processors. The performance and usability of the MTA for 14 defense relevant applications were evaluated in a two year project described here. The applications included seven standard kernels, five mini-applications, and two large applications. The evaluation was led by researchers at UCSD with collaborators at Caltech, Tera, Boeing, and Sanders/Lockheed Martin. UCSD researchers also carried out multithreaded scheduler and compiler studies. The principal findings of the project follow in the enclosed final report.

Purpose: Graphics Processing Unit (GPU) has become increasingly important in radiotherapy. However, it is still difficult for general clinical researchers to access GPU codes developed by other researchers with GPU expertise, and also for developers to benchmark their codes. It is quite often to see repeated efforts spent on developing GPU codes with low quality. The goal of this project is to establish an infrastructure for sharing GPU codes in the community. Methods: We deployed a GPU code sharing infrastructure on a GPU cluster. A number of codes developed in our group can be accessed via a web interface. To use the services, researchers first upload their test data or use the data provided by our system. Then they have to select the GPU device they are going to run the codes. Our system offers all mainstream GPU hardware for code benchmarking purpose. After the code running is complete, the system will automatically summarize and display the computing results. We will also release a SDK to allow the developers to build their own algorithm implementation and submit their binary codes to the system. The submitted code will be systematically benchmarked using a variety of GPU hardware and representative clinical data provided by our system. Results: This project provides a platform to the public to access a variety of GPU codes for radiotherapy research via a web interface. With the help of this platform, researchers are able to focus their efforts on clinical research. Developers will also benefit from this platform by benchmarking their codes on various GPU platforms and clinical data sets and comparing with other people's codes for the same application. Conclusion: The gateway for GPU code developers and clinical research users can greatly facilitate the adoption of GPU codes in radiotherapy.

Purpose: To develop and evaluate a new DRR algorithm on GPU and compare it's accuracy and efficiency to two other commonly used DRR algorithms. Methods: We developed a new fixed grid algorithm optimized for GPU. This new algorithm first rotates and re‐samples CT data layer by layer and then traces rays through voxel slabs perpendicular to the projection axis. We also implemented the fixed stride algorithm and Siddon's incremental algorithm on GPU. We compared the accuracy of each algorithm to a ground truth that was generated by sampling 1024 evenly spaced ray paths per pixel using Siddon's incremental algorithm. We then computed the relative error for 36 equally spaced projections in a full rotation about the SI axis for 5 different voxel resolutions (ranging from 512 to 32 cubed). We compared the efficiency of each algorithm by simulating 360 projections, one per degree, of a clinically sized phantom (512×512×128) in a full rotation about the SI axis. Both tests simulated common CBCT geometry with a 40.0×30.0 cm, 1024×768 pixel detector. XCAT phantoms were used in all tests. Results: We found that all three DRR algorithms attain small error as a function of decreasing voxel size ranging from 10% at 16.0 mm/voxel to less than 0.2% at 1.0 mm/voxel. We also found the average projection time for the fixed grid algorithm is 21.8 ms/projection, 40% faster than Siddon's incremental algorithm which had an average projection time of 36.5 ms. The fixed stride algorithm is 18% faster than Siddon's incremental algorithm with an average projection time of 28.7 ms. Conclusion: By specifically designing a DRR algorithm for GPU, we were able to improve the computation efficiency without losing accuracy. This work will be useful for computational tasks requiring frequent rendering of DRR projections.