Search

The U.S. National Science Foundation (NSF) has invested heavily in research computing, funding the XSEDE network of supercomputers and enabling their integration with science gateways. To ensure maximal return on this substantial investment, it is essential to monitor the use of these computing resources. XD Metrics on Demand (XDMoD) is an NSF-funded tool that was developed to help manage high performance computational resources. XDMoD metrics describe accounting and performance data for computational jobs, including resources consumed, wait times, and quality of service. XDMoD can provide information on individual jobs, or data aggregated over an ensemble of jobs. Its web interface offers centralized charting, exploration, and reporting of these metrics, for user-selected time ranges, and across resources. XDMoD is directly relevant to the gateways community. In this paper, we introduce XDMoD, demonstrate its utility for gateways, and outline our plans to further enhance its capabilities for the gateways community. Furthermore, we demonstrate how XDMoD can help gateway users and gateway and resource managers answer many questions about utilization, performance, and availability. In doing so, we showcase the evolution of gateways in the XSEDE ecosystem.

Purpose Significant cancer-related distress affects 30–60% of women diagnosed with breast cancer. Fewer than 30% of distressed patients receive psychosocial care. Unaddressed distress is associated with poor treatment adherence, reduced quality of life, and increased healthcare costs. This study aimed to evaluate the preliminary efficacy of a new web-based, psychoeducational distress self-management program, CaringGuidance™ After Breast Cancer Diagnosis, on newly diagnosed women’s reported distress. Methods One-hundred women, in five states, diagnosed with breast cancer within the prior 3 months, were randomized to 12 weeks of independent use of CaringGuidance™ plus usual care or usual care alone. The primary multidimensional outcome, distress, was measured with the Distress Thermometer (DT), the Center for Epidemiologic Studies Depression Scale (CES-D), and the Impact of Events Scale (IES) at baseline and months 1, 2, and 3. Intervention usage was continually monitored by the data analytic system imbedded within CaringGuidance™. Results Although multilevel models showed no significant overall effects, post hoc analysis showed significant group differences in slopes occurring between study months 2 and 3 on distress (F(1,70) = 4.91, p = .03, η2 = .065) measured by the DT, and depressive symptoms (F(1, 76) = 4.25, p = .043, η2 = .053) favoring the intervention. Conclusions Results provide preliminary support for the potential efficacy of CaringGuidance™ plus usual care over usual care alone on distress in women newly diagnosed with breast cancer. This analysis supports and informs future study of this self-management program aimed at filling gaps in clinical distress management.

Objective Limited clinical resources create barriers to quality management of cancer-related distress. CaringGuidance After Breast Cancer Diagnosis is a web-based, patient-controlled, psychoeducational program of cognitive-behavioral, coping and problem-solving strategies aimed at early post-diagnosis distress reduction without clinical resources. This study evaluated the feasibility of recruiting and retaining newly diagnosed women to 12 weeks of CaringGuidance and program acceptance. Methods Women with stage 0 to II breast cancer diagnosed within the prior 3 months were recruited from clinics and communities in four states, from 2013 to 2015 and randomized to 12 weeks of CaringGuidance plus usual care (n = 57) or usual care alone (n = 43). Recruitment, retention, and program use were tracked. Using standard and study-derived measures, demographic and psychological variables were assessed at baseline and monthly and program satisfaction at 12 weeks. Results Of 139 women screened, 100 enrolled, five withdrew, and 12 were lost to follow-up (83% retention rate). Total program engagement was positively associated with greater baseline intrusive/avoidant thoughts. Intervention participants (92%) believed CaringGuidance would benefit future women and was easy to use. Sixty-six percent believed CaringGuidance helped them cope. Women used program content to change thoughts (49%) or behaviors (40%). Stress in the previous year was positively associated with reports that CaringGuidance was reassuring and helpful. Conclusions Feasibility and acceptance of CaringGuidance was demonstrated pointing to the program's potential as a cancer-distress self-management intervention. Future research will explore program feasibility and acceptability in other regions of the United States, leading to clinical implementation trials.

Purpose: Social constraints are interactions between individuals that result in preventing one’s disclosure of thoughts and emotions needed to facilitate cognitive processing of a traumatic event s...

High performance computing workloads often benefit from human in the loop interactions. Steps in complex pipelines ranging from quality control to parameter adjustments are critical to the successful and efficient completion of modern problems. We give several example workflows in bioinformatics and deep learning where computing decisions are made throughout the processing pipelines ultimately changing the course of the compute. We also show how users can interact with the pipeline using Open OnDemand plus XDMoD or Plot.ly.

The REDfly database provides a comprehensive curation of experimentally-validated Drosophila transcriptional cis-regulatory elements and includes information on DNA sequence, experimental evidence, patterns of regulated gene expression, and more. Now in its thirteenth year, REDfly has grown to over 23 000 records of tested reporter gene constructs and 2200 tested transcription factor binding sites. Recent developments include the start of curation of predicted cis-regulatory modules in addition to experimentally-verified ones, improved search and filtering, and increased interaction with the authors of curated papers. An expanded data model that will capture information on temporal aspects of gene regulation, regulation in response to environmental and other non-developmental cues, sexually dimorphic gene regulation, and non-endogenous (ectopic) aspects of reporter gene expression is under development and expected to be in place within the coming year. REDfly is freely accessible at http://redfly.ccr.buffalo.edu, and news about database updates and new features can be followed on Twitter at @REDfly_database.

Science Gateways have gained wide use by research community by providing a rich and easy to use web interface to access and utilize complex cyberinfrastructure. Science Gateways are consuming an increasing proportion of computational capacity provided by XSEDE. A typical approach employed by Science Gateways is to use a single community account with a compute allocation to process compute jobs on behalf of their users. The computation usage for Science Gateways is compiled from batch job submission systems and reported by the XSEDE service providers. However, this reporting does not capture and provide any information about the user who actually initiated the computation, as the batch systems do not have record this information. To overcome this reporting limitation, Science Gateways utilize a separate pipeline to submit job-specific attributes to XSEDE, which is then later co-joined with batch system information submitted by the Service Providers to create detailed usage reports. In this article we describe improvements to the Gateway attribute reporting system, which better serves the needs of the growing number of Science Gateways by providing their operators a simpler and versatile interface to easily report a richer set of attributes and ultimately publish this information via XDMoD that enables various stake holders to assess resource utilization and their potential impact.

Open OnDemand is an open source project designed to lower the barrier to HPC use across many diverse disciplines. Here we describe the main features of the platform, give several use cases of Open OnDemand and discuss how we measure success. We end the paper with a discussion of the future project roadmap.

The XD Metrics on Demand (XDMoD) tool was designed to collect and present detailed utilization data from the XSEDE network of supercomputers. XDMoD displays a wealth of information on computational resources. Its metrics report accounting and performance figures for computational jobs, including resources consumed, wait times, and quality of service. XDMoD can aggregate this information over an ensemble of jobs, or report it for individual jobs. Its web-based interface supports charting, exploration, and reporting for any time range, across all computing resources. Over the last eight years, we have open-sourced XDMoD, generalizing and packaging it to produce Open XDMoD. Open XDMoD may be installed, configured, and run on any computing cluster, and we have extended and refined it to include support for storage, cloud resources, science gateways, and more. In this paper, we describe the federation of XDMoD instances. This new functionality makes it possible to associate and monitor networks of related resources, using the full power of XDMoD to aggregate and display data from disparate XDMoD installations. Charting and reports from XDMoD federations can encompass anything ranging from a single resource to the entire network. The federation capability is flexible; federated resources can be coupled loosely or tightly, and a variety of different information sharing and authentication paradigms are supported. We also highlight new functionality that incorporates storage and cloud facilities into XDMoD federations. Federation will enable XDMoD users to monitor and manage ever more diverse computing resources.

We use the bibliometrics approach to evaluate the scientific impact of XSEDE. By utilizing publication data from various sources, e.g., ISI Web of Science and Microsoft Academic Graph, we calculate the impact metrics of XSEDE publications and show how they compare with non-XSEDE publication from the same field of study, or non-XSEDE peers from the same journal issue. We explain the dataset and data soruces involved and how we retrieved, cleaned, and curated millions of related publication entries. We then introduce the metrics we used for evaluation and comparison, and the methods used to calculate them. Detailed analysis results of Field Weighted Citation Impact (FWCI) and the peers comparison will be presented and discussed. We also explain how the same approaches could be used to evaluate publications from a similar organization or institute, to demonstrate the general applicability of the present evaluation approach providing impact even beyond XSEDE.

A Slurm simulator was used to study the potential benefits of using multiple Slurm controllers and node-sharing on the TACC Stampede 2 system. Splitting a large cluster into smaller sub-clusters with separate Slurm controllers can offer better scheduling performance and better responsiveness due to an increased computational capability which increases the backfill scheduler efficiency. The disadvantage is additional hardware, more maintenance and an incapability to run jobs across the sub-clusters. Node sharing can increase system throughput by allowing several sub-node jobs to be executed on the same node. However, node sharing is more computationally demanding and might not be advantageous on larger systems. The Slurm simulator allows an estimation of the potential benefits from these configurations and provides information on the advantages to be expected from such a configuration deployment. In this work, multiple Slurm controllers and node-sharing were tested on a TACC Stampede 2 system consisting of two distinct node types: 4,200 Intel Xeon Phi Knights Landing (KNL) nodes and 1,736 Intel Xeon Skylake-X (SLX) nodes. For this system utilization of separate controllers for KNL and SLX nodes with node sharing allowed on SLX nodes resulted in a 40% reduction in waiting times for jobs executed on the SLX nodes. This improvement can be attributed to the better performance of the backfill scheduler. It scheduled 30% more SLX jobs, has a 30% reduction in the fraction of cycles that hit the time-limit and nearly doubles the jobs scheduling attempts.

Slurm is an open-source resource manager for HPC that provides high configurability for inhomogeneous resources and job scheduling. Various Slurm parametric settings can significantly influence HPC resource utilization and job wait time, however in many cases it is hard to judge how these options will affect the overall HPC resource performance. The Slurm simulator can be a very helpful tool to aid parameter selection for a particular HPC resource. Here, we report our implementation of a Slurm simulator and the impact of parameter choice on HPC resource performance. The simulator is based on a real Slurm instance with modifications to allow simulation of historical jobs and to improve the simulation speed. The simulator speed heavily depends on job composition, HPC resource size and Slurm configuration. For an 8000 cores heterogeneous cluster, we achieve about 100 times acceleration, e.g. 20 days can be simulated in 5 h. Several parameters affecting job placement were studied. Disabling node sharing on our 8000 core cluster showed a 45% increase in the time needed to complete the same workload. For a large system (>6000 nodes) comprised of two distinct sub-clusters, two separate Slurm controllers and adding node sharing can cut waiting times nearly in half.

Application kernels are computationally lightweight benchmarks or applications run repeatedly on high performance computing HPC clusters in order to track the Quality of Service QoS provided to the users. They have been successful in detecting a variety of hardware and software issues, some severe, that have subsequently been corrected, resulting in improved system performance and throughput. In this work, the application kernels performance monitoring module of eXtreme Data Metrics on Demand XDMoD is described. Through the XDMoD framework, the application kernels have been run repetitively on the Texas Advanced Computing Center's Stampede and Lonestar4 clusters for a total of over 14,000 jobs. This provides a body of data on the HPC clusters operation that can be used to statistically analyze how the application performance, as measured by metrics such as execution time and communication bandwidth, is affected by the cluster's workload. We discuss metric distributions, carry out regression and correlation analyses, and use a PCA study to describe the variance and relate the variance to factors such as the spatial distribution of the application in the cluster. Ultimately, these types of analyses can be used to improve the application kernel mechanism, which in turn results in improved QoS of the HPC infrastructure that is delivered to the end users. Copyright © 2015 John Wiley & Sons, Ltd.

Understanding the financial and intellectual value of campus-based cyberinfrastructure (CI) to the institutions that invest in such CI is intrinsically difficult. Given today's financial pressures, there is often administrative pressure questioning the value of campus-based and campus-funded CI resources. In this paper we describe new financial analytics capabilities being added to the widely used system analysis tool Open XDMoD (XSEDE Metrics on Demand) to create a new realm of metrics that will allow us to correlate usage of high performance computing with funding and publications. The capabilities to be added will eventually allow CI centers to view metrics relevant to both scientific output in terms of publications, and financial data in terms of awarded grants.The creation of Open XDMoD Value Analytics was funded by the National Science Foundation as a two year project. We are now nearing the end of the first year of this award, during which we focused on financial analytics. During the second year of this project we will focus on analytics of intellectual output. This module will allow the same sorts of analyses about systems and users as the financial analytics module, but in terms of intellectual outputs such as number of publications, citations to publications, and H indices. This module will also have capabilities to visualize such data, integrated with financial data. We plan to present these tools at PEARC '18.

Blue Waters [4] is a petascale-level supercomputer whose mission is to greatly accelerate insight to the most challenging computational and data analysis problems. We performed a detailed workload analysis of Blue Waters [8] using Open XDMoD [10]. The analysis used approximately 35,000 node hours to process the roughly 95 TB of input data from over 4.5M jobs that ran on Blue Waters during the period that was studied (April 1, 2013 - September 30, 2016).This paper describes the work that was done to collate, process and analyze the data that was collected on Blue Waters, the design decisions that were made, tools that we created and the various software engineering problems that we encountered and solved. In particular, we describe the challenges to data processing unique to Blue Waters engendered by the extremely large jobs that it typically executed.

The important role high-performance computing HPC resources play in science and engineering research, coupled with its high cost capital, power and manpower, short life and oversubscription, requires us to optimize its usage - an outcome that is only possible if adequate analytical data are collected and used to drive systems management at different granularities - job, application, user and system. This paper presents a method for comprehensive job, application and system-level resource use measurement, and analysis and its implementation. The steps in the method are system-wide collection of comprehensive resource use and performance statistics at the job and node levels in a uniform format across all resources, mapping and storage of the resultant job-wise data to a relational database, which enables further implementation and transformation of the data to the formats required by specific statistical and analytical algorithms. Analyses can be carried out at different levels of granularity: job, user, application or system-wide. Measurements are based on a new lightweight job-centric measurement tool 'TACC_Stats', which gathers a comprehensive set of resource use metrics on all compute nodes and data logged by the system scheduler. The data mapping and analysis tools are an extension of the XDMoD project. The method is illustrated with analyses of resource use for the Texas Advanced Computing Center's Lonestar4, Ranger and Stampede supercomputers and the HPC cluster at the Center for Computational Research. The illustrations are focused on resource use at the system, job and application levels and reveal many interesting insights into system usage patterns and also anomalous behavior due to failure/misuse. The method can be applied to any system that runs the TACC_Stats measurement tool and a tool to extract job execution environment data from the system scheduler. Copyright © 2014 John Wiley & Sons, Ltd.

Oligonucleotides are designed to target RNA using base pairing rules, however, they are hampered by poor cellular delivery and non-specific stimulation of the immune system. Small molecules are preferred as lead drugs or probes, but cannot be designed from sequence. Herein, we describe an approach termed Inforna that designs lead small molecules for RNA from solely sequence. Inforna was applied to all human microRNA precursors and identified bioactive small molecules that inhibit biogenesis by binding to nuclease processing sites (41% hit rate). Amongst 29 lead interactions, the most avid interaction is between a benzimidazole (1) and precursor microRNA-96. Compound 1 selectively inhibits biogenesis of microRNA-96, upregulating a protein target (FOXO1) and inducing apoptosis in cancer cells. Apoptosis is ablated when FOXO1 mRNA expression is knocked down by an siRNA, validating compound selectivity. Importantly, microRNA profiling shows that 1 only significantly effects microRNA-96 biogenesis and is more selective than an oligonucleotide.

In this investigation, we study how application performance is affected when jobs are permitted to share compute nodes. A series of application kernels consisting of a diverse set of benchmark calculations were run in both exclusive and node-sharing modes on the Center for Computational Research's high-performance computing (HPC) cluster. Very little increase in runtime was observed due to job contention among application kernel jobs run on shared nodes. The small differences in runtime were quantitatively modeled in order to characterize the resource contention and attempt to determine the circumstances under which it would or would not be important. A machine learning regression model applied to the runtime data successfully fitted the small differences between the exclusive and shared node runtime data; it also provided insight into the contention for node resources that occurs when jobs are allowed to share nodes. Analysis of a representative job mix shows that runtime of shared jobs is affected primarily by the memory subsystem, in particular by the reduction in the effective cache size due to sharing; this leads to higher utilization of DRAM. Insights such as these are crucial when formulating policies proposing node sharing as a mechanism for improving HPC utilization.

SUMMARY This paper describes XSEDE Metrics on Demand, a comprehensive auditing framework for use by high-performance computing centers, which provides metrics regarding resource utilization, resource performance, and impact on scholarship and research. This role-based framework is designed to meet the following objectives: (1) provide the user community with a tool to manage their allocations and optimize their resource utilization; (2) provide operational staff with the ability to monitor and tune resource performance; (3) provide management with a tool to monitor utilization, user base, and performance of resources; and (4) provide metrics to help measure scientific impact. Although initially focused on the XSEDE program, XSEDE Metrics on Demand can be adapted to any high-performance computing environment. The framework includes a computationally lightweight application kernel auditing system that utilizes performance kernels to measure overall system performance. This allows continuous resource auditing to measure all aspects of system performance including filesystem performance, processor and memory performance, and network latency and bandwidth. Metrics that focus on scientific impact, such as publications, citations and external funding, will be included to help quantify the important role high-performance computing centers play in advancing research and scholarship. Copyright © 2012 John Wiley & Sons, Ltd.

The REDfly database of Drosophila transcriptional cis-regulatory elements provides the broadest and most comprehensive available resource for experimentally validated cis-regulatory modules and transcription factor binding sites among the metazoa. The third major release of the database extends the utility of REDfly as a powerful tool for both computational and experimental studies of transcription regulation. REDfly v3.0 includes the introduction of new data classes to expand the types of regulatory elements annotated in the database along with a roughly 40% increase in the number of records. A completely redesigned interface improves access for casual and power users alike; among other features it now automatically provides graphical views of the genome, displays images of reporter gene expression and implements improved capabilities for database searching and results filtering. REDfly is freely accessible at http://redfly.ccr.buffalo.edu.

ORegAnno is an open-source, open-access database and literature curation system for community-based annotation of experimentally identified DNA regulatory regions, transcription factor binding sites and regulatory variants. The current release comprises 30 145 records curated from 922 publications and describing regulatory sequences for over 3853 genes and 465 transcription factors from 19 species. A new feature called the 'publication queue' allows users to input relevant papers from scientific literature as targets for annotation. The queue contains 4438 gene regulation papers entered by experts and another 54 351 identified by text-mining methods. Users can enter or 'check out' papers from the queue for manual curation using a series of user-friendly annotation pages. A typical record entry consists of species, sequence type, sequence, target gene, binding factor, experimental outcome and one or more lines of experimental evidence. An evidence ontology was developed to describe and categorize these experiments. Records are cross-referenced to Ensembl or Entrez gene identifiers, PubMed and dbSNP and can be visualized in the Ensembl or UCSC genome browsers. All data are freely available through search pages, XML data dumps or web services at: http://www.oreganno.org.

We present a framework that compares the publication impact based on a comprehensive peer analysis of papers produced by scientists using XSEDE and NCAR resources. The analysis is introducing a percentile ranking based approach of citations of the XSEDE and NCAR papers compared to peer publications in the same journal that do not use these resources. This analysis is unique in that it evaluates the impact of the two facilities by comparing the reported publications from them to their peers from within the same journal issue. From this analysis, we can see that papers that utilize XSEDE and NCAR resources are cited statistically significantly more often. Hence we find that reported publications indicate that XSEDE and NCAR resources exert a strong positive impact on scientific research.

Machine learning techniques were applied to job accounting and performance data for application classification. Job data were accumulated using the XDMoD monitoring technology named SUPReMM, they consist of job accounting information, application information from Lariat/XALT, and job performance data from TACC_Stats. The results clearly demonstrate that community applications have characteristic signatures which can be exploited for job classification. We conclude that machine learning can assist in classifying jobs of unknown application, in characterizing the job mixture, and in harnessing the variation in node and time dependence for further analysis.

The Technology Audit Service has developed, XDMoD, a resource management tool. This paper utilizes XDMoD and the XDMoD data warehouse that it draws from to provide a broad overview of several aspects of XSEDE users and their usage. Some important trends include: 1) in spite of a large yearly turnover, there is a core of users persisting over many years, 2) user job submission has changed from primarily faculty members to students and postdocs, 3) increases in usage in Molecular Biosciences and Materials Research has outstripped that of other fields of science, 4) the distribution of user external funding is bimodal with one group having a large ratio of external funding to internal XSEDE funding (ie, CPU cycles) and a second group having a small ratio of external to internal (CPU cycle) funding, 5) user job efficiency is also bimodal with a group of presumably new users running mainly small inefficient jobs and another group of users running larger more efficient jobs, 6) finally, based on an analysis of citations of published papers, the scientific impact of XSEDE coupled with the service providers is demonstrated in the statistically significant advantage it provides to the research of its users.