1. The Cancer Genomics Hub (CGHub): overcoming cancer through the power of torrential data
- Author
-
Jim Warner, Michael Ainsworth, David Haussler, Elizabeth Mitchell, Lori Maltbie, Hannes Schmidt, Patrick Allen, Daniel Maltbie, Linda Rosewood, Brian Craft, Christy Martin, Melissa S. Cline, John Black, Thomas Hatton, Christopher Wilks, Haifang Telc, John Groboske, Daniel Wilson, Donavan Nelson, Erich Weiler, Daniel Murphy, Howdy Pierce, Mark Diehkans, Brian Sanford, Brian Litzinger, and Bradley R. Smith
- Subjects
Computer science ,Genomics ,computer.software_genre ,General Biochemistry, Genetics and Molecular Biology ,Upload ,Databases ,Genetic ,Library and Information Studies ,Neoplasms ,Databases, Genetic ,Genetics ,Humans ,Electronic Health Records ,Confidentiality ,Computer Security ,Cancer ,Internet ,Database ,Application programming interface ,business.industry ,Cancer cell line encyclopedia ,Human Genome ,Performance results ,Data Format ,Computer data storage ,Database Management Systems ,Original Article ,The Internet ,General Agricultural and Biological Sciences ,business ,computer ,2.6 Resources and infrastructure (aetiology) ,Information Systems - Abstract
The Cancer Genomics Hub (CGHub) is the online repository of the sequencing programs of the National Cancer Institute (NCI), including The Cancer Genomics Atlas (TCGA), the Cancer Cell Line Encyclopedia (CCLE) and the Therapeutically Applicable Research to Generate Effective Treatments (TARGET) projects, with data from 25 different types of cancer. The CGHub currently contains >1.4 PB of data, has grown at an average rate of 50 TB a month and serves >100 TB per week. The architecture of CGHub is designed to support bulk searching and downloading through a Web-accessible application programming interface, enforce patient genome confidentiality in data storage and transmission and optimize for efficiency in access and transfer. In this article, we describe the design of these three components, present performance results for our transfer protocol, GeneTorrent, and finally report on the growth of the system in terms of data stored and transferred, including estimated limits on the current architecture. Our experienced-based estimates suggest that centralizing storage and computational resources is more efficient than wide distribution across many satellite labs. Database URL: https://cghub.ucsc.edu.
- Published
- 2014